Veritas Volume Manager (VxVM) cannot work with Linux Device Mapper Multipathing (DM-Multipath) managed devices

Problem

Veritas does not support Veritas Dynamic Multi-pathing & Linux Device Mapper Multi-pathing ( DM-Multipath) supporting the same devices.

Only a single multi-pathing solution can be responsible for managing the multi-pathing functionality of a device.

Linux multi-pathing can be used to manage OS related devices as long as they are blacklisted & excluded correctly.
The steps are outlined in the "Solution" section of this article.
 

Veritas Volume Manager (VxVM) objects (i.e. VxVM volumes) cannot work (not supported) with Linux Device Mapper Multi-pathing ( DM-Multipath) managed devices.

VxVM can only work with DMP and Third-Party Drivers (TPDs) such as EMC PowerPath (with EMC storage).


Physical Environments

Whilst the internal boot device can be managed by Linux Device Mapper Multi-pathing ( DM-Multipath), the VxVM configured SAN data disks must be managed by Veritas DMP or an applicable TPD driver. 
Veritas Volume Manager does not support Linux Device Mapper Multi-pathing ( DM-Multipath) managing VxVM disks.


Virtual Environments

In relation to Virtual environments, a SAN provisioned boot device will be considered as an internal boot device, thus supported in connection with Linux Device Mapper Multi-pathing (DM-Multipath). The SAN provisioned VxVM data disks must be managed by DMP and not Linux Device Mapper Multi-pathing ( DM-Multipath).
 

Cause
 

Veritas does not test nor qualify Linux Multi-pathing in connection with the Storage Foundation (SF) product stack.

There is no ASL to support Linux Multi-pathing (DM-Multipath) managed devices.

- SCSI-3 Persistent Reservation Fencing is NOT supported when Linux OS native Multi-pathing (DM-Multipath) is operating beneath DMP

- SCSI-3 reservations created outside the Veritas vxfen driver are NOT supported

- The Veritas Thin Reclamation feature is NOT supported with Linux OS native Multi-pathing (DM-Multipath) managed devices
 

Solution
 

When using Veritas Volume Manger (VxVM) in connection with Linux Multi-pathing (DM-Multipath), you may need to blacklist OS managed devices.

Veritas Dynamic Multi-pathing (DMP) and Linux Multi-pathing (DM-Multipath) cannot co-exist without implementing specific "blacklist" & VxVM device exclusion restrictions.
 

Blacklisting guidance

The World Wide Identifier (WWID) can be used to reliably identify devices in Linux environments. The WWID is a persistent system independent ID for all SCSI devices.

It is strongly recommended to blacklist all disks and add blacklist_exceptions for specific WWIDs that will be managed solely by Linux Multi-pathing and not Veritas DMP.

STEPS
 

1.] Identify the WWID for the OS managed disk to be controlled by Linux Multi-pathing

This will be configured as a blacklist_exception in the /etc/multipath.conf file
 

The WWID can be obtained for a disk by running the following command:

Example:

# /lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdc
3600508b1001cbd89313f1d9cdd60dfa6

The above WWID "3600508b1001cbd89313f1d9cdd60dfa6" would then be populated in the /etc/multipath.conf.
Additional WWIDs can be added as shown in the example below.
 

2.] Edit the /etc/multipath.conf and add specific WWIDs for the OS configured devices

# cat /etc/multipath.conf
defaults {
        find_multipaths yes
        user_friendly_names yes
}

blacklist {
wwid ".*"
}

blacklist_exceptions {
           wwid "3600508b1001cbd89313f1d9cdd60dfa6"
           wwid "3600508b1001cbd89353f1d9cdd60dfa6"
}


3.]  Restart the multipathd service

# systemctl reload multipathd.service

Verify Linux multi-pathing detects the device

# multipath -ll

4.] Exclude the DMPNODE which is now under Linux Multi-pathing control, so the two multi-pathing solution do not conflict with each other.
 

VxVM exclusion

Identify the DMPNODE name for the OS device to be managed by Linux Multi-pathing and exclude it from VxVM & DMP.

The corresponding DMPNODE for the sdc must be excluded from VxVM's view, by using:

# vxdisk -eo alldgs list | grep sdc
fred_hpe_sata1_0 auto:LVM       -            -           LVM                  sdc              ssd

# vxdmpadm exclude dmpnodename=fred_hpe_sata1_0

The above "exclude" operation will populate the /etc/vx/vxvm.exclude file. This ensures the DMPNODE is consistently excluded from VxVM's view and avoids DMP probing the Linux multi-pathed device.


5.] Rescan and refresh the complete I/O stack

# /usr/lib/vxvm/voladm.d/bin/dmpdr -o refresh

 

The above process ensures a single multi-pathing driver is responsible for those devices.

Sample configurations

Figure 1.0:  LVM and VxVM - mixed multi-pathing solutions



Figure 2.0:  LVM using mixed multi-pathing solutions




Figure 3.0: LVM using single multi-pathing solution, LInux DM-Multi-pathing




Figure 4.0: LVM using single multi-pathing solution, Linux DM-Multi-pathing




Figure 5.0: Single multi-pathing solution, LVM SAN bootdisk and SAN data disks both managed by DMP




DMP native support must be enabled to correctly manage the LVM (OS) devices using Veritas DMP.


Figure 6.0: Mixed multi-pathing solution, LVM SAN bootdisk managed by Linux DM-Multi-pathing and SAN data disks managed by DMP



If a single enclosure consists of an LVM SAN bootdisk and VxVM SAN data disks, the only limitation is with regards to Linux LVM trying to work with both mulit-pathing solutions for the same device at the same time.

Blacklisting and VxVM device exclusion is required here.



DMP NATIVE SUPPORT

Veritas Storage Foundation 5.1.x onwards, provides DMP native support to manage Linux LVM (Logical Volume Manager) configured devices in connection with DMP

To enable Veritas DMP native support:

# vxdmpadm settune dmp_native_support=on
 

For DMP to control Linux LVM volumes, the Linux Multi-pathing (DM-Multipath) related devices must be blacklisted with DMP native support turned on.


By default, the /etc/multipath.conf file has all devices blacklisted. If the default configuration has been changed, it can be reverted back by doing the following:

# Blacklist all devices by default. Remove this to enable multipathing
# on the default devices.
blacklist {
devnode "*"
}

Note: The above is only required if all disk(s)/LUN(s) presented to server are NOT be managed by Linux Multi-pathing.

The administrator can specifically black list specific device classes, i.e. only VxVM and VxDMP managed devices from DM-Multipath control.

For further details, please consult the Linux administrator guide for more information on black listing specific devices from Linux DM-Multipath.

Applies To

VxDMP (VERITAS Dymanic Multi-pathing)

DM-Multipath (Device Mapper Multipathing)