Multi-pathing does not work when ZFS pools are multi-pathed by Veritas DMP

Problem

ZFS is a type of file system presenting a pooled storage model developed by SUN (Oracle). File systems can directly draw from a common storage pool (zpool).

The Solaris Operating System (OS) can use two different types of disk labels: Sun Microsystems, Inc. (SMI) labels, and Extensible Firmware Interface (EFI) labels. Traditionally, LUNs which are larger than 1TB in size, are configured with an EFI label (instead of SMI labels). ZFS configured devices will default with an EFI disk label regardless of the physical LUN size.

Veritas Volume Manager (VxVM) in connection with Veritas DMP can be used to manage ZFS pools and any ZFS file system(s) that operate on those pools. If enabled, DMP can be used as the multi-pathing driver for the management of ZFS pools.

Veritas VCS uses the Zpool agent and enables the user to specify the faster /dev/vx/dmp/<da-name>/<slice-number> path details when importing the ZFS zpool. Taking microseconds to import rather than minutes in some cases.

Figure 1.0

The above sample Solaris environment consists of two Solaris Sparc (SOL) servers with several ZFS devices visible to both servers.

Following the Veritas InfoScale product installation, it is possible to configure DMP to manage all ZFS pools by enabling the dmp_native_support DMP tunable. When this tunable is turned on, DMP enables support for ZFS on any device that does not have a VxVM label and is not in control of any third party multi-pathing (TPD) software.

To verify the value of the dmp_native_support tunable, use the following command:

# vxdmpadm gettune dmp_native_support

Tunable                     Current Value   Default Value
--------------------------  -------------   ---------------
dmp_native_support          on              off

Enabling DMP native support

The dmp_native_support tunable can be enabled using the following command:

# vxdmpadm settune dmp_native_support=on

Issue:

Unless the ZFS zpool is explicitly imported using the /dev/vx/dmp path with the -d argument, the OS native device path can be used by defauit. This can result in Veritas DMP being unable to multi-path the ZFS device correctly, resulting in an unplanned outage where the imported path used to import the ZFS pool fails.

# zpool import -d /dev/vx/dmp/emc0_0142s0 TESTPOOL

Cause

Unless the ZFS zpool is explicitly imported using the full /dev/vx/dmp path with the -d argument, the OS native device path could be used.

This can result in Veritas DMP being unable to multi-path the ZFS device correctly.

Imagine losing access to the OS device path used to import the ZFS pool. As the DMP node was not used to import the ZFS zpool, the zpool is not managed by DMP and may not have any multi-pathing redundancy. This can result in an unplanned outage.

Solution

When ZFS devices are managed by the Veritas dmp_native_support DMP tunable, all ZFS zpools should be imported using the full blockdevice with zpool import -d option.

Example:
 

The ZFS zpool named "testpool" can be imported using the -d argument. By specifying the Veritas Disk Access (DA) name along with the slice, the ZFS import operation should be much faster and ensure DMP is managing multi-pathing functionality for the zpool.

# zpool import -d /dev/vx/dmp/emc0_0142s0 TESTPOOL

The imported ZFS pool details can be displayed using the "vxdmpadm native ls" command:

# vxdmpadm native ls

DMPNODENAME                      POOL NAME
=============================================
disk_0                           -
disk_1                           rpool
emc0_0142                        TESTPOOL
emc0_0143                        -
3pardata0_16                     -
3pardata0_17                     -

NOTE: When the zpool is exported, the DMP symbolic link path to the DMP device will be used during the next import operation, unless an alternate path is explicitly specified with the -d argument.
The product is designed this way to cater for situations where no import arguments are specified during the zpool import operation. Thus ensuring DMP is used for future imports whilst maintaining multi-pathing support for ZFS zpools.