How to avoid reclaiming thin space with Veritas Volume Manager (VxVM)

Description

The following procedure can be used to avoid reclaiming space at the Veritas Volume Manager (VxVM) layer.
This is normally required when a large amount of thin space is to be reclaimed at both the VxVM and storage layer.

The reclaim process can potentially take multiple hours to reclaim large quantities of thin space.
 

In this example, the thin disk "emc0_629" will be assigned to disk group "testdg":

# vxprint -qhtg testdg
dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -

# vxdg -g testdg adddisk emc0_0629

# vxdisk -o thin list
DEVICE          SIZE(MB)     PHYS_ALLOC(MB)  GROUP           TYPE                  RECLAIM_CMD
emc0_062a       20480        19822           vvr             thinrclm                         WRITE_SAME
emc0_062b       20480        4576            vvr             thinrclm                         WRITE_SAME
emc0_062c       20480        32              vvr             thinrclm                         WRITE_SAME
emc0_0627       20480        32              vvr             thinrclm                         WRITE_SAME
emc0_0628       20480        10272           vvr             thinrclm                         WRITE_SAME
emc0_0629       20480        207             testdg          thinrclm                         WRITE_SAME
emc0_0634       20480        206             testdg          thinrclm                         WRITE_SAME
emc0_0635       20480        32              testdg          thinrclm                         WRITE_SAME

The volume "vol1" is created and a file system created on top of the VxVM volume.
 

# vxassist -g testdg make vol1 10g

VxFS example:
 

Create VxFS file system on top of volume "vol1" residing in Disk Group "testdg"
 

# mkfs -F vxfs /dev/vx/rdsk/testdg/vol1
    version 15 layout
    20971520 sectors, 10485760 blocks of size 1024, log size 16384 blocks
    rcq size 1024 blocks
    largefiles supported
    maxlink supported
    WORM not supported


Solaris Example:

The file system is then mounted and a file is then copied to the mount point:
 

# mount -F vxfs /dev/vx/dsk/testdg/vol1 /leg

# cp /etc/hosts /leg

The mount point is then unmounted
 

# umount /leg

# vxprint -qhtzg testdg
dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0629    emc0_0629    auto     65536    41870848 -
dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -

v  vol1         -            ENABLED  ACTIVE   20971520 SELECT    -        fsgen
pl vol1-01      vol1         ENABLED  ACTIVE   20971520 CONCAT    -        RW
sd emc0_0629-01 vol1-01      emc0_0629 0       20971520 0         emc0_0629 ENA

To trigger the thin reclaim event, delete the volume "vol1" which requires the thin space to be reclaimed

# vxedit -g testdg -rf rm vol1

NOTE: The RECLAIM/RCLM flag will be hidden unless the -z option is used with the vxprint command.

# vxprint -qhtzg testdg
dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0629    emc0_0629    auto     65536    41870848 -
dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -
sd emc0_0629-01 -            emc0_0629 0       20971520 -         emc0_0629 RCLM

Note: The disk cannot be removed from the disk group until the thin space has been reclaimed.

# vxdg -g testdg rmdisk emc0_0629
VxVM vxdg ERROR V-5-1-16414 Disk emc0_0629 is used by one or more subdisks which
are pending to be reclaimed.
        Use "vxdisk reclaim emc0_0629" to reclaim space used by these subdisks,
        and retry "vxdg rmdisk" command.
        Note: reclamation is irreversible.

NOTE: The subdisk(s) reported in a RECLAIM/RCLM state cannot be removed at this time.

# vxedit -g testdg -rf rm emc0_0629-01

The vxprint output still reflects the subdisk to be reclaimed:

# vxprint -qhtzg testdg
dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0629    emc0_0629    auto     65536    41870848 -
dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -
sd emc0_0629-01 -            emc0_0629 0       20971520 -         emc0_0629 RCLM

Procedure

__________________________________________________________________________________________

To bypass the VxVM thin reclaim operation, the disk can be marked from removal as follows:

1. Turn off the reclaim attribute at the disk level

# vxdisk set emc0_0629 reclaim=off
VxVM vxdisk ERROR V-5-1-18536 Disk emc0_0629 is used by one or more subdisks which
are pending to be reclaimed.
        Use "vxdisk reclaim emc0_0629" to reclaim space used by these subdisks,
        and retry the command.
        Note: reclamation is irreversible.

2. Mark the disk for removal using the -k option

# vxdg -g testdg -k rmdisk emc0_0629

# vxprint -qhtzg testdg
dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0629    -            -        -        -        REMOVED
dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -
sd emc0_0629-01 -            emc0_0629 0       20971520 -         emc0_0629 RCLM

3. Remove the subdisk(s) that still require the thin space to be reclaimed

# vxedit -g testdg -rf rm emc0_0629-01

4. Once the subdisk in the RECLAIM/RCLM state has been removed, the disk can then be removed from the disk group
 

# vxprint -qhtzg testdg

dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0629    -            -        -        -        REMOVED
dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -

# vxdg -g testdg rmdisk emc0_0629

5. The vxprint will now reflect that the disk has been removed from the disk group 

# vxprint -qhtzg testdg

dg testdg       default      default  3000     1570523941.26.server101

dm emc0_0634    emc0_0634    auto     65536    41870848 -
dm emc0_0635    emc0_0635    auto     65536    41870848 -

6. The thin space can now be reclaimed at the storage layer

# vxdisk -o thin list
DEVICE          SIZE(MB)     PHYS_ALLOC(MB)  GROUP           TYPE                  RECLAIM_CMD
emc0_062a       20480        19822           vvr             thinrclm                         WRITE_SAME
emc0_062b       20480        4576            vvr             thinrclm                         WRITE_SAME
emc0_062c       20480        32              vvr             thinrclm                         WRITE_SAME
emc0_0627       20480        32              vvr             thinrclm                         WRITE_SAME
emc0_0628       20480        10272           vvr             thinrclm                         WRITE_SAME
emc0_0629       20480        222             -               thinrclm                         WRITE_SAME
emc0_0634       20480        206             testdg          thinrclm                         WRITE_SAME
emc0_0635       20480        32              testdg          thinrclm                         WRITE_SAME