How to remove linkset/link object for a linked volume

Description

Background:

A linked volume is basically a mirror that is attached to the parent volume to be used for distance mirroring for Disaster Recover or Breakoff snapshots.

Linked-to volume: A volume that receives all the updates from another volume (which is linked-from volume). This volume basically acts as a mirror of the other volume, with some differences. It is not directly accessible to the applications. This is also referred to as a target volume, secondary volume, or a remote volume.

Linked-from volume: A volume that has another volume linked to it. It sends all the updates it receives to the linked volume. The linked-from volume is accessible to the application and acts as a primary data volume. This is also referred to as a source volume, primary volume, or a local volume.

Linkset (dglink) object: This object represents linking (mirroring) relationship between two disk groups. It also represents a group of volumes that is involved in linking relationship with another group.

Link object: This object represents a linking relationship between two volumes. This is sometimes referred to as volume link object.

Note: The linked-to volume and linked-from volume can be a part of different disk groups. This way, no DG split/join operations are required to do off-host processing. However both the volumes are accessible through the same host and the data is transferred from linked-from volume to the linked-to volume through normal I/O interface.


Issue:

As part of creating a linked volume the user aborted the vxsnap and was left with the following status :

# vxprint -g datadg -ht

v  datavol       -             ENABLED  ACTIVE   7129264128 ROUND    - fsgen
pl datavol-01    datavol       ENABLED  ACTIVE   7129264128 STRIPE 2/128  RW
sd datadg10-01 datavol-01 datadg10 0  1887305472 0/0   EMC_0 ENA
sd datadg14-01 datavol-01 datadg14 0  1677326592 0/1887305472 EMC_7 ENA
sd datadg09-01 datavol-01 datadg09 0  1887305472 1/0   EMC_3 ENA
sd datadg15-01 datavol-01 datadg15 0  1677326592 1/1887305472 EMC_9 ENA
pl datavol-02    datavol       ENABLED  ACTIVE   7129264128 STRIPE  2/128 RW
sd datadg01-01 datavol-02 datadg01 0  1887305472 0/0   EMC2_10 ENA
sd datadg03-01 datavol-02 datadg03 0  1677326592 0/1887305472 EMC2_12 ENA
sd datadg02-01 datavol-02 datadg02 0  1887305472 1/0   EMC2_11 ENA
sd datadg04-01 datavol-02 datadg04 0  1677326592 1/1887305472 EMC2_13 ENA
dc datavol_dco   datavol       datavol_dcl
v  datavol_dcl   -            ENABLED  ACTIVE   490256   ROUND     -      gen
pl datavol_dcl-01 datavol_dcl  ENABLED  ACTIVE   490256   CONCAT   -      RW
sd datadg11-01 datavol_dcl-01 datadg11 0 490256 0       EMC2_1 ENA
pl datavol_dcl-02 datavol_dcl  ENABLED  ACTIVE   490256   CONCAT   -      RW
sd datadg12-01 datavol_dcl-02 datadg12 0 490256 0       EMC_4 ENA
ln datavol_ln    datavol       datavol_dco
ln datavol_ln1   datavol       datavol_dco
ln datavol_ln2   datavol       datavol_dco  

There are a total of 3 links, all with same status and no are longer required. The last linkset=datadg-lset-ispbo-lf-02 still has a snapshot dg=snapdg and volume=snapvol and currently deported.

Reproduce:

A. Create datadg=ddg with volume=vol1 and snapdg=sdg with volume=snapvol1 ready for snapshot

# vxassist -g ddg make vol1 1433600s  logtype=dco drl=off dcoversion=20 regionsz=128 init=active

# vxassist -g ddg make snapvol1 1433600s logtype=dco drl=off dcoversion=20 regionsz=128 init=active


# vxprint -g ddg -htv vol1

v  vol1         -            ENABLED  ACTIVE   1433600  SELECT    -        fsgen
pl vol1-01      vol1         ENABLED  ACTIVE   1433600  CONCAT    -        RW
sd d1-01        vol1-01      d1       0        204800   0         HDS9500-ALUA0_0 ENA
sd d1-03        vol1-01      d1       205344   1228800  204800    HDS9500-ALUA0_0 ENA
dc vol1_dco     vol1         vol1_dcl
v  vol1_dcl     -            ENABLED  ACTIVE   560      SELECT    -        gen
pl vol1_dcl-01  vol1_dcl     ENABLED  ACTIVE   560      CONCAT    -        RW
sd d1-02        vol1_dcl-01  d1       3531296  560      0         HDS9500-ALUA0_0 ENA


# vxprint -g sdg -htv snapvol1

v  snapvol1     -            ENABLED  ACTIVE   1433600  SELECT    -        fsgen
pl snapvol1-01  snapvol1     ENABLED  ACTIVE   1433600  CONCAT    -        RW
sd s1-01        snapvol1-01  s1       0        1433600  0         HDS9500-ALUA0_2 ENA
dc snapvol1_dco snapvol1     snapvol1_dcl
v  snapvol1_dcl -            ENABLED  ACTIVE   544      SELECT    -        gen
pl snapvol1_dcl-01 snapvol1_dcl ENABLED ACTIVE 544      CONCAT    -        RW
sd s1-02        snapvol1_dcl-01 s1    1433600  544      0         HDS9500-ALUA0_2 ENA

B. Create filesystem and add data to the datadg /volume  and mount

# mkfs -F vxfs /dev/vx/rdsk/ddg/vol1
# mount -F vxfs /dev/vx/dsk/ddg/vol1 /d1
# cp /etc/a* /d1
# df -kl /d1
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/vx/dsk/ddg/vol1  716800   67513  608719    10%    /d1

C. Create Link between volumes

#  vxsnap -g ddg -b addmir vol1 mirvol=snapvol1 mirdg=sdg

While syncing  deport the sdg ( to simulate aborting of the above command)

# vxdg deport sdg

As a result the vxsnap aborts as follows:

#  vxsnap -g ddg addmir vol1 mirvol=snapvol1 mirdg=sdg
VxVM vxsnap ERROR V-5-1-0 Links synchronization stopped on the volume vol1 due to I/O error
VxVM vxsnap ERROR V-5-1-607 Diskgroup sdg not found
VxVM vxsnap WARNING V-5-1-11696 Initial synchronization of the link(s) failed. Start the links explicitly.
VxVM vxsnap ERROR V-5-1-607 Diskgroup sdg not found
VxVM vxsnap ERROR V-5-1-12182 Could not delete linkset for ISP breakoff snapshot
VxVM vxsnap ERROR V-5-1-12745 Unable to create mirror relationship between volume vol1 (dg ddg) and volume snapvol1 (dg sdg)

D. check status of the linkset and its similar to the system being recovered

# /sbin/vxlink -g ddg print
TY NAME       ROLE  VOLUME     LINKSET               LNSTATE    LSSTATE    DIFF
VxVM  ERROR V-5-1-12043  Sibling linkset of ddg-lset-ispbo-lf-00 not found.
ls ddg-lset-ispbo-lf-00 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of ddg-lset-ispbo-lf-00 not found.
ln vol1_ln    LF    vol1       ddg-lset-ispbo-lf-00  BROKEN     ACTIVE     N/A

E. Apply the  vxlink commands below to print,stop and delete the "ln" objects.

# /sbin/vxlink -g ddg volume print vol1
# /sbin/vxlink -g ddg volume stoplink vol1  linkset=ddg-lset-ispbo-lf-00
# /sbin/vxlink -g ddg linkset  delete ddg-lset-ispbo-lf-00 recurse=yes


Solution:

1. Check current link status

# /sbin/vxlink -g datadg volume print datavol
TY NAME       ROLE  VOLUME     LINKSET     LNSTATE    LSSTATE    DIFF
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-00 not found.
ls datadg-lset-ispbo-lf-00 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-00 not found.
ln datavol_ln  LF    datavol     datadg-lset-ispbo-lf-00  BROKEN     ACTIVE     N/A

VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-01 not found.
ls datadg-lset-ispbo-lf-01 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-01 not found.
ln datavol_ln1 LF    datavol     datadg-lset-ispbo-lf-01  BROKEN     ACTIVE     N/A

VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-02 not found.
ls datadg-lset-ispbo-lf-02 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-02 not found.
ln datavol_ln2 LF    datavol     datadg-lset-ispbo-lf-02  BROKEN     ACTIVE     N/A

2. Stop Link for all linksets involved

# /sbin/vxlink  -g datadg volume stoplink datavol  linkset=datadg-lset-ispbo-lf-02
# /sbin/vxlink  -g datadg volume stoplink datavol  linkset=datadg-lset-ispbo-lf-01
# /sbin/vxlink  -g datadg volume stoplink datavol  linkset=datadg-lset-ispbo-lf-00

3.Check status ( state changed from BROKEN to STOPPED)

# /sbin/vxlink -g datadg print

TY NAME       ROLE  VOLUME     LINKSET     LNSTATE    LSSTATE    DIFF
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-00 not found.
ls datadg-lset-ispbo-lf-00 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-00 not found.
ln datavol_ln  LF    datavol     datadg-lset-ispbo-lf-00  STOPPED      ACTIVE   N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-01 not found.

ls datadg-lset-ispbo-lf-01 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-01 not found.
ln datavol_ln1 LF    datavol     datadg-lset-ispbo-lf-01  STOPPED     ACTIVE    N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-02 not found.

ls datadg-lset-ispbo-lf-02 LF    --         --          --         ACTIVE     N/A
VxVM  ERROR V-5-1-12043  Sibling linkset of datadg-lset-ispbo-lf-02 not found.
ln datavol_ln2 LF    datavol     datadg-lset-ispbo-lf-02  STOPPED     ACTIVE    N/A

4. Delete linkset

# /sbin/vxlink  -g datadg linkset delete datadg-lset-ispbo-lf-02 recurse=yes
# /sbin/vxlink  -g datadg linkset delete datadg-lset-ispbo-lf-01 recurse=yes
# /sbin/vxlink  -g datadg linkset delete datadg-lset-ispbo-lf-00 recurse=yes

5. Recheck and there is no more links or linksets

# /sbin/vxlink -g datadg print
TY NAME       ROLE  VOLUME     LINKSET     LNSTATE    LSSTATE    DIFF

6. Finally check the volume

# vxprint -g datadg -htv datavol

v  datavol       -             ENABLED  ACTIVE   7129264128 ROUND   -      fsgen  
pl datavol-01    datavol       ENABLED  ACTIVE   7129264128 STRIPE  2/128  RW  
sd datadg10-01 datavol-01 datadg10 0  1887305472 0/0    EMC_0 ENA  
sd datadg14-01 datavol-01 datadg14 0  1677326592 0/1887305472  EMC_7 ENA  
sd datadg09-01 datavol-01 datadg09 0  1887305472 1/0    EMC_3 ENA  
sd datadg15-01 datavol-01 datadg15 0  1677326592 1/1887305472  EMC_9 ENA  
pl datavol-02    datavol       ENABLED  ACTIVE   7129264128 STRIPE  2/128  RW  
sd datadg01-01 datavol-02 datadg01 0  1887305472 0/0    EMC2_10 ENA  
sd datadg03-01 datavol-02 datadg03 0  1677326592 0/1887305472  EMC2_12 ENA  
sd datadg02-01 datavol-02 datadg02 0  1887305472 1/0    EMC2_11 ENA  
sd datadg04-01 datavol-02 datadg04 0  1677326592 1/1887305472  EMC2_13 ENA  
dc datavol_dco   datavol       datavol_dcl  
v  datavol_dcl   -             ENABLED  ACTIVE   490256   ROUND     -      gen  
pl datavol_dcl-01 datavol_dcl  ENABLED  ACTIVE   490256   CONCAT    -      RW  
sd datadg11-01 datavol_dcl-01 datadg11 0 490256 0       EMC2_1 ENA  
pl datavol_dcl-02 datavol_dcl  ENABLED  ACTIVE   490256   CONCAT    -      RW  
sd datadg12-01 datavol_dcl-02 datadg12 0 490256 0       EMC_4 ENA


Note: Please destroy the diskgroup=snapdg and volume=snapvol before redoing the complete link volume and breakoff snapshot process.