Veritas InfoScale™ 7.3.1 Troubleshooting Guide - Solaris
- Introduction
- Section I. Troubleshooting Veritas File System
- Section II. Troubleshooting Veritas Volume Manager
- Recovering from hardware failure
- Failures on RAID-5 volumes
- Recovery from failure of a DCO volume
- Recovering from instant snapshot failure
- Recovering from failed vxresize operation
- Recovering from boot disk failure
- Hot-relocation and boot disk failure
- Recovery from boot failure
- Repair of root or /usr file systems on mirrored volumes
- Replacement of boot disks
- Recovery by reinstallation
- Managing commands, tasks, and transactions
- Backing up and restoring disk group configurations
- Troubleshooting issues with importing disk groups
- Recovering from CDS errors
- Logging and error messages
- Troubleshooting Veritas Volume Replicator
- Recovery from configuration errors
- Errors during an RLINK attach
- Errors during modification of an RVG
- Recovery on the Primary or Secondary
- Recovering from Primary data volume error
- Primary SRL volume error cleanup and restart
- Primary SRL header error cleanup and recovery
- Secondary data volume error cleanup and recovery
- Troubleshooting issues in cloud deployments
- Recovering from hardware failure
- Section III. Troubleshooting Dynamic Multi-Pathing
- Section IV. Troubleshooting Storage Foundation Cluster File System High Availability
- Troubleshooting Storage Foundation Cluster File System High Availability
- Troubleshooting CFS
- Troubleshooting fenced configurations
- Troubleshooting Cluster Volume Manager in Veritas InfoScale products clusters
- Troubleshooting Storage Foundation Cluster File System High Availability
- Section V. Troubleshooting Cluster Server
- Troubleshooting and recovery for VCS
- VCS message logging
- Gathering VCS information for support analysis
- Troubleshooting the VCS engine
- Troubleshooting Low Latency Transport (LLT)
- Troubleshooting Group Membership Services/Atomic Broadcast (GAB)
- Troubleshooting VCS startup
- Troubleshooting service groups
- Troubleshooting resources
- Troubleshooting I/O fencing
- System panics to prevent potential data corruption
- Fencing startup reports preexisting split-brain
- Troubleshooting CP server
- Troubleshooting server-based fencing on the Veritas InfoScale products cluster nodes
- Issues during online migration of coordination points
- Troubleshooting notification
- Troubleshooting and recovery for global clusters
- Troubleshooting licensing
- Licensing error messages
- VCS message logging
- Troubleshooting and recovery for VCS
- Section VI. Troubleshooting SFDB
Recovery from failure of a DCO volume
The procedure to recover from the failure of a data change object (DCO) volume depends on the DCO version number.
For information about DCO versioning, see the Storage Foundation Administrator's Guide.
Persistent FastResync uses a DCO volume to perform tracking of changed regions in a volume. If an error occurs while reading or writing a DCO volume, it is detached and the badlog flag is set on the DCO. All further writes to the volume are not tracked by the DCO.
The following sample output from the vxprint command shows a complete volume with a detached DCO volume (the TUTIL0 and PUTIL0 fields are omitted for clarity):
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE ... dg mydg mydg - - - - dm mydg01 c4t50d0s2 - 35521408 - - dm mydg02 c4t51d0s2 - 35521408 - - dm mydg03 c4t52d0s2 - 35521408 - FAILING dm mydg04 c4t53d0s2 - 35521408 - FAILING dm mydg05 c4t54d0s2 - 35521408 - - v SNAP-vol1 fsgen ENABLED 204800 - ACTIVE pl vol1-03 SNAP-vol1 ENABLED 204800 - ACTIVE sd mydg05-01 vol1-03 ENABLED 204800 0 - dc SNAP-vol1_dco SNAP-vol1 - - - - v SNAP-vol1_dcl gen ENABLED 144 - ACTIVE pl vol1_dcl-03 SNAP-vol1_dcl ENABLED 144 - ACTIVE sd mydg05-02 vol1_dcl-03 ENABLED 144 0 - sp vol1_snp SNAP-vol1 - - - - v vol1 fsgen ENABLED 204800 - ACTIVE pl vol1-01 vol1 ENABLED 204800 - ACTIVE sd mydg01-01 vol1-01 ENABLED 204800 0 - pl vol1-02 vol1 ENABLED 204800 - ACTIVE sd mydg02-01 vol1-01 ENABLED 204800 0 - dc vol1_dco vol1 - - - BADLOG v vol1_dcl gen DETACHED 144 - DETACH pl vol1_dcl-01 vol1_dcl ENABLED 144 - ACTIVE sd mydg03-01 vol1_dcl-01 ENABLED 144 0 - pl vol1_dcl-02 vol1_dcl DETACHED 144 - IOFAIL sd mydg04-01 vol1_dcl-02 ENABLED 144 0 RELOCATE sp SNAP-vol1_snp vol1 - - - -
This output shows the mirrored volume, vol1, its snapshot volume, SNAP-vol1, and their respective DCOs, vol1_dco and SNAP-vol1_dco. The two disks, mydg03 and mydg04, that hold the DCO plexes for the DCO volume, vol1_dcl, of vol1 have failed. As a result, the DCO volume, vol1_dcl, of the volume, vol1, has been detached and the state of vol1_dco has been set to BADLOG. For future reference, note the entries for the snap objects, vol1_snp and SNAP-vol1_snp, that point to vol1 and SNAP-vol1 respectively.
You can use such output to deduce the name of a volume's DCO (in this example, vol1_dco), or you can use the following vxprint command to display the name of a volume's DCO:
# vxprint [-g diskgroup] -F%dco_name volume
You can use the vxprint command to check if the badlog flag is set for the DCO of a volume as shown here:
# vxprint [-g diskgroup] -F%badlog dco_name
For example:
# vxprint -g mydg -F%badlog vol1_dco on
In this example, the command returns the value on, indicating that the badlog flag is set.
Use the following command to verify the version number of the DCO:
# vxprint [-g diskgroup] -F%version dco_name
For example:
# vxprint -g mydg -F%version vol1_dco
The command returns a value of 0, 20, or 30. The DCO version number determines the recovery procedure that you should use.