MSDP disk VOLUME down, backups fail with STATUS 2074 ( remote server in storage server Replication properties )

Problem

Unable to perform backups to source MSDP disk pool, and replications are failing with status 2074.

Example command output:
Disk Pool Name      : PDPool01
Disk Type           : PureDisk
Disk Volume Name    : PDVolume01
Disk Media ID       : @aaaaa
Total Capacity (GB) : 40.00
Free Space (GB)     : 35.45
Use%                : 11
Status              : DOWN
Flag                : ReadOnWrite
Flag                : AdminDown
Flag                : InternalUp
Num Read Mounts     : 0
Num Write Mounts    : 1
Cur Read Streams    : 0
Cur Write Streams   : 0

Error Message

Status Code:

Disk volume is down (2074)

Unified nbrmms log (OID:222) with DPS (OID:220) logging enabled:

11/25/11 20:26:39.712 [Application] NB 51216 dps 220 PID:6404 TID:6768 File ID:222 [No context] [Warning] V-220-62 volume is DOWN due to storage server related error: PDPool01:PDVolume01 E{gc-254} S{gc-3}

Cause

The Disk Pool and Storage Server are UP, but the Disk Volume is DOWN. This can be caused by another server associated with the Disk Pool, which is currently unavailable.

NetBackup can have more than one server associated with it. This is called a load balancing media server and it is a media server either hosting the storage server or has the ability to use the disk pool, as a remote media server or a replication media server.

If the disk volume only remains in an UP state for a few seconds, verify that all servers associated with the storage server are accessible. To test this, remove the other servers from the disk pool configurations, restart the local services (especially nbrmms), and attempt to UP the disk volume back online.

All servers associated with the disk pool must be able to communicate and successfully resolve hostnames and IP addresses. Any communication issues between the servers and the disk pool host server can result in the disk volume being unavailable.

Solution

Confirm Disk Pool Status is UP and Disk Volume Status is UP.

Command syntax:

1. nbdevquery -listdp -stype server_type -U
2. nbdevquery -listdv -stype server_type -dp disk_pool_name -U
3. nbdevconfig -changestate -stype server_type -dp disk_pool_name -dv disk_volume_name -state [UP | RESET]

Command paths:

• UNIX/Linux: /usr/openv/netbackup/bin/admincmd/
• Windows: install_path\NetBackup\bin\admincmd\

Example commands:

1. nbdevquery -listdp -stype PureDisk -U
2. nbdevquery -listdv -stype PureDisk -dp PDPool01 -U
3. nbdevconfig -changestate -stype PureDisk -dp PDPool01 -dv PDVolume01 -state RESET

Example Outputs:

1. nbdevquery -listdp -stype PureDisk -U
   • Disk Pool Name   : PDPool01
Disk Pool Id     : PDPool01
Disk Type        : PureDisk
Status           : UP
Flag             : Patchwork
Flag             : Visible
Flag             : OpenStorage
Flag             : SingleStorageServer
Flag             : CopyExtents
Flag             : AdminUp
Flag             : InternalUp
Flag             : LifeCycle
Flag             : CapacityMgmt
Flag             : FragmentImages
Flag             : Cpr
Flag             : FT-Transfer
Flag             : OptimizedImage
Flag             : ReplicationSource
Flag             : ReplicationTarget
Flag             : TargetedReplication
Raw Size (GB)    : 17.62
Usable Size (GB) : 17.62
High Watermark   : 98
Low Watermark    : 80
Num Volumes      : 1
Max IO Streams   : -1
Comment          :
Storage Server   : storage_name (UP)
2. nbdevquery -listdv -stype PureDisk -dp PDPool01 -U
   • Disk Pool Name             : PDPool01
Disk Type                  : PureDisk
Disk Volume Name           : PureDiskVolume
Disk Media ID              : @aaaad
Total Capacity (GB)        : 17.62
Free Space (GB)            : 17.32
Use%                       : 1
Status                     : UP
Flag                       : ReadOnWrite
Flag                       : AdminUp
Flag                       : InternalUp
Flag                       : ReplicationSource
Flag                       : ReplicationTarget
Num Read Mounts            : 0
Num Write Mounts           : 1
Cur Read Streams           : 0
Cur Write Streams          : 0
Num Repl Sources           : 0
Num Repl Targets           : 1
Replication Target  : media_server:PureDiskVolume
WORM Lock Min Time         : 0
WORM Lock Max Time         : 0
3. nbdevconfig -changestate -stype PureDisk -dp PDPool01 -dv PDVolume01 -state RESET
   • successfully changed the state of disk volume

Note: Pre-step 1 nbdevquery: A web login is required.

• Run the command to login: bpnbat -login -loginType [WEB|WEBUI|APIKEY]