Tape drives down, inventory failed, and robtest returning "read_element_status ioctl() failed: Success"

Problem

After performing tape drive cleaning via the vendor's Library Web GUI (graphical user interface), outside of NetBackup, viewing media in the library was not working, inventory was failing with status 228, drives went to a DOWN state, and robtest was returning an ioctl failed message.

Error Message

In the NetBackup Java Administration Console, robot inventory failed:

Requested slot does not exist in robot (228)

In command line, the robtest command returned a failure:

# /usr/openv/volmgr/bin/robtest
Configured robots with local control supporting test utilities:
  TLD(0)     robotic path = /dev/sg20
Robot Selection
---------------
  1)  TLD 0
  2)  none/quit
Enter choice: 1
Robot selected: TLD(0)   robotic path = /dev/sg20
Invoking robotic test utility:
/usr/openv/volmgr/bin/tldtest -rn 0 -r /dev/sg20
Opening /dev/sg20
MODE_SENSE complete
Enter tld commands (? returns help information)
s d
read_element_status ioctl() failed: Success
s s
read_element_status ioctl() failed: Success
...

Cause

Suspect tape drive cleaning in Library Web GUI caused a SCSI conflict issue with tape drives and NetBackup software, which resulted in tapes being stuck in the drives, some of the NetBackup Device Manager (ltid) services to stop, and prevented tape library queries.

To prevent the issue from happening...

• Avoid restarting the library while jobs are running to tape, and 
• Follow vendor documentation for cleaning drives via the library.

Solution

Recommend following these steps to resolved this issue:

1. Reset of Tape Library:
   • Shutdown the library.
   • Leave it down for 5+ minutes.
   • Start up the library.
   • Then, proceed to step 2.
2. Restart of Device Manager services:
   Note: Perform the steps below on the RCH (robotic control host) first and then on any additional media servers which are configured for tape storage on the tape library.
   • Check status of Volume Manger services: /usr/openv/volmgr/bin/vmps
   • Stop Device Manger services: /usr/openv/volmgr/bin/stopltid
   • Check Volume Manger services stopped: /usr/openv/volmgr/bin/vmps
   • Start Device Manger services: /usr/openv/volmgr/bin/ltid -v
   • Check Volume Manger services running: /usr/openv/volmgr/bin/vmps

Afterwards, in the NetBackup Java Administration Console...

• Inventory of the tape library should complete successfully.
• Set the tape drive(s) to an UP state, restart tape backup jobs, and monitor the jobs.
• Backups to tape storage should now utilize the tape drive(s) properly.