How to identify and correct certificate and CRL related errors in NetBackup 8.1 and higher

Article: 100039941
Last Published: 2017-09-15
Ratings: 3 0
Product(s): NetBackup & Alta Data Protection

Problem

Corruption of files can occur due to unexpected system events, such as the sudden shutdown or reboot of the system, hardware errors, such as a faulty disks, or software/operating system type errors triggering incorrect writes of data to disk.  This article will explain how to identify and correct certificate or Certificate Revocation List (CRL) related issues in NetBackup 8.1 and higher to include the following issues:

  1.  A corrupted certmapinfo.json file
    • Resulting in status codes/errors include:  13, 5949, 7660
  2.  A corrupted certificate authority (CA) certificate
    • Status codes/errors:  hung jobs, offline media servers, 13, 7625, 7660
  3.  A corrupted local host ID-based certificate.
    • Status codes/errors: 25, 7625, 5942
  4.  A corrupt certificate revocation list (CRL) 
    • Status codes/errors: 23, 25, 7640, 7654, 9301

Error Message

Error codes, status codes and messages that may be experienced include, but are not limited to the following:
 
Error Message
13 File read failed
23 Socket read failed.
25 Cannot connect on socket.
61 The vnetd proxy encountered an error
5949 Certificate does not exist.
5978 Unable to read the certificate mapping file.
5942 Certificate could not be read from the local certificate store
7624 SSL accept failed.
7625 SSL socket connect failed
7627 PEM_X509_INFO_read_bio failed
7640 The peer closed the connection
7660 The peer host certificate cannot be verified using the Certificate Revocation List.
9301 Failed to decode certificate revocation list
N/A Backups hung waiting for resources.
 

Cause

Best Practices
Certificate, CRL and certmapinfo.json file problems are easiest to identify on the client, or media server reporting the issue.

Log files that should be enabled on the server reporting the issue, when troubleshooting the error codes listed above when they are related to certificate or CRL type issues are:

Log file directory VERBOSE or Debug Level OID
*nbpxyhelper DebugLevel=4 486
nbcert VERBOSE = 5  
bpcd VERBOSE = 5  
bprd (master server only) VERBOSE = 5  

 
* Unix/Linux operating systems, this is located in /usr/openv/logs.  All other log directories referenced are located in netbackup/logs for both Unix/Linux and Windows.

**Note that the DiagnosticLevel should always be set to 6.

The commands, nbcertcmd, bptestbpcd and bpclntcmd, are useful in troubleshooting certificate and CRL related issues.  Common syntax of the commands utilized in troubleshooting are:

bpclntcmd -pn
bpclntcmd -hn <hostname>
bptestbpcd -host [host]
nbcertcmd -getCRL
nbcertcmd -getCertificate
nbcertcmd -getCertificate -force
nbcertcmd -getCertificate -token <reissue_token> -force
nbcertcmd -getCACertificate
nbcertcmd -hostselfcheck
nbcertcmd -listCACertDetails
nbcertcmd -listCertDetails

The commands are all located in:
Windows<install_path>\Veritas\NetBackup\bin
Unix/Linux/usr/openv/netbackup/bin

For additional information, see the NetBackup 8.1 Commands Reference Guide.

Solution

Select the following links to be directed identification and solution of each issue:
Issue 1:  A corrupted certmapinfo.json file
Issue 2: A corrupted certificate authority (CA) certificate
Issue 3: A corrupted local host ID-based certificate
Issue 4 : A corrupt certificate revocation list (CRL)

Was this content helpful?