NetBackup Appliance upgrade to 5.3 failing due to "Cannot deploy the host ID-based certificate on the appliance." while getting certificates after Precheck tests.

Problem

During an upgrade of an appliance to version 5.3, using the VRTS_NBAPP_update-5.3-1.x86_64.rpm, customers may experience seeing an error stating "Cannot deploy the host ID-based certificate on the appliance" during  the step where the upgrade retrieves the certificates.

This issue is relevant to media server NetBackup Appliances, running all supported NetBackup Appliance and NetBackup HA Appliance hardware.

Error Message

Example of the error printed on the screen:

>> Do you want to trust the CA certificate? [yes, no](yes) yes
An unknown error has occurred. Contact Veritas Support to resolve the issue.
Cannot deploy the host ID-based certificate on the appliance
- [Info] The software upgrade has been aborted!

The nbapp_seccom log will show:

2024-03-04 15:00:36.365442
try to deploy host-id certificate without token
2024-03-04 15:00:36.365497
[exec_sys_cmd] /inst/patch/incoming/nbcertcmdtool/NB_10.3_0062/linuxR_x86_3.10.0/nbcertcmdtool -atLibPath /inst/patch/incoming/nbcertcmdtool/NB_10.3_0062/linuxR_x86_3.10.0 -getCertificate
2024-03-04 15:00:36.427982
{
  "Command": "/inst/patch/incoming/nbcertcmdtool/NB_10.3_0062/linuxR_x86_3.10.0/nbcertcmdtool -atLibPath /inst/patch/incoming/nbcertcmdtool/NB_10.3_0062/linuxR_x86_3.10.0 -getCertificate",
  "stdout": "Host certificate and certificate revocation list already exist for master server [any-master]\nUse the -force option to overwrite the existing certificate.\n",
  "stderr": "double free or corruption (fasttop)\n",
  "returncode": -6
}
2024-03-04 15:00:36.428430
token_message=An unknown error has occurred. Contact Veritas Support to resolve the issue.

Review of the nbcert logs ( /log/netbackup/nbcert ) will show the following:

14:45:48.554 [133691.133691] <2> validateAndUpdateMasterHostID: Checking if master hostId is present for master server [any-master]
14:45:48.554 [133691.133691] <2> DoesMasterHostIDExist: Checking if Master hostID exist any-master
14:45:48.555 [133691.133691] <2> DoesMasterHostIDExist: Master hostID exists for master [any-master] as Host ID : [12345678901234567890123]
14:45:48.555 [133691.133691] <2> CrlUtil::getX509Crl: CRL is not in DER format
14:45:48.555 [133691.133691] <2> CrlUtil::getX509Crl: CRL is in PEM format
14:45:48.555 [133691.133691] <4> GetHostCertificate: Host certificate and CRL already exist for master server [any-master]
14:45:48.555 [133691.133691] <2> LoginWithCertManager::deleteWebToken: Deleting jwt against key tokenId: [any-master.1.nbsvc-webtoken.dat] from credcache with force delete:0
14:45:48.555 [133691.133691] <2> io_send: About to send data..
14:45:48.564 [133691.133691] <16> credential_fromwire: Memory allocation for CredentialBlob of size 7588901093796112719 failed.

Cause

The nbcertcmdtool, used to retrieve the NetBackup certificate, is updated in NetBackup version 10.3 (NetBackup Appliance version 5.3). As a cache data structure changes, it malfunctions when running on old NetBackup versions.

Solution

This issue is scheduled to be fixed in the 5.3.0.1 MR1 release (due for release in late March 2024).

Workaround:

1. Launch the upgrade as normal, monitoring closely until the prompt shows the message:

“If an error occurs during the upgrade, do you want to immediately enforce an automatic rollback? [yes, no] (yes)”

2. At this stage, do not proceed, instead open a second SSH connection to the server.
3. From this second connection, stop NetBackup services from the NetBackup Appliance Shell Menu:

Main > Support > Processes NetBackup Stop

4. Once you ensure all NetBackup processes have stopped, return to the upgrade prompt and continue.  To confirm processes stopped, run the following command from the NetBackup Appliance Shell Menu:

Main > Support > Processes NetBackup Show

5. When you see the message, "Starting the upgrade. Use the Manage > Software > Upgrade Status command to monitor the progress.", start the NetBackup services from the NetBackup Appliance Shell Menu:: 

Main > Support > Processes> NetBackup Start