NetBackup™ Troubleshooting Guide

Last Published:
Product(s): NetBackup & Alta Data Protection (10.5)
  1. Introduction
    1.  
      Additional resources on NetBackup logging and status code information
    2.  
      Troubleshooting a problem
    3.  
      Problem report for Technical Support
    4.  
      About gathering information for NetBackup-Java applications
  2. Troubleshooting procedures
    1.  
      About troubleshooting procedures
    2. Troubleshooting NetBackup problems
      1.  
        Verifying that all processes are running on UNIX or Linux servers
      2.  
        Verifying that all processes are running on Windows servers
    3.  
      Troubleshooting installation problems
    4.  
      Troubleshooting configuration problems
    5.  
      Device configuration problem resolution
    6.  
      Testing the primary server and clients
    7.  
      Testing the media server and clients
    8.  
      Resolving network communication problems with UNIX clients
    9.  
      Resolving network communication problems with Windows clients
    10. Troubleshooting vnetd proxy connections
      1.  
        vnetd proxy connection requirements
      2.  
        Where to begin to troubleshoot vnetd proxy connections
      3.  
        Verify that the vnetd process and proxies are active
      4.  
        Verify that the host connections are proxied
      5.  
        Test the vnetd proxy connections
      6.  
        Examine the log files of the connecting and accepting processes
      7.  
        Viewing the vnetd proxy log files
    11. Troubleshooting security certificate revocation
      1.  
        Troubleshooting cloud provider's revoked SSL certificate issues
      2.  
        Troubleshooting cloud provider's CRL download issues
      3.  
        How a host's CRL affects certificate revocation troubleshooting
      4.  
        NetBackup job fails because of revoked certificate or unavailability of CRLs
      5.  
        NetBackup job fails because of apparent network error
      6.  
        NetBackup job fails because of unavailable resource
      7.  
        Primary server security certificate is revoked
      8.  
        Determining a NetBackup host's certificate state
      9.  
        Troubleshooting issues with external CA-signed certificate revocation
    12.  
      About troubleshooting networks and host names
    13. Verifying host name and service entries in NetBackup
      1.  
        Example of host name and service entries on UNIX primary server and client
      2.  
        Example of host name and service entries on UNIX primary server and media server
      3.  
        Example of host name and service entries on UNIX PC clients
      4.  
        Example of host name and service entries on UNIX server that connects to multiple networks
    14.  
      About the bpclntcmd utility
    15.  
      Using the Host properties to access configuration settings
    16.  
      Resolving full disk problems
    17. Frozen media troubleshooting considerations
      1.  
        Logs for troubleshooting frozen media
      2.  
        About the conditions that cause media to freeze
    18. Troubleshooting problems with the NetBackup web services
      1.  
        Viewing NetBackup web services logs
      2.  
        Troubleshooting web service issues after external CA configuration
    19.  
      Troubleshooting problems with the NetBackup web server certificate
    20. Resolving PBX problems
      1.  
        Checking PBX installation
      2.  
        Checking that PBX is running
      3.  
        Checking that PBX is set correctly
      4.  
        Accessing the PBX logs
      5.  
        Troubleshooting PBX security
      6.  
        Determining if the PBX daemon or service is available
    21. Troubleshooting problems with validation of the remote host
      1.  
        Viewing logs pertaining to host validation
      2.  
        Enabling insecure communication with NetBackup 8.0 and earlier hosts
      3.  
        Approving pending host ID-to-host name mappings
      4.  
        Clearing host cache
    22. Troubleshooting Auto Image Replication
      1.  
        Rules for primary servers used with Auto Image Replication and SLPs
      2. Targeted A.I.R. trusted primary server operation failed in case of external certificate configuration
        1.  
          Add or update trust
        2.  
          Remove trust
      3.  
        About troubleshooting automatic import jobs that SLP components manage
    23.  
      Troubleshooting network interface card performance
    24.  
      About SERVER entries in the bp.conf file
    25.  
      About unavailable storage unit problems
    26.  
      Resolving a NetBackup Administration operations failure on Windows
    27.  
      Resolving garbled text displayed in NetBackup Administration Console on a UNIX computer
    28.  
      Troubleshooting error messages in the NetBackup web UI and the NetBackup Administration Console
    29.  
      Extra disk space required for logs and temporary files for the NetBackup Administration Console
    30.  
      Unable to logon to the NetBackup Administration Console after external CA configuration
    31.  
      Troubleshooting file-based external certificate issues
    32.  
      Troubleshooting issues with external certificate configuration
    33.  
      Troubleshooting Windows certificate store issues
    34.  
      Troubleshooting backup failures
    35.  
      Troubleshooting backup failure issues with NAT clients or NAT servers
    36.  
      Troubleshooting issues with the NetBackup Messaging Broker (or nbmqbroker) service
    37.  
      Troubleshooting issues with email notifications for Windows systems
    38.  
      Troubleshooting issues with KMS configuration
    39.  
      Troubleshooting issues with initiating the NetBackup CA migration because of large key size
    40.  
      Troubleshooting issues with the non-privileged user (service user) account
    41.  
      Troubleshooting issues with group name format in the auth.conf file
    42.  
      Troubleshooting the VxUpdate add package process
    43.  
      Troubleshooting issues with FIPS mode
    44.  
      Troubleshooting issues with malware scanning
    45.  
      Troubleshooting issues with NetBackup jobs that are enabled for data-in-transit encryption
    46.  
      Troubleshooting issues with Unstructured Data Instant Access
    47.  
      Troubleshooting issues with multifactor authentication
    48.  
      Troubleshooting issues with multi-person authorization
    49.  
      Troubleshooting connections to the NetBackup Scale-Out Relational Database
    50.  
      Troubleshooting issues with private key encryption
    51.  
      Troubleshooting issues with the security configuration risk feature
  3. Using NetBackup utilities
    1.  
      About NetBackup troubleshooting utilities
    2.  
      About the analysis utilities for NetBackup debug logs
    3.  
      About the Logging Assistant
    4.  
      About network troubleshooting utilities
    5. About the NetBackup support utility (nbsu)
      1.  
        Output from the NetBackup support utility (nbsu)
      2.  
        Example of a progress display for the NetBackup support utility (nbsu)
    6. About the NetBackup consistency check utility (NBCC)
      1.  
        Output from the NetBackup consistency check utility (NBCC)
      2.  
        Example of an NBCC progress display
    7.  
      About the NetBackup consistency check repair (NBCCR) utility
    8.  
      About the nbcplogs utility
    9. About the robotic test utilities
      1.  
        Robotic tests on UNIX
      2.  
        Robotic tests on Windows
    10. About the NetBackup Smart Diagnosis (nbsmartdiag) utility
      1.  
        Workflow to use the nbsmartdiag utility for NetBackup host communication
    11.  
      About log collection by job ID
  4. Disaster recovery
    1.  
      About disaster recovery
    2.  
      Recommended backup practices
    3.  
      Requirements and notes for disaster recovery
    4.  
      Disaster recovery packages
    5.  
      About disaster recovery settings
    6. About disk recovery procedures for UNIX and Linux
      1. About recovering the primary server disk on Linux
        1.  
          Recovering the primary server when root is intact
        2.  
          Recovering the primary server when the root partition is lost
      2.  
        About recovering the NetBackup media server disk for UNIX
      3.  
        Recovering the system disk on a UNIX client workstation
    7. About clustered NetBackup server recovery for UNIX and Linux
      1.  
        Replacing a failed node on a UNIX or Linux cluster
      2.  
        Recovering the entire UNIX or Linux cluster
    8. About disk recovery procedures for Windows
      1. About recovering the primary server disk for Windows
        1.  
          Recovering the primary server with Windows intact
        2.  
          Recovering the primary server and Windows
      2.  
        About recovering the NetBackup media server disk for Windows
      3.  
        Recovering a Windows client disk
    9. About clustered NetBackup server recovery for Windows
      1.  
        Replacing a failed node on a Windows VCS cluster
      2.  
        Recovering the shared disk on a Windows VCS cluster
      3.  
        Recovering the entire Windows VCS cluster
    10.  
      Generating a certificate on a clustered primary server after disaster recovery installation
    11.  
      About the DR_PKG_MARKER_FILE environment variable
    12.  
      Restoring the disaster recovery package on Windows
    13.  
      Restoring the disaster recovery package on Linux
    14. Options to recover the NetBackup catalog
      1. Prerequisites for recovering the NetBackup catalog or NetBackup catalog image files
        1.  
          Establishing a connection with NAT media server before catalog recovery
      2.  
        About NetBackup catalog recovery on Windows computers
      3.  
        About NetBackup catalog recovery from disk devices
      4.  
        About NetBackup catalog recovery and symbolic links
      5.  
        NetBackup disaster recovery email example
      6. About recovering the entire NetBackup catalog
        1.  
          Recovering the entire NetBackup catalog using the NetBackup catalog recovery wizard
        2.  
          Recovering the entire NetBackup catalog using bprecover -wizard
        3.  
          Specifying the NetBackup job ID number after a catalog recovery
      7. About recovering the NetBackup catalog image files
        1.  
          Recovering the NetBackup catalog image files using the NetBackup catalog recovery wizard
        2.  
          Recovering the NetBackup catalog image files using bprecover -wizard
      8. About recovering the NetBackup databases
        1.  
          Recovering the NetBackup database from a backup
        2.  
          Recovering the NetBackup database from staging
        3.  
          About processing the NetBackup database in staging
        4.  
          Terminating database connections
      9.  
        Recovering the NetBackup catalog when NetBackup Access Control is configured
      10.  
        Recovering the NetBackup catalog from a nonprimary copy of a catalog backup
      11.  
        Recovering the NetBackup catalog without the disaster recovery file
      12.  
        Recovering a NetBackup user-directed online catalog backup from the command line
      13.  
        Restoring files from a NetBackup online catalog backup
      14.  
        Unfreezing the NetBackup online catalog recovery media
      15.  
        Steps to carry out when you see exit status 5988 during catalog recovery

Restoring the disaster recovery package on Linux

After a disaster, you need to restore the disaster recovery package that corresponds to the catalog backup that you want to restore. This package is created during the catalog backup and contains the NetBackup primary server host identity. You must restore the host identity before you can perform a catalog recovery.

Important notes

Note the following about restoring the disaster recovery package and catalog recovery:

  • Catalog recovery does not recover the host identity. To restore the host identity or disaster recovery package, you must install NetBackup in the disaster recovery mode and import the required package. After you recover the disaster recovery package, you can recover the catalog.

  • After you restore the disaster recovery package, you must immediately perform catalog recovery.

  • Note the following when you want to recover a clustered primary server:

    • The disaster recovery package contains the identity files and configuration only for the virtual name.

    • After the DR installation, the virtual name's certificate is restored.

    • Cluster node-specific certificates and configuration options are not backed up and therefore are not recovered. You must redeploy or reconfigure NetBackup or external certificates after the DR installation.

  • After a catalog recovery, NetBackup freezes the removeable media that contains the catalog backup. This operation prevents a subsequent accidental overwrite action on the final catalog backup image on the media. This final image pertains to the actual catalog backup itself, and its recovery is not part of the catalog recovery. You can unfreeze the media.

Prerequisites

If external CA-signed certificates are used in your NetBackup domain, ensure the following:

  • In case of file-based external certificates, ensure that the certificate file path is configured, accessible, and is the same as the one that was backed up.

  • Note the following if you used a Windows certificate store as a certificate store before the disaster and the certificate files were not backed up during catalog backup. You must manually configure the external certificate for the host after the disaster. Refer to the following article:

    https://www.veritas.com/support/en_US/article.100044249

  • You have configured the required certificate revocation lists (CRL) before you begin the disaster recovery installation, if applicable.

    For more information on the CRLs, refer to the NetBackup Security and Encryption Guide.

  • In some cases when an external certificate was configured on the primary server before the disaster, the DR installation fails. You can set an environment variable that lets you correct the external certificate configuration.

    See About the DR_PKG_MARKER_FILE environment variable.

Options for restoring the disaster recovery package on Linux

You can restore the disaster recovery package of the NetBackup primary server either during installation or after installation.

See Restore the disaster recovery package during NetBackup installation on Linux.

See Restore the disaster recovery package after NetBackup installation on Linux.

Restore the disaster recovery package during NetBackup installation on Linux

The following procedure describes how to restore the disaster recovery package during the installation of NetBackup.

To restore the disaster recovery package of the NetBackup Appliance, you must follow a different procedure.

See Restore the disaster recovery package after NetBackup installation on Linux.

To restore the disaster recovery package during NetBackup installation

  1. Start the NetBackup software installation.

    Refer to the "Installing server software on UNIX systems" section in the NetBackup Installation Guide.

  2. When the following message appears, press Enter to continue:

    Is this host a master server? [y/n] (y)

  3. When the following message appears, select Y.

    Are you currently performing a disaster recovery of a master server? [y/n] (y)

  4. When the following message appears, provide the name and the path of the disaster recovery package that you want to restore.

    Enter the name of your disaster recovery package along with the path, or type q to exit the install script:

    If external certificates are used in your domain, a warning message is displayed. When the installer waits during subsequent steps, configure the external certificate configuration options as per step 6.

  5. When you are prompted, provide the passphrase that is associated with the disaster recovery package that you want to restore.

    Caution:

    Ensure that you specify the appropriate passphrase.

    If you specify a wrong passphrase or the passphrase is lost, you need to deploy security certificates on all hosts after installation. The disaster recovery package cannot be restored during installation. To restore the disaster recovery package after installation, refer to the following article:

    http://www.veritas.com/docs/000125933

    Enter your disaster recovery passphrase, or enter q to exit installation:

    The following message appears:

    Validating disaster recovery passphrase...

    If the passphrase is validated, continue with the installation.

  6. (Conditional) If external CA-signed certificates are used in your NetBackup domain, do the following:

    • Review the value of the ECA_CRL_CHECK configuration option.

      For more information on catalog backup and external certificate configuration options, refer to the NetBackup Administrator's Guide, Volume I.

      • If the ECA_CRL_CHECK configuration option is set to DISABLE, you do not need to do the CRL configurations.

      • If the ECA_CRL_CHECK configuration option is enabled, you are prompted to configure the CRL.

        The UNIX installer does not wait for any action but proceeds to the next step in the installer. When the installer waits after the following step, you can configure the CRLs and continue with the DR installation.

        Configure the CRLs and continue with the DR installation.

    • Depending on the value that is specified for the ECA_CRL_PATH option, make the required CRLs available.

      • If ECA_CRL_PATH is not specified, NetBackup uses the CRLs from the CRL distribution point (CDP) of the peer host's certificate. Ensure that the URLs that are available in the CDP are accessible.

      • If ECA_CRL_PATH is specified, NetBackup uses the CRLs that are available in the directory that is specified for this option. Copy the valid CRLs in the directory that you specify for ECA_CRL_PATH.

    • (Conditional) If the DR_PKG_MARKER_FILE environment variable was set before the DR installation, a message indicates that the touch file exists. After you complete the external certificate configuration, delete the touch file.

      NetBackup services are started.

  7. (Conditional) If the key management service (KMS) was configured on the primary server before the disaster, run the following command to start the KMS service:

    /usr/openv/netbackup/bin/nbkmscmd -discoverNBKMS

  8. Refer to the "Installing server software on UNIX systems" section in the NetBackup Installation Guide.
Restore the disaster recovery package after NetBackup installation on Linux

The following procedure describes how to restore the disaster recovery package after the installation of NetBackup.

Use this option to restore the disaster recovery package of the NetBackup Appliance.

To restore the disaster recovery package after NetBackup installation

  1. Run the nbhostidentity -import -infile file_path command after NetBackup installation.

    Refer to the NetBackup Commands Reference Guide.

  2. Clean up the allowed list cache and restart the NetBackup services on all hosts in the domain.
  3. Refresh the certificate revocation list (CRL) using the following command:

    nbcertcmd -getcrl

  4. If the key management service (KMS) was configured on the primary server before the disaster, run the following command to start the KMS service:

    /usr/openv/netbackup/bin/nbkmscmd -discoverNBKMS

  5. Carry out the given step to remove the NetBackup certificate files in the following scenario:

    NetBackup was configured to use only external CA-signed certificates before the disaster and it was configured to use NetBackup certificates or both NetBackup and external certificates before you manually imported the disaster recovery package.

    Remove the NetBackup certificate files using the following command:

    configureWebServerCerts -removeNBCert

More Information

Unfreezing the NetBackup online catalog recovery media