Veritas NetBackup™ for Hadoop Administrator's Guide

Last Published:
Product(s): NetBackup (8.3.0.1)
  1. Introduction
    1.  
      Protecting Hadoop data using NetBackup
    2.  
      Backing up Hadoop data
    3.  
      Restoring Hadoop data
    4.  
      NetBackup for Hadoop terminologies
    5.  
      Limitations
  2. Verifying the pre-requisites and best practices for the Hadoop plug-in for NetBackup
    1.  
      About deploying the Hadoop plug-in
    2. Pre-requisites for the Hadoop plug-in
      1.  
        Operating system and platform compatibility
      2.  
        NetBackup server and client requirements
      3.  
        License for Hadoop plug-in for NetBackup
    3.  
      Preparing the Hadoop cluster
    4.  
      Best practices for deploying the Hadoop plug-in
  3. Configuring NetBackup for Hadoop
    1.  
      About configuring NetBackup for Hadoop
    2. Managing backup hosts
      1.  
        Whitelisting a NetBackup client on NetBackup master server
      2.  
        Configure a NetBackup Appliance as a backup host
    3.  
      Adding Hadoop credentials in NetBackup
    4. Configuring the Hadoop plug-in using the Hadoop configuration file
      1.  
        Configuring NetBackup for a highly-available Hadoop cluster
      2.  
        Configuring a custom port for the Hadoop cluster
      3.  
        Configuring number of threads for backup hosts
      4. Configuring communication between NetBackup and Hadoop clusters that are SSL-enabled (HTTPS)
        1.  
          ECA_TRUST_STORE_PATH for NetBackup servers and clients
        2.  
          ECA_CRL_PATH for NetBackup servers and clients
        3.  
          HADOOP_SECURE_CONNECT_ENABLED for servers and clients
        4.  
          HADOOP_CRL_CHECK for NetBackup servers and clients
        5.  
          Example values for the parameters in the bp.conf file
    5.  
      Configuration for a Hadoop cluster that uses Kerberos
    6. Configuring NetBackup policies for Hadoop plug-in
      1. Creating a BigData backup policy
        1. Creating BigData policy using the NetBackup Administration Console
          1.  
            Using the Policy Configuration Wizard to create a BigData policy for Hadoop clusters
          2.  
            Using the NetBackup Policies utility to create a BigData policy for Hadoop clusters
        2.  
          Using NetBackup Command Line Interface (CLI) to create a BigData policy for Hadoop clusters
    7.  
      Disaster recovery of a Hadoop cluster
  4. Performing backups and restores of Hadoop
    1. About backing up a Hadoop cluster
      1.  
        Pre-requisite for running backup and restore operations for a Hadoop cluster with Kerberos authentication
      2.  
        Best practices for backing up a Hadoop cluster
      3.  
        Backing up a Hadoop cluster
    2. About restoring a Hadoop cluster
      1.  
        Best practices for restoring a Hadoop cluster
      2. Restoring Hadoop data on the same Hadoop cluster
        1.  
          Using the Restore Wizard to restore Hadoop data on the same Hadoop cluster
        2.  
          Using the bprestore command to restore Hadoop data on the same Hadoop cluster
      3.  
        Restoring Hadoop data on an alternate Hadoop cluster
  5. Troubleshooting
    1.  
      About troubleshooting NetBackup for Hadoop issues
    2.  
      About NetBackup for Hadoop debug logging
    3. Troubleshooting backup issues for Hadoop data
      1.  
        Backup operation fails with error 6609
      2.  
        Backup operation failed with error 6618
      3.  
        Backup operation fails with error 6647
      4.  
        Extended attributes (xattrs) and Access Control Lists (ACLs) are not backed up or restored for Hadoop
      5.  
        Backup operation fails with error 6654
      6.  
        Backup operation fails with bpbrm error 8857
      7.  
        Backup operation fails with error 6617
      8.  
        Backup operation fails with error 6616
      9.  
        NetBackup configuration and certificate files do not persist after the container-based NetBackup appliance restarts
      10.  
        Unable to see incremental backup images during restore even though the images are seen in the backup image selection
      11.  
        One of the child backup jobs goes in a queued state
    4. Troubleshooting restore issues for Hadoop data
      1.  
        Restore fails with error code 2850
      2.  
        NetBackup restore job for Hadoop completes partially
      3.  
        Extended attributes (xattrs) and Access Control Lists (ACLs) are not backed up or restored for Hadoop
      4.  
        Restore operation fails when Hadoop plug-in files are missing on the backup host
      5.  
        Restore fails with bpbrm error 54932
      6.  
        Restore operation fails with bpbrm error 21296
      7.  
        Configuration file is not recovered after a disaster recovery
  6.  
    Index

Adding Hadoop credentials in NetBackup

To establish a seamless communication between Hadoop clusters and NetBackup for successful backup and restore operations, you must add and update Hadoop credentials to the NetBackup master server.

Use the tpconfig command to add Hadoop credentials in NetBackup master server.

For information on parameters to delete and update the credentials using the tpconfig command, see the NetBackup Commands Reference Guide.

Consider the following when you add Hadoop credentials:

  • For a highly-available Hadoop cluster, ensure that the user for the primary and fail-over NameNode is the same.

  • Use the credentials of the application server that you will use when configuring the BigData policy.

  • For a Hadoop cluster that uses Kerberos, specify "kerberos" as application_server_user_id value.

  • Hostname and port of the NameNode must be same as you have specified with the http address parameter in the core-site.xml of the Hadoop cluster.

  • For password, provide any random value. For example, Hadoop.

To add Hadoop credentials in NetBackup

  1. Run tpconfig command from the following directory paths:

    On UNIX systems, /usr/openv/volmgr/bin/

    On Windows systems, install_path\Volmgr\bin\

  2. Run the tpconfig --help command. A list of options which are required to add, update, and delete Hadoop credentials is displayed.
  3. Run the tpconfig -add -application_server application_server_name -application_server_user_id user_ID -application_type application_type -requiredport IP_port_number [-password password [-key encryption_key]] command by providing appropriate values for each parameter to add Hadoop credentials.

    For example, if you want to add credentials for Hadoop server which has application_server_name as hadoop1, then run the following command using the appropriate <user_ID> and <password> details.

    tpconfig -add -application_server hadoop1 -application_type hadoop -application_server_user_id Hadoop -requiredport 50070 -password Hadoop

    Here, the value hadoop specified for -application_type parameter corresponds to Hadoop.

  4. Run the tpconfig -dappservers command to verify if the NetBackup master server has the Hadoop credentials added.