NetBackup™ for OpenStack Administrator's Guide

Last Published:
Product(s): NetBackup & Alta Data Protection (10.2)
  1. Introduction
    1.  
      About NetBackup for OpenStack
    2. NetBackup for OpenStack Architecture
      1.  
        Backup as a Service
      2.  
        Main Components
      3.  
        Service Endpoints
      4.  
        Network topology
  2. Deploying NetBackup for OpenStack
    1. Requirements
      1. System requirements NetBackup for OpenStack VM
        1.  
          Software Requirements
    2. NetBackup for OpenStack network considerations
      1.  
        Existing endpoints in OpenStack
      2.  
        OpenStack endpoints required by NetBackup for OpenStack
      3.  
        Recommendation: Provide access to all OpenStack Endpoint types
      4.  
        Backup target access required by NetBackup for OpenStack
      5.  
        Example of a typical NetBackup for OpenStack network integration
      6.  
        Other examples of NetBackup for OpenStack network integrations
    3. Preparing the installation
      1.  
        Tenant Quotas
      2.  
        AWS S3 eventual consistency
      3.  
        NetBackup for OpenStack Cluster
    4. Spinning up the NetBackup for OpenStack VM
      1. Creating the cloud-init image
        1.  
          Needed tools
        2.  
          Providing the Metadata
        3.  
          Creating the image file
      2.  
        Spinning up the NetBackup for OpenStack appliance
      3.  
        Uninstalling cloud-init after first start
    5.  
      About NetBackup for OpenStack backup target types
    6. Installing NetBackup for OpenStack Components
      1. Installing on RHOSP
        1. Prepare for deployment
          1.  
            Copying nbos-cfg-scripts to the undercloud
          2.  
            If backup target type is Ceph-based S3 with SSL:
        2.  
          Uploading the NetBackup for OpenStack puppet module
        3. Updating the overcloud roles data file to include NetBackup for OpenStack services
          1.  
            Adding the NetBackup for OpenStack Datamover API Service to role data file
          2.  
            Adding NetBackup for OpenStack Datamover Service to role data file
        4.  
          Preparing the NetBackup for OpenStack container images
        5.  
          Providing the environment details in nbos_env.yaml
        6.  
          Deploying the overcloud with NetBackup OpenStack environment
        7. Verifying the deployment
          1.  
            On the controller node
          2.  
            On the compute node
          3.  
            On the node with Horizon service
        8. Additional Steps on NetBackup for OpenStack Appliance
          1.  
            Changing the nova user ID on the NetBackup for OpenStack Nodes
        9.  
          Troubleshooting for overcloud deployment failures
      2. Installing on Ansible OpenStack Ussuri
        1.  
          Verify that file-level logging is configured for OpenStack components on Horizon container
        2.  
          Changing the nova user ID on the NetBackup for OpenStack Nodes
        3.  
          Preparing the deployment host
        4.  
          Deploying the NetBackup for OpenStack components
        5.  
          Verifying the NetBackup for OpenStack deployment
      3. Installing on Kolla Ussuri
        1.  
          Changing the nova user ID on the NetBackup for OpenStack Nodes
        2.  
          Copying the NetBackup for OpenStack deployment scripts
        3.  
          Copying the NetBackup for OpenStack deployment scripts to Kolla-ansible deploy scripts
        4. Pushing NetBackup for OpenStack images to the local registry
          1.  
            Running the local registry
          2.  
            Loading the images from tar and pushing them to the local repository
        5.  
          Editing globals.yml to set NetBackup for OpenStack parameters
        6.  
          Enabling the NetBackup for OpenStack snapshot mount feature
        7.  
          Pulling the NetBackup for OpenStack container images
        8.  
          Deploying the NetBackup for OpenStack components
        9.  
          Verifying the NetBackup for OpenStack deployment
    7. Configuring NetBackup for OpenStack
      1. Details needed for the NetBackup for OpenStack Appliance
        1.  
          NetBackup for OpenStack Cluster information
        2.  
          OpenStack Credentials information
        3. Backup Storage Configuration information
          1.  
            Using the NFS protocol
          2.  
            Using the S3 protocol
          3.  
            Policy Import
      2. Advanced settings
        1.  
          Setup NetBackup for OpenStack Job Manager and NetBackup for OpenStack Datamover API
        2.  
          Set up an external database
        3.  
          Define the NetBackup for OpenStack service user password
      3.  
        Starting the configurator
    8. Post Installation Health-Check
      1.  
        Verify the NetBackup for OpenStack Appliance services are up
      2.  
        Check the NetBackup for OpenStack pacemaker and NGINX cluster
      3.  
        Verify API connectivity of the NetBackup for OpenStack Appliance
      4.  
        Verify the nbosdm services are up and running
      5.  
        Verify that the NFS Volume is correctly mounted
    9. Uninstalling NetBackup for OpenStack
      1. Uninstalling from RHOSP
        1.  
          Clean NetBackup for OpenStack Datamover API service
        2.  
          Clean NetBackup for OpenStack Datamover Service
        3.  
          Clean NetBackup for OpenStack haproxy resources
        4.  
          Clean NetBackup for OpenStack Keystone resources
        5.  
          Clean NetBackup for OpenStack database resources
        6.  
          Revert overcloud deploy command
        7.  
          Revert back to original RHOSP Horizon container
        8.  
          Destroy the NetBackup for OpenStack VM Cluster
      2. Uninstalling from Ansible OpenStack
        1.  
          Uninstall NetBackup for OpenStack Services
        2.  
          Destroy NetBackup for OpenStack Datamover API container
        3.  
          Clean openstack_user_config.yml
        4.  
          Remove NetBackup for OpenStack haproxy settings in user_variables.yml
        5.  
          Remove NetBackup for OpenStack Datamover API inventory file
        6.  
          Remove NetBackup for OpenStack Datamover API service endpoints
        7.  
          Delete NetBackup for OpenStack Datamover API database and user
        8.  
          Remove nbosdmapi rabbitmq user from rabbitmq container
        9.  
          Clean haproxy
        10.  
          Remove certificates from Compute nodes
        11.  
          Destroy the NetBackup for OpenStack VM Cluster
      3. Uninstalling from Kolla Openstack
        1.  
          Cleaning NetBackupOpenStack_datamover_api container
        2.  
          Cleaning NetBackupOpenStack_datamover container
        3.  
          Cleaning haproxy of NetBackupOpenStack Datamover API
        4.  
          Cleaning Kolla Ansible deployment procedure
        5.  
          Reverting to original Horizon container
        6.  
          Cleaning Keystone resources
        7.  
          Cleaning NetBackup for OpenStack database resources
        8.  
          Destroy the NetBackup for OpenStack VM Cluster
    10. Install nbosjm CLI client
      1.  
        About the nbosjm CLI client
      2.  
        Installing the nbosjm client
    11.  
      About log rotation in NetBackup for OpenStack
  3. Configuring NetBackup OpenStack Appliance
    1.  
      Reconfigure the NetBackup for OpenStack Cluster
    2. Configuring the NetBackup master server details
      1.  
        About security management and certificates in NetBackup
    3.  
      Change NetBackup for OpenStack dashboard password
    4.  
      Reset NetBackup for OpenStack dashboard password
    5.  
      Reinitialize NetBackup for OpenStack
    6.  
      Download NetBackup for OpenStack logs
  4. Configuring NetBackup Master Server
    1.  
      License for OpenStack plug-in for NetBackup
    2.  
      Allow NetBackup for OpenStack VM on NetBackup master server
    3. About launching the OpenStack Horizon UI from the NetBackup web UI
      1.  
        Adding the OpenStack Horizon instance on NetBackup web UI
      2.  
        Creating the custom role for NetBackup for OpenStack administrator
      3.  
        Launching the Horizon UI from the NetBackup web UI
  5. NetBackup for OpenStack policies
    1.  
      About policies
    2.  
      List of policies
    3.  
      Create a policy
    4.  
      Policy overview
    5.  
      Edit a policy
    6.  
      Delete a policy
    7.  
      Unlock a policy
    8.  
      Reset a policy
  6. Performing backups and restores of OpenStack
    1.  
      About snapshots
    2.  
      List of snapshots
    3.  
      Creating a snapshot
    4.  
      Snapshot overview
    5.  
      Delete snapshots
    6.  
      Cleaning up the volume snapshots
    7.  
      Snapshot Cancel
    8. About restores
      1.  
        About restoring multi-attach volumes
    9.  
      List of Restores
    10.  
      Restores overview
    11.  
      Delete a Restore
    12.  
      Cancel a Restore
    13.  
      One-Click Restore
    14.  
      Selective Restore
    15.  
      In-place restore
    16. Required restore.json for CLI
      1.  
        General required information
      2. Selective Restore required information
        1.  
          Information required in instances
        2.  
          Information required in network topology restore or network mapping
        3.  
          Full selective restore example
      3. Inplace Restore required information
        1.  
          Information required in instances
        2.  
          Network mapping information required
        3.  
          Full Inplace restore example
    17.  
      About file search
    18.  
      Navigating to the file search tab in Horizon
    19. Configuring and starting a file search in Horizon
      1.  
        Choose the VM the file search shall run against
      2.  
        Set the file path
      3.  
        Define the Snapshots to search in
    20.  
      Start the File Search and retrieve the results in Horizon
    21.  
      Doing a CLI File Search
    22.  
      About snapshot mount
    23.  
      Creating a file recovery manager instance
    24.  
      Mounting a snapshot
    25.  
      Accessing the File Recovery Manager
    26.  
      Identifying mounted snapshots
    27.  
      Unmounting a snapshot
    28.  
      About schedulers
    29.  
      Disable a schedule
    30.  
      Enable a schedule
    31.  
      Modify a schedule
    32.  
      About email notifications
    33.  
      Requirements to activate email Notifications
    34.  
      Activate/Deactivate the email Notifications
  7. Performing Backup Administration tasks
    1. NBOS Backup Admin Area
      1.  
        Access the NBOS Backup Admin area
      2.  
        Status overview
      3.  
        Policies tab
      4.  
        Usage tab
      5.  
        Nodes tab
      6.  
        NBOSDM tab (NetBackup for OpenStack Datamover Service)
      7.  
        Storage tab
      8.  
        Audit tab
      9.  
        Policy Attributes tab
      10. Settings tab
        1.  
          Email Settings
        2.  
          Disable or enable Job Scheduler
    2. Policy Attributes
      1.  
        List the available policies
      2.  
        Create policy attributes
      3.  
        Edit the policy attribute
      4.  
        Assign/Remove a policy
      5.  
        Delete a policy
    3. Policy Quotas
      1.  
        Work with Policy Quotas via Horizon
      2. Work with Policy Quotas via CLI
        1.  
          List available Quota Types
        2.  
          Show Quota Type Details
        3.  
          Create a Quota
        4.  
          List allowed Quotas
        5.  
          Show allowed Quota
        6.  
          Update allowed Quota
        7.  
          Delete allowed Quota
    4. Managing Trusts
      1.  
        List all trusts
      2.  
        Show a trust
      3.  
        Create a trust
      4.  
        Delete a trust
    5. Policy import and migration
      1.  
        Import policies
      2.  
        Orphaned policies
      3.  
        Reassigning policies
    6. Disaster Recovery
      1.  
        Disaster Recovery Process
      2.  
        Mount-paths
    7. Example runbook for disaster recovery using NFS
      1.  
        Scenario
      2.  
        Prerequisites for the disaster recovery process
      3. Disaster recovery of a single policy
        1. Copy the policy directories to the configured NFS Volume
          1.  
            Mount NFS B2 Volume to a NetBackup for OpenStack VM
          2.  
            Identify the policy on the NFS B2 Volume
          3.  
            Copy the policy
        2. Make the Mount-Paths available
          1.  
            Identify the base64 hash values
          2.  
            Create and bind the paths
        3. Reassign the policy
          1.  
            Add admin-user to required domains and projects
          2.  
            Discover orphaned policies from NFS-Storage of Target Cloud
          3.  
            List available projects on Target Cloud in the Target Domain
          4.  
            List available users on the Target Cloud in the Target Project that have the right backup trustee role
          5.  
            Reassign the policy to the target project
          6.  
            Verify that the policy is available at the desired target_project
        4. Restore the policy
          1.  
            Prepare the selective restore by getting the snapshot information
          2.  
            Prepare the selective restore by creating the restore.json file
          3.  
            Run the selective restore
          4.  
            Verify the restore
        5. Clean up
          1.  
            Delete the policy
          2.  
            Remove the database entry
          3.  
            Remove the admin user from the project
      4. Disaster recovery of a complete cloud
        1. Reconfigure the Target NetBackup for OpenStack installation
          1.  
            Add NFS B2 to the NetBackup for OpenStack Appliance Cluster
          2.  
            Add NFS B2 to the NetBackup for OpenStack Datamovers
        2. Make the Mount-Paths available
          1.  
            Identify the base64 hash values
          2.  
            Create and bind the paths
        3. Reassign the policy
          1.  
            Add admin-user to required domains and projects
          2.  
            Discover orphaned policies from NFS-Storage of Target Cloud
          3.  
            List available projects on Target Cloud in the Target Domain
          4.  
            List available users on the Target Cloud in the Target Project that have the right backup trustee role
          5.  
            Reassign the policy to the target project
          6.  
            Verify that the policy is available at the desired target_project
        4. Restore the policy
          1.  
            Prepare the selective restore by getting the snapshot information
          2.  
            Prepare the selective restore by creating the restore.json file
          3.  
            Run the selective restore
          4.  
            Verify the restore
        5. Reconfigure the Target NetBackup for OpenStack installation back to the original one
          1.  
            Delete NFS B2 from the NetBackup for OpenStack Appliance Cluster
          2.  
            Delete NFS B2 from the NetBackup for OpenStack Datamovers
        6. Clean up
          1.  
            Remove the database entry
          2.  
            Remove the admin user from the project
  8. Troubleshooting
    1. General Troubleshooting Tips
      1. What is happening where
        1. NetBackup for OpenStack cluster
          1.  
            During a backup process
          2.  
            During a restore process
        2.  
          nbosdmapi
        3.  
          nbosdm
      2.  
        Everything on the Backup Target happens as user nova
      3.  
        NetBackup for OpenStack Trustee Role
      4.  
        OpenStack Quotas
      5.  
        Ephemeral disk backup
    2.  
      Using the nbosjm CLI tool on the NetBackup for OpenStack Appliance
    3. Health check of NetBackup for OpenStack
      1.  
        On the NetBackup for OpenStack Cluster
      2.  
        The nbosdmapi service
      3.  
        The nbosdm service
    4. Important log files
      1.  
        On the NetBackup for OpenStack Nodes
      2.  
        NetBackup for OpenStack Datamover service logs on RHOSP
      3.  
        NetBackup for OpenStack Datamover service logs on Ansible OpenStack
      4.  
        NetBackup for OpenStack Datamover service logs on Kolla Ussuri
    5.  
      Troubleshooting NBOSDM container in offline state due to unavailable mount point
    6.  
      About permission denied error when same NFS share path is used across multiple OpenStack distributions
    7.  
      After restore of the Windows instance, disk is in offline state
  9.  
    Index

AWS S3 eventual consistency

AWS S3 object consistency model includes:

  1. Read-after-write

  2. Read-after-update

  3. Read-after-delete

Each of them describes how an object reaches its consistent state after an object is created, updated, or deleted. None of them provides strong consistency and there is a lag time for an object to reach the consistent state. Though NetBackup for OpenStack employed mechanisms to work around the limitations of eventual consistency of AWS S3, when an object reaches its consistency state is not deterministic. There is no official statement from AWS on how long it takes for an object to reach consistent state. However read-after-write has a shorter time to reach the consistency compared to other IO patterns. Our solution is designed to maximize read-after-write IO pattern. The time in which an object reaches eventual consistency also depends on the AWS region. For example, aws-standard region does not have strong consistency model compared to us-east or us-west. We suggest using these regions when you create s3 buckets for NetBackup for OpenStack. Though read-after-update IO pattern is hard to avoid completely, we employed ample delays in accessing objects to accommodate larger durations for objects to get into consistent state. However in rare occasions, backups may still fail and need to be restarted.