NetBackup™ for Kubernetes Administrator's Guide

Last Published:
Product(s): NetBackup & Alta Data Protection (11.0)
  1. Overview of NetBackup for Kubernetes
    1.  
      Overview
    2.  
      Features of NetBackup support for Kubernetes
  2. Deploying and configuring the NetBackup Kubernetes operator
    1.  
      Prerequisites for NetBackup Kubernetes Operator deployment
    2.  
      Deploy service package on NetBackup Kubernetes operator
    3.  
      Port requirements for Kubernetes operator deployment
    4.  
      Upgrade the NetBackup Kubernetes operator
    5.  
      Delete the NetBackup Kubernetes operator
    6.  
      Configure NetBackup Kubernetes data mover
    7.  
      Automated configuration of NetBackup protection for Kubernetes
    8. Customize Kubernetes workload
      1.  
        Prerequisites for backup from snapshot and restore from backup operations
      2.  
        DTE client settings supported in Kubernetes
      3.  
        Customization of datamover properties
    9.  
      Troubleshooting NetBackup servers with short names
    10.  
      Data mover pod schedule mechanism support
    11.  
      Validating accelerator storage class
  3. Deploying certificates on NetBackup Kubernetes operator
    1.  
      Deploy certificates on the Kubernetes operator
    2.  
      Perform Host-ID-based certificate operations
    3.  
      Perform ECA certificate operations
    4.  
      Identify certificate types
  4. Managing Kubernetes assets
    1.  
      Add a Kubernetes cluster
    2. Configure settings
      1.  
        Change resource limits for Kuberentes resource types
      2.  
        Configure autodiscovery frequency
      3.  
        Configure permissions
      4.  
        Asset cleanup
    3.  
      Add protection to the assets
    4. Scan for malware
      1.  
        Assets by workload type
  5. Managing Kubernetes intelligent groups
    1.  
      About intelligent group
    2.  
      Create an intelligent group
    3.  
      Delete an intelligent group
    4.  
      Edit an intelligent group
  6. Managing Kubernetes policies
    1.  
      Create a policy
  7. Protecting Kubernetes assets
    1.  
      Protect an intelligent group
    2.  
      Remove protection from an intelligent group
    3.  
      Configure backup schedule
    4.  
      Configure backup options
    5.  
      Configure backups
    6.  
      Configure Auto Image Replication (A.I.R.) and duplication
    7.  
      Configure storage units
    8.  
      Volume mode support
    9.  
      Configure application consistent backup
  8. Managing image groups
    1. About image groups
      1.  
        Image expire
      2.  
        Image copy
  9. Protecting Rancher managed clusters in NetBackup
    1.  
      Add Rancher managed RKE cluster in NetBackup using automated configuration
    2.  
      Add Rancher managed RKE cluster manually in NetBackup
  10. Recovering Kubernetes assets
    1.  
      Explore and validate recovery points
    2.  
      Restore from snapshot
    3.  
      Restore from backup copy
  11. About incremental backup and restore
    1.  
      Incremental backup and restore support for Kubernetes
  12. Enabling accelerator based backup
    1.  
      About NetBackup Accelerator support for Kubernetes workloads
    2.  
      Controlling disk space for track logs on primary server
    3.  
      Effect of storage class behavior on Accelerator
    4.  
      About Accelerator forced rescan
    5.  
      Warnings and probable reason for Accelerator backup failures
  13. Enabling FIPS mode in Kubernetes
    1.  
      Enable Federal Information Processing Standards (FIPS) mode in Kubernetes
  14. About Openshift Virtualization support
    1.  
      OpenShift Virtualization support
    2.  
      Application consistent virtual machines backup
    3.  
      Troubleshooting for virtualization
  15. Troubleshooting Kubernetes issues
    1.  
      Error during the primary server upgrade: NBCheck fails
    2.  
      Error during an old image restore: Operation fails
    3.  
      Error during persistent volume recovery API
    4.  
      Error during restore: Final job status shows partial failure
    5.  
      Error during restore on the same namespace
    6.  
      Datamover pods exceed the Kubernetes resource limit
    7.  
      Error during restore: Job fails on the highly loaded cluster
    8.  
      Custom Kubernetes role created for specific clusters cannot view the jobs
    9.  
      Openshift creates blank non-selected PVCs while restoring applications installed from OperatorHub
    10.  
      NetBackup Kubernetes operator become unresponsive if PID limit exceeds on the Kubernetes node
    11.  
      Failure during edit cluster in NetBackup Kubernetes 10.1
    12.  
      Backup or restore fails for large sized PVC
    13.  
      Restore of namespace file mode PVCs to different file system partially fails
    14.  
      Restore from backup copy fails with image inconsistency error
    15.  
      Connectivity checks between NetBackup primary, media, and Kubernetes servers.
    16.  
      Error during accelerator backup when there is no space available for track log
    17.  
      Error during accelerator backup due to track log PVC creation failure
    18.  
      Error during accelerator backup due to invalid accelerator storage class
    19.  
      Error occurred during track log pod start
    20.  
      Failed to setup the data mover instance for track log PVC operation
    21.  
      Error to read track log storage class from configmap

Perform ECA certificate operations

Before performing External Certificate Authority (ECA) create, update, and remove operations; you must configure the backup server in ECA mode.

To check if the ECA mode is on, run the command: /usr/openv/netbackup/bin/nbcertcmd -getSecConfig -caUsage.

The output looks like this:

NBCA: ON
ECA: ON

To configure the backup server in ECA mode, refer to the About external CA support in NetBackup section in the NetBackup™ Security and Encryption Guide

ECA certificate specification looks like this:

apiVersion: netbackup.veritas.com/v1
kind: BackupServerCert
metadata:
  name: backupservercert-sample-eca
  namespace: kops-ns
spec:
  clusterName: cluster.sample.com:port
  backupServer: primaryserver.sample.domain.com
  certificateOperation: Create | Update | Remove
  certificateType: ECA
  ecaAttributes:
    ecaCreateOptions:
      ecaSecretName: "Secret name consists of cert, key, passphrase, cacert"
      copyCertsFromSecret: true | false
      isKeyEncrypted: true | false
    ecaUpdateOptions:
      ecaCrlCheck: DISABLE | LEAF | CHAIN
      ecaCrlRefreshHours: range[0,4380]

Table: ECA certificate operations

Operation type

Options and comments

Create

  • secretName: Name of secret containing cert, key, passphrase, cacert.

  • copyCertsFromSecret: Possible values are true and false. This option is added as the External CA is common across all primary servers. Same certificates can be enrolled to Kubernetes operator for all primary servers.

    Thus, there is no need to copy certs and keys every time. Copying of certificates and keys can be controlled with this option.

    If ECAHealthCheck fails due to something wrong with certs and keys, then the certificates must be copied again.

  • isKeyEncrypted; If the private key is encrypted, set this field as true else set it as false.

Remove

NA

Update

  • ecaCrlCheck: Lets you specify the revocation check level for external certificates.

    Possible values are DISABLE, LEAF, and CHAIN.

  • ecaCrlRefreshHours specifies the time interval in hours to download Certificate Revocation Lists.

    Possible values range between 0-4380

Creating ECA signed certificate

NetBackup supports Kubernetes operator on multiple primary servers for ECA. If the external CA is common across primary servers. It is mandatory to use Certificate Revocation List distribution point for fetching Certificate Revocation List dynamically during the communication.

To create ECA signed certificate

  1. Use the Certificate Revocation List distribution point to fetch Certificate Revocation List.
  2. Keep ECA signed certificate chain, private key, and passphrase (if required) ready in your home directory.
  3. To identify different formats (like, DER, PEM and so on) that are supported for each of the files mentioned in step 2. For more information, refer to the Configuration options for external CA-signed certificates section in the NetBackup™ Security and Encryption Guide.
  4. Create a secret using the files mentioned in step 3.
    • To create a secret if private key is unencrypted, run the command: kubectl create secret generic <Name of secret>

      --from-file=cert_chain=<File path to ECA signed certificate chain> --from-file=key=<File path to private key>

      --from-file=cacert=<File path to External CA certificate> -n <Namespace where kops is deployed>

    • To create a secret if private key is encrypted, run the command: kubectl create secret generic <Name of secret>

      --from-file=cert_chain=<File path to ECA signed certificate chain> --from-file=key=<File path to private key>

      --from-file=cacert=<File path to External CA certificate> --from-file=passphrase=<File path to passphrase

      of encrypted private key> -n <Namespace where kops is deployed>

    Directory structure looks like this:

    ├── cert_chain.pem
    ├── private
    |  |___key.pem
    |  |___passphrase.txt
    |___trusted
         |__cacerts.pem
    

    cert_chain.pem is ECA signed certificate chain

    private/key.pem is private key

    private/passphrase.txt is passpharse for private key

    trusted/cacerts.pem is External CA certificate

    • To create a secret of name eca-secret when private key is unencrypted, run the command:

      kubectl create secret generic eca-secret--from-file=cert_chain=cert_chain.pem

      --from-file=key=private/key.pem

      --from-file=cacert=trusted/cacerts.pem -n kops-ns

    • To create a secret of name eca-secret when private key is encrypted, run the command:

      kubectl create secret generic eca-secret

      --from-file=cert_chain=cert_chain.pem

      --from-file=key=private/key.pem

      --from-file=cacert=trusted/cacerts.pem

      --from- file=passphrase=private/passphrase.txt

      -n kops-ns

  5. Once the secret is created, then create a backupservercert object custom resource.

    eca-create-backupservercert.yaml file looks like this:

    apiVersion: netbackup.veritas.com/v1
    kind: BackupServerCert
    metadata:
      name: backupservercert-eca-create
      namespace: kops-ns
    spec:
      clusterName: cluster.sample.com:port
      backupServer: backupserver.sample.domain.com
      certificateOperation: Create
      certificateType: ECA
      ecaAttributes:
        ecaCreateOptions:
          ecaSecretName: eca-secret
          copyCertsFromSecret: true
          isKeyEncrypted: false
    • Copy the eca-create-backupservercert.yaml file text.

    • Open the text editor and paste the yaml file text.

    • Then, save the text with the yaml file extension to the home directory from where the Kubernetes clusters are accessible.

  6. To copy certificate and keys to the Kubernetes operator, do any of the following:
    • Set copyCertsFromSecret as true

    • Set copyCertsFromSecret as false to avoid copying certificates and keys existing on the Kubernetes Operator.

    Note:

    ECA is common across all primary server thus Kubernetes operator require one set of certificates and keys that can be enrolled with all primary servers as required. No need to copy certificates and keys every time unless there's issue with the previous copied certificates and keys.

    Note:

    If ecaHealthCheck fails due to any reason related to certificates and keys (corrupted or expired or changed ECA) then you identify the reason for failure and perform a copy of a valid certificate using a flag.

  7. If private key is encrypted, set isKeyEncrypted flag as true or else false for unencrypted key. Ensure passphrase is provided in secret if private key is encrypted.
  8. Set ecaSecretName with the secret name, created backupservercert yaml in step 5.
  9. To create the eca-create-backupservercert.yaml file, run the command: kubectl create -f eca-create-backupservercert.yaml
  10. Once the backupservercert custom resource is created, check the custom resource status.
  11. To view the external certificate details on the NetBackup web UI, refer to the View external certificate information for the NetBackup hosts in the domain section in the NetBackup™ Web UI Administrator's Guide.
Removing the ECA signed certificate

You can remove the ECA signed certificate from the primary server.

To remove ECA signed certificate

  1. Create a backupservercert with operation as remove and certificate type as ECA.

    eca-remove-backupservercert.yaml file looks like this:

    apiVersion: netbackup.veritas.com/v1
    kind: BackupServerCert
    metadata:
      name: backupservercert-eca-remove
      namespace: kops-ns
    spec:
      clusterName: cluster.sample.com:port
      backupServer: backupserver.sample.domain.com
      certificateOperation: Remove
      certificateType: ECA
    • Copy the eca-remove-backupservercert.yaml file text.

    • Open the text editor and paste the yaml file text.

    • Then, save the text with the yaml file extension to the home directory from where the Kubernetes clusters are accessible.

  2. To create the eca-remove-backupservercert.yaml file, run the command: kubectl create -f eca-remove-backupservercert.yaml
  3. Once the object is created, then you need to check the custom resource status. If failed, then you can take necessary actions.

These steps removes the external certificate details with respect to the specified primary server from the local certificate store. The certificate is neither deleted from the system nor from the NetBackup database.

If you want to disable ECA then refer to the Disabling an external CA in a NetBackup domain section in the NetBackup™ Security and Encryption Guide

If you enrolled ECA on the Kubernetes operator for a backup server but later reinstalled the backup server which supports just NBCA. Then, you have to remove ECA enrolment from Kubernetes operator because during nbcertcmd communication with backupserver CA support might get compared and if it mismatches then an error occurs.

Updating the ECA signed certificate

There are certain options that are configurable in ECA. You can configure these options through the update operations.

To update the ECA signed certificate

  1. Create a backupservercert object with operation type update.

    eca-update-backupservercert.yaml file looks like this:

    apiVersion: netbackup.veritas.com/v1
    kind: BackupServerCert
    metadata:
      name: backupservercert-eca-update
      namespace: kops-ns
    spec:
      clusterName: cluster.sample.com:port
      backupServer: backupserver.sample.domain.com
      certificateOperation: Update
      certificateType: ECA
      ecaAttributes:
        ecaUpdateOptions:
          ecaCrlCheck: DISABLE | LEAF | CHAIN 
          ecaCrlRefreshHours: [0,4380]
    • Copy the eca-update-backupservercert.yaml file text.

    • Open the text editor and paste the yaml file text.

    • Then, save the text with the yaml file extension to the home directory from where the Kubernetes clusters are accessible.

  2. To create the eca-update-backupservercert.yaml file, run the command: kubectl create -f eca-update-backupservercert.yaml
  3. The ECA_CRL_CHECK option lets you specify the revocation check level for external certificates of the host. It also lets you disable the revocation check for the external certificates. Based on the check, the revocation status of the certificate is validated against the Certificate Revocation List (CRL) during host communication. For more information, refer to the ECA_CRL_CHECK for NetBackup servers and clients section in the NetBackup™ Security and Encryption Guide.
  4. The ECA_CRL_REFRESH_HOURS option specifies the time interval in hours to download the CRLs from the URLs that are specified in the peer host certificate's Certificate Revocation List distribution points (CDP). For more information, refer to the ECA_CRL_REFRESH_HOURS for NetBackup servers and clients section in the NetBackup™ Security and Encryption Guide