NetBackup™ Deployment Guide for Kubernetes Clusters
- Introduction
- Section I. Configurations
- Prerequisites
- Recommendations and Limitations
- Configurations
- Configuration of key parameters in Cloud Scale deployments
- Section II. Deployment
- Section III. Monitoring and Management
- Monitoring NetBackup
- Monitoring Snapshot Manager
- Monitoring MSDP Scaleout
- Managing NetBackup
- Managing the Load Balancer service
- Managing PostrgreSQL DBaaS
- Performing catalog backup and recovery
- Managing MSDP Scaleout
- Section IV. Maintenance
- MSDP Scaleout Maintenance
- PostgreSQL DBaaS Maintenance
- Patching mechanism for Primary and Media servers
- Upgrading
- Cloud Scale Disaster Recovery
- Uninstalling
- Troubleshooting
- Troubleshooting AKS and EKS issues
- Troubleshooting AKS-specific issues
- Troubleshooting EKS-specific issues
- Troubleshooting AKS and EKS issues
- Appendix A. CR template
Environment backup
Note down the MSDP operator Namespace, NodeSelector, StorageClassName, Tolerations and Image tag as follows:
Obtain the name of the msdp operator statefulset using the following command:
kubectl get statefulset -n <msdp-operator-system-namespace>
Use the following command to backup MSDP operator Image tag, Tolerations, and NodeSelector:
kubectl get sts <msdp-operator-statefulset-name> -n <msdp-operator-sample-namespace> -o=jsonpath='{"Namespace :"}{$.metadata.namespace}{$"\nImage :"}{$.spec.template.spec.containers[0].image}{$"\nNodeSelector :"}{$.spec.template.spec.nodeSelector}{$"\nTolerations :"}{$.spec.template.spec.tolerations[2]}{$"\nStorageClassName :"}{$.spec.volumeClaimTemplates[0].spec.storageClassName}{$"\n"}'
From the output, note down the Image tag, StorageClassName, Tolerations and NodeSelector:
Sample Output: Namespace :msdp-operator-system Image :nbuk8sreg.azurecr.io/msdp-operator:20.4 NodeSelector :{"agentpool":"nbuxpool"} Tolerations :{"key":"agentpool","operator":"Equal","value":"nbuxpool"} StorageClassName :managed-csi-hdd
If toleration is not provided for msdp operator, then use the following command:
kubectl get sts <msdp-operator-statefulset-name> -n <msdp-operator-sample-namespace> -o=jsonpath='{"Namespace :"}{$.metadata.namespace}{$"\nImage :"}{$.spec.template.spec.containers[0].image}{$"\nNodeSelector :"}{$.spec.template.spec.nodeSelector}{$"\nStorageClassName :"}{$.spec.volumeClaimTemplates[0].spec.storageClassName}{$"\n"}'
Sample Output: Namespace :msdp-operator-system Image :nbuk8sreg.azurecr.io/msdp-operator:20.4 NodeSelector :{"agentpool":"nbuxpool"} StorageClassName :managed-csi-hdd
Note down the NetBackup operator Namespace, NodeSelector, Tolerations and Image tag as follows:
Obtain the name of the NetBackup operator deployment using the following command:
kubectl get deployment -n <netbackup-operator-system-namespace>
Use the following command to backup NetBackup operator Image tag, Tolerations, and NodeSelector:
kubectl get deployment <netbackup-operator-deployment-name> -n <netbackup-operator-system-namespace> -o=jsonpath='{"Namespace :"}{$.metadata.namespace}{$"\nImage :"}{$.spec.template.spec.containers[0].image}{$"\nNodeSelector :"}{$.spec.template.spec.nodeSelector}{$"\nTolerations: "}{$.spec.template.spec.tolerations}{$"\n"}'
From the output, note down the Image tag, Tolerations and NodeSelector:
Sample Output: Namespace :netbackup-operator-system Image :nbuk8sreg.azurecr.io/netbackup/operator:10.4 NodeSelector :{"agentpool":"agentpool"} Tolerations: [{"key":"agentpool","operator":"Equal","value":"agentpool"}]
Note down the flexsnap-operator Namespace, NodeSelector, Tolerations and Image tag as follows:
Obtain the name of the flexsnap-operator deployment using the following command:
kubectl get deployment -n <netbackup-operator-system-namespace>
Use the following command to backup flexsnap operator Image tag, Tolerations, and NodeSelector:
kubectl get deployment <flexsnap-operator-deployment-name> -n <netbackup-operator-system-namespace> -o=jsonpath='{"Namespace :"}{$.metadata.namespace}{$"\nImage :"}{$.spec.template.spec.containers[0].image}{$"\nNodeSelector :"}{$.spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0]}{$"\nTolerations :"}{$.spec.template.spec.tolerations}{$"\n"}'
From the output, note down the Image tag, Tolerations and NodeSelector:
Sample Output: Namespace :netbackup-operator-system Image :nbuk8sreg.azurecr.io/veritas/flexsnap-deploy:10.4 NodeSelector :{"key":"agentpool","operator":"In","values":["agentpool"]} Tolerations :[{"effect":"NoSchedule","key":"agentpool","operator":"Equal","value":"agentpool"}]
(For DBaaS) Note the FQDN of the Postgres server created.
Note the Postgres unified container image tag, containerPort (use only in case you created unified container):
k get statefulset.apps/nb-postgresql -n <sample-namespace> -o=jsonpath='{$"\nImage :"}{$.spec.template.spec.containers[0].image}{$"\ncontainerPort :"}{$.spec.template.spec.containers[0].ports[0].containerPort}{$"\n"}'
Sample output:
Image :cpautomation.azurecr.io/netbackup/postgresql:10.4 containerPort :13787
Save the environment CR as follows:
Obtain the name of environment using the following command:
kubectl get environment -n <sample-namespace>
Save the environment yaml file:
kubectl get environment <environment-name> -n <sample-namespace> -o yaml> environment_backup.yaml
For example, kubectl get environment environment-sample -n example-ns -o yaml> environment_backup.yaml
Note down and save the following values (names) of the secrets obtained from
environment_backup.yaml
file in the above step:credSecretName, kmsDBSecret, drInfoSecretName, dbSecretName, keySecret, secretName (Msdp credential), secretName (s3Credential), secretName (Snapshot Manager credential)
For example,
credSecretName: primary-credential-secret
Save the secrets yaml file as follows:
kubectl get secret <secret-name1> <secret-name2> <secret-name3> -n <sample-namespace> -o yaml > secret_backup.yaml
For example, kubectl get secret primary-credential-secret kms-secret example-key-secret -n example-ns -o yaml > secret_backup.yaml
Note:
The dbSecretName, drInfoSecretName, secretName (s3Credential) fields are optional. Skip this step if these fields are not present in
environment_backup.yaml
file.Save the secrets named as Msdp credential and drInfoSecret during creation. As the operator would delete these secrets after using it.
MSDP credential: Step 2 in the following section:
drInfoSecret: Step 2 in the following section:
Note the values (names) of the secretProviderClass.
For example,
dbSecretProviderClass: db-secret-provider-class
Save the secretProviderClassyaml file using the following command:
kubectl get secretproviderclass <secretproviderclass-name> -n <sample-namespace> -o yaml > secretproviderclass_backup.yaml
Note:
The dbSecretProviderClass is an optional field. If it is not present in the
environment_backup.yaml
file, then skip this step.Note the following values (names) of configMap from
environment_backup.yaml
file saved in step 1 above:emailServerConfigmapName, proxySettings
For example,
emailServerConfigmapName: email-server-configuration
Save the configMaps yaml using the following command:
kubectl get configmap <configmap-name1> <configmap-name2> <configmap-name3> -n <sample-namespace> -o yaml >configmap_backup.yaml
For example, kubectl get configmap email-server-configuration -n example-ns -o yaml > configmap_backup.yaml
Note:
The emailServerConfigmapName and proxySettings are optional. If these are not present in
environment_backup.yaml
file, then remove those from the above command.Save internal configmap yaml using the following command:
kubectl get configmap nbu-media-autoscaler-configmap flexsnap-conf nbuconf -n <sample-namespace> -o yaml > internalconfigmap_backup.yaml
Note:
The
nbu-media-autoscaler-configmap
is an optional internal configmap. If it is not present in environment namespace, then removenbu-media-autoscaler-configmap
from the above command.Save the value of emailServerConfigmap. The operator would delete this configmap after using it.
Note the details of cloud STU used for MSDP storage, such as name of bucket, volume, credential and the respective details added through Credential management in UI.
(Applicable only for DBaaS based deployment environment) Snapshot Manager backup steps:
For AKS
Search the disk (PV) to which psql pvc is attached in Azure cloud portal and click on
in the different resource group other than the cluster infra resource group and note down this resource group. Wait for the resource to be available.Note:
Snapshot must be created in resource group in different availability zone to take care of the recovery in case of zone failures/corrupted.
Save the
pgsql-pv.yaml
file:kubectl get pv | grep psql-pvc
pvc-079b631e-a905-4586-80b5-46acc7011669 30Gi RWO Retain Bound nbu/psql-pvc managed-csi-hdd 3h10m
kubectl describe pv <PV which is bound to psql-pvc> > pgsql-pv.yaml
For example, kubectl describe pv pvc-079b631e-a905-4586-80b5-46acc7011669 > pgsql-pv.yaml
Note down the snapshot id, which would be used to create a disk from snapshot during recovery.
Note:
Disk Snapshot must be taken after every plugin addition as the latest database is required to recover all the plugins during Database recovery.
For EKS
Describe the PV attached to psql-pvc and save the VolumeID (for example,
vol-xxxxxxxxxxxxxxx
), storage class name and availability zone (AZ) from the output of following command:kubectl get pv | grep psql-pvc
pvc-079b631e-a905-4586-80b5-46acc7011669 30Gi RWO Retain Bound nbu/psql-pvc managed-csi-hdd 3h10m
kubectl describe pv <PV which is bound to psql-pvc> > pgsql-pv.yaml
For example, kubectl describe pv pvc-079b631e-a905-4586-80b5-46acc7011669 > pgsql-pv.yaml
Search above VolumeID in the
in AWS cloud portal.Create snapshot (expand the
drop down) from the volume and wait for the completion. Note down the snapshot id (for example,snap-xxxxxxxxxxxx
)Note:
Disk Snapshot must be taken after every plugin addition as the latest database is required to recover all the plugins during Database recovery.