Search <book_title>...

NetBackup™ Deployment Guide for Kubernetes Clusters

Last Published: 2024-10-04

Product(s): NetBackup & Alta Data Protection (10.5)

Managing fluentbit for logging strategy

You need to log in into the fluentbit collector pod to view, extract or to resize the log volumes.

Table: Managing logs of fluentbit

Actions	Command / Description
View logs:	First find the collector pod: $ kubectl get pod -n netbackup \| grep fluentbit-collector nb-fluentbit-collector-7465d6fd75-g5j9x 2/2 Running 0 126m Exec into the collector pod using the pod name with the unique identifier from the previous command: $ kubectl exec -it -n netbackup nb-fluentbit-collector-7465d6fd75-g5j9x -- /bin/bash Move into the fluentbit log location: $ cd /usr/openv/fluentbit/logs Folders are labeled by date: $ ls 2024-02-02 2024-02-03 Move into the folder of the date you are looking at and the logs are grouped by namespace:$ cd 2024-02-03/ $ ls netbackup netbackup-operator-system kube-system trust-manager Move into the folder of the namespace you are looking at and the logs are in there labeled by pod: $ cd netbackup/ $ ls bhoffman-nbatd-0 bhoffman-policyjob-0 bhoffman-primary-0 nb-fluentbit-daemonset-2l4sh nb-fluentbit-daemonset-7ndsg nb-fluentbit-daemonset-qlv4m bhoffman-nbwsapp-0 bhoffman-policyjobmgr-0 nb-fluentbit-collector-7dfc4d95b8-mzrkr nb-fluentbit-daemonset-6nj66 nb-fluentbit-daemonset-9pslq nb-fluentbit-daemonset-wpwp4 The fluentbit collector pod has `vi` built into it so you can use that to view the logs in the container itself.
Extract logs: In order to extract the logs out of the collector pod there are a few options. You can tar and compress the logs before extraction or extract them immediately.	(Optional) Tar up the files you want to extract. Select a folder and run: $ tar -cvf <name of tar> <name of folder> Copy the files out of the container. Exit the container using the command: $ kubectl cp -n netbackup <pod-name>:/usr/openv/fluentbit/logs/<folder or tar> <output folder or tar> (Optional) Extract the tar outside the container if necessary: $ tar xvf <output tar>
Resize Log Volumes: It is important that all the non-collector log volumes must remain the same size. This is because they are all using a common variable in the bp.conf file to determine when to be cleaned up and how much size to use.	Except for the primary server, perform this action on every log volume. It opens the edit menu that allows you to change the volume size and save. Upon saving it takes few minutes to resize the volume. In order to resize the log volumes you must run the command: $ kubectl edit pvc -n <netbackup namespace> <pvc name> To resize the primary volume run: $ kubectl edit environment -n <netbackup namespace> <netbackup environment name> It opens an edit menu where you can change the value of spec.primary.storage.log.capacity to a new value (which must be higher). The new values take a few minutes to apply but it will change the log volume size as well as edit the associated bp.conf value. After executing the edit command, run the delete command to restart the pod. kubectl delete pod -n <netbackup namespace> <application pod name>

Actions

Command / Description

View logs:

First find the collector pod: $ kubectl get pod -n netbackup | grep fluentbit-collector nb-fluentbit-collector-7465d6fd75-g5j9x 2/2 Running 0 126m
Exec into the collector pod using the pod name with the unique identifier from the previous command:
$ kubectl exec -it -n netbackup nb-fluentbit-collector-7465d6fd75-g5j9x -- /bin/bash
Move into the fluentbit log location:
$ cd /usr/openv/fluentbit/logs
Folders are labeled by date:
$ ls
2024-02-02 2024-02-03
Move into the folder of the date you are looking at and the logs are grouped by namespace:$ cd 2024-02-03/
$ ls
netbackup netbackup-operator-system kube-system trust-manager
Move into the folder of the namespace you are looking at and the logs are in there labeled by pod:
$ cd netbackup/
$ ls
bhoffman-nbatd-0 bhoffman-policyjob-0 bhoffman-primary-0 nb-fluentbit-daemonset-2l4sh nb-fluentbit-daemonset-7ndsg nb-fluentbit-daemonset-qlv4m bhoffman-nbwsapp-0 bhoffman-policyjobmgr-0 nb-fluentbit-collector-7dfc4d95b8-mzrkr nb-fluentbit-daemonset-6nj66 nb-fluentbit-daemonset-9pslq nb-fluentbit-daemonset-wpwp4
The fluentbit collector pod has vi built into it so you can use that to view the logs in the container itself.

Extract logs:

In order to extract the logs out of the collector pod there are a few options. You can tar and compress the logs before extraction or extract them immediately.

(Optional) Tar up the files you want to extract. Select a folder and run:
$ tar -cvf <name of tar> <name of folder>
Copy the files out of the container. Exit the container using the command:
$ kubectl cp -n netbackup <pod-name>:/usr/openv/fluentbit/logs/<folder or tar> <output folder or tar>
(Optional) Extract the tar outside the container if necessary:
$ tar xvf <output tar>

Resize Log Volumes:

It is important that all the non-collector log volumes must remain the same size. This is because they are all using a common variable in the bp.conf file to determine when to be cleaned up and how much size to use.

Except for the primary server, perform this action on every log volume. It opens the edit menu that allows you to change the volume size and save. Upon saving it takes few minutes to resize the volume. In order to resize the log volumes you must run the command:
$ kubectl edit pvc -n <netbackup namespace> <pvc name>
To resize the primary volume run:
$ kubectl edit environment -n <netbackup namespace> <netbackup environment name>
It opens an edit menu where you can change the value of spec.primary.storage.log.capacity to a new value (which must be higher). The new values take a few minutes to apply but it will change the log volume size as well as edit the associated bp.conf value.
After executing the edit command, run the delete command to restart the pod.
kubectl delete pod -n <netbackup namespace> <application pod name>