Search <book_title>...

NetBackup™ Troubleshooting Guide

Last Published: 2024-09-30

Product(s): NetBackup & Alta Data Protection (10.5)

Troubleshooting issues with malware scanning

Failed to get response from NetBackup malware utility

(Applicable on scan host RHEL 8.x and NFS version 4.x) When scanning large size backup (~ 200 million files), following error is displayed on the Web UI for failed job:

Failed to get response from NetBackup malware utility.

While scan is in progress on scan host, NFS mount points are not accessible from scan host. Scan job remains in progress and timeout after two days. NFS exports on storage server are accessible.

Workaround: Ensure that you use NFS version 3 for mounting IA mounts on scan host over NFS by setting the following configuration in /etc/nfsmount.conf file on scan host:

# grep Defaultvers /etc/nfsmount.conf Defaultvers=3

Failed to connect to the scan host

SSH connection to scan host from media server failed.

Workaround: Verify the following scan host credentials:

RSA (SHA256) key
User name
Password

Refer to NetBackup Web UI Administrator's Guide for the scan host configuration.

Failed to determine the scan host OS

Error can be due to unsupported scan host.

Workaround: For a complete list of supported platforms for the scan host, refer to the Software Compatibility list document.

Failed to copy NetBackup malware utility to the scan host

Not enough space is available on the scan host.
SSH user does not have access to the required directories on the scan host.

Workaround

On a Windows scan host, check for space availability in C:\ folder.
On a Linux scan host, check for space availability in /tmp folder.

Failed to get the scan host credentials

Media server is not able to fetch the credentials to access scan host from the Primary.

Workaround: Check that credentials for scan host are specified.

Time-out has occurred during the scan

Default scan operation time out is two days. Time to scan may vary depending on the factors sch as workload type, network bandwidth, backup size.

Workaround: Scan time-out is configurable and can be changed by setting the MALWARE_SCAN_OPERATION_TIMEOUT configuration key.

Minimum value: 1 hour
Maximum value: 30 days

Failed to get response from NetBackup malware utility

Mismatch between nbmalwareutil binary and the ScanManager

Workaround:

Contact NetBackup support.

Failed to launch the scanner

Malware scanner-specific failure message.

Workaround: Refer to nbmalwarescanner logs on the media server.

Failed to mount the backup image

IA share is not accessible from the scan host.

Workaround: Check IA configuration on storage server. Verify on activity monitor that IA job is successful.

Failed to unmount the backup image

IA share is busy or not accessible.

Workaround: Refer to nbmalwarescanner logs on the media server.

Failed to run a scan

Generic failure during the scan of a backup image.

Workaround: Refer to nbmalwarescanner logs on the media server.

Instant access mount created but not deleted by malware scan

Generic failure during the scan of a backup image.

Workaround:

Verify if any scan is in progress.
If no scan is in progress, then obtain the list of such instant access mounts with ID's of the instant access mount created using the GET IA API from the following directory:
/netbackup/recovery/workloads/{workload}/instant-access-mounts
Using the DELETE API, delete the instant access mount:
/netbackup/recovery/workloads/{workload}/instant-access-mounts/{mounId}

All mount drives are exhausted

Only five backup images can be mounted at the same time on windows scan host.

Workaround:

Ensure that scan host is not part of multiple NetBackup domains.
Check if there are any Stale mounts on the scan host by running net use.
Following drive letters are used for mounting the IA shares on the windows scan host. Ensure that they are not in use. L:\ M:\ N:\ O:\ P:\

Either the Windows Defender is not installed or the environment variable is not set

Microsoft Windows Defender is not installed on the scan host or not configured properly.

Workaround: Ensure that Microsoft Windows Defender is installed on scan host.

Refer NetBackup Web UI Administrator's Guide for the scan host configuration.

Either Symantec protection engine is not installed or the environment variable is not set

Symantec Protection Engine is not installed on the scan host or not configured properly.

Workaround: Ensure that Symantec Protection Engine is installed on scan host.

Refer NetBackup Web UI Administrator's Guide for the scan host configuration.

Failed to perform malware scan of the backup image

Generic error for Scan failure.

Workaround: Contact NetBackup support.

Net bios name can be at most 15 chars long

Storage server host name cannot be more than 15 characters for the SMB share support.

If Windows Server 2016 is used to set up Active Directory domain, then it does not allow a connection to a storage server with host name of length more than 15 characters.

Workaround: Ensure that the character limit is not more than 15 characters.

Failed to run a scan

Generic failure during scanning backup image.

Workaround: Check for the following errors:

Refer to nbmalwarescanner logs on the media server.
Check for space on media server storage.
Check for NFS service failure on media server.

Too many infected files in the selected time range

Review the nbmalwarescanner to view the infected files list for the backup images in the selected date range.

Workaround: Update the date range or recovery files and folders selection to reduce the number of infected files. Retry the operation. You can also perform one of the following:

Select the Allow recovery of files impacted by malware option which can be used to recover selective clean files.
Skip that backup image from recovery.

Large number of infected files

There are too many infected files in the selected scan result. If the scan result has infected files greater than 5000, the following message is displayed:
Large number of infected files. To view the complete list of infected files, export the list.
Workaround: Export the infected file list in .csv format and download it to view it.
There are many infected files in the selected scan result or the infected file paths are long to be captured in the database. Following error message is displayed:
Large number of infected files.
Workaround: This result cannot be exported or viewed.
: As the results cannot be exported or viewed, review the scan logs to view a detailed list of the infected files for the selected scan result.

Scan operation is divided into parts

For large size backup, scan operation is divided into parts. For example, if total number of files in the backup are 1,000,000, the scan operation will be divided into two parts of 500,000 files each.

Each part would be created and scanned separately. Each part can be assigned with different scan host. The Malware detection UI displays only single entry for backup.

Workaround: Each divided part details can be obtained by using the REST API.

The NB_MALWARE_SCANNER_PATH environment variable is missing

When performing a malware scan operation with the NetBackup Malware Scanner installed on the scan host, it fails with the following error message:

Missing environment variable NB_MALWARE_SCANNER_PATH

Workaround: Ensure that NetBackup Malware Scanner is installed. Note the install location.

Login on the scan host as user using the same user credentials that were provided during scan host configuration on the primary server. Add the following lines to ~/.bashrc:

export NB_MALWARE_SCANNER_PATH=<installLocation>/savapi-sdk-linux64/bin

export PATH=$PATH:$NB_MALWARE_SCANNER_PATH

Issues related to space and directory access on the scan host

Error/Issue	Description	Workaround
Failed to open the file. Unable to create a directory. Failed to generate the result file. Failed to open the output file. Unable to create directory for result file. Failed to open the result file. Unable to create mount destination directory. Unable to create directory for a log file.	Not enough space is available on the scan host. SSH user does not have access to the required directories on the scan host.	On a Windows scan host, check space availability in `C:\` On a Linux scan host check space availability in `/tmp`

Issue related to NAS-Data-Protection

When upgrading NetBackup from previous version to NetBackup version 10.2.1 or later with the following options selected, the No images match the search criteria message is displayed:

Options	Fields
Search by: Backup images	Policy type: NAS-Data-Protection Copies: Copy2 Malware scan status: Not scanned (Default)
Search by: Assets by policy type	Policy type: NAS-Data-Protection Copies: Copy2 Scanner host pool: Select the required scanner host pool. Malware scan status: Not scanned (Default)

Options

Fields

Search by: Backup images

Policy type: NAS-Data-Protection

Copies: Copy2

Malware scan status: Not scanned (Default)

Search by: Assets by policy type

Policy type: NAS-Data-Protection

Copies: Copy2

Scanner host pool: Select the required scanner host pool.

Malware scan status: Not scanned (Default)

Workaround

To view the images that are backed up, ensure that you select the Malware scan status option as All to scan the NAS-Data-Protection backup images created on earlier version of NetBackup media server.

Issues in scan performance

When using Instant Access mount points for malware scan (traditional malware scan) in NetBackup versions prior to 10.3, performance issues were observed.

Workaround: Upgrade to NetBackup media and storage server 10.3 or later. NetBackup 10.3 introduces the dynamic scan feature. This improves the instant access time as well as the scan performance.

The following table provides the differences between the traditional malware scan and dynamic scan:

Key scanning procedure	Traditional malware scan using Instant Access mount points	Dynamic scan
Instant access stage.	Analyzes the tar stream and builds each file's header and extent map file (LMDB database), which is time consuming for large number of files in the backup.	Restores TIR (catalog database) and IM (image metadata) information from fragment.
Instant access share (NFS/SMB) is mounted and user tries to list or access the file.	Accesses it's header file and reads the attribute from it.	Query's the directory from catalog database to get all the files and directories which are under this directory. It can also query each files and directories attribute to the output.
Scan host opens a file	Opens and loads the LMDB database.	Builds the index in memory and reads directly from data container. To get file's extent by locating and reading the tar header and analyze the content. To get SO list (PureDisk only) by searching the SO list from fragment FP map To build mapping table by inserting the SO list (PureDisk only)
Scan host reads a file	Searches from LMDB database and reads from data container.	If storage server is 3rd party storage vendor, it reads data through OST interface directly. If storage server is PureDisk, it searches from mapping table and reads data from data container.