How to troubleshoot NetBackup for VMWare vSphere backup/restore in NetBackup 8.x, 9.x, 10.x in Windows environment
Problem
How to troubleshoot NetBackup for VMWare vSphere backup/restore in NetBackup 8.x, 9.x, 10.x in Windows environment
Solution
NetBackup now uses the vSphere vStorage api to perform backups and restore directly to the ESX or Virtual Center server. With this new method comes some changes in our troubleshooting procedure which will be seen below.
Here are the logs to review and the event to review them in:
Browsing for Virtual Machine:
Create a DWORD (32-bit) with name VERBOSE and value of 5 in
HKEY_LOCAL_MACHINE\SOFTWARE\Veritas\NetBackup\CurrentVersion\Config
INSTALL_PATH\Veritas\NetBackup\logs\bpVMutil\<date>.log
Parent Job: (Snapshot Job)
INSTALL_PATH\Veritas\NetBackup\logs\bpcd\ <date>.log
Child Job: (Backup Job)
TAR is now nbrestore for VMware Restores:
OID 357 - /usr/openv/logs/ncfnbrestore
To configure ncfnbrestore run:
vxlogcfg -a -p NB -o 356 -s DebugLevel=6
vxlogcfg -a -p NB -o 357 -s DebugLevel=6
To view nbrestore for the last 30 minutes run:
vxlogview -p NB -o 356 -i 357 -d all -t 00:30:00 > nbrestore.txt
Modify the following registry on the VMware Backup Host to gather additional information in the NetBackup log perform the VMWare task.
Using vmcloglevel 3 will show API errors and 6 is used for debugging the API code. Level 3 should be sufficient for technical support.
Additional information on configuring VxMS logging:
To configure VxMS logging on a Windows backup host
Create the VxMS log directory:
install_path\NetBackup\logs\vxms
Note: For logging to occur, the VxMS folder must exist.
In the Windows registry, create the DWORD registry entry VXMS_VERBOSE in the following location:
HKEY_LOCAL_MACHINE>SOFTWARE>Veritas>NetBackup>CurrentVersion>Config
For possible VDDK issues:
VXMS_VERBOSE = 5
For Mapping issues during the VMware Backup, use:
VXMS_VERBOSE = 8
Note: VXMS_VERBOSE = 8 will slow down backups and restore operations.
By default VDDK logs are embedded into VxMS logs. To collect VDDK logs separately, then add the following registry entry:Add "createVDDKlog" DWORD entry under "HKEY_LOCAL_MACHINE\SOFTWARE\VERITAS\VxMS\" and set its value to 1. This will generate VDDK-*.log files in the VxMS log folder.
To change the log location:
Open regedit and go to the following location:
HKEY_LOCAL_MACHINE>SOFTWARE>Veritas>NetBackup>CurrentVersion
Create the registry entry vxmslogdir with a string value (REG_SZ). For the string value, specify the full path to an existing folder.
Note: You can use NTFS compression on VxMS log folders to compress the log size. The new logs are written in compressed form only.
Logging for Query based policy:
VMware Intelligent Policy query will be logged in the following unified log:
ncfnbcs - OID 366
ncfvmwarepi - OID 386
To enable the logging
INSTALL_PATH\Veritas\NetBackup\bin\vxlogcfg -a -p 51216 -o 366 -s DebugLevel=6
INSTALL_PATH\Veritas\NetBackup\bin\vxlogcfg -a -p 51216 -o 386 -s DebugLevel=6
The following command will return the last 5 minutes of the above logs:
INSTALL_PATH\Veritas\NetBackup\bin\vxlogview -p NB -i ncfnbcs -t 00:05:00
NBDISCOVERY will also be logged in the following Unified Log:
ncfnbdiscover - OID 377
INSTALL_PATH\Veritas\NetBackup\bin\vxlogcfg -a -p 51216 -o 377 -s DebugLevel=6
The following example will return the last 5 minutes of a nbdiscovery command:
# INSTALL_PATH\Veritas\NetBackup\bin\vxlogview -i 377 -t 00:05:00 >
For Instant Recovery Logging
Enable bprd logging on Master Server
Create folder bprd in INSTALL_PATH\Veritas\NetBackup\Logs
They are 3 main steps performed during Instant Recovery:
- Export a backup image
- Mount a backup image
- Activate VM
If any of these steps failed “Activate Instant Recovery” job logs will have details about the failure and error code.
If Export a backup image fails look at INSTALL_PATH\Veritas\NetBackup\nbfsd on media server.
Mount backup image fails look at INSTALL_PATH\Veritas\NetBackup\nbfsd on VM Proxy host and also the Virtual Center Task and Events.
Activate VM fails look at INSTALL_PATH\Veritas\NetBackup\bpVMutil on VM Proxy host and also the Virtual Center Task and Events.