NetBackup™ Web UI Cloud Administrator's Guide
- Managing and protecting cloud assets
- Configure Snapshot Manager's in NetBackup
- Managing intelligent cloud groups
- Protecting cloud assets or intelligent cloud groups
- About protecting Microsoft Azure resources using resource groups
- About the NetBackup Accelerator for cloud workloads
- Protecting PaaS assets
- Installing the native client utilities
- Add credentials to a database
- Recovering cloud assets
- Performing granular restore
- Troubleshooting protection and recovery of cloud assets
Troubleshooting
When you trigger a subsequent (twice) restore operation on the same VM, an error occurs during restore operation. This error causes the following issues:
The tags from original OS disk are not copied to newly created restored OS disk.
User login might fail after the VM restore due to ssh failure.
Workaround:
Check if the ssh daemon is running on the system. If not, then perform the steps that are mentioned in the the following document provided by Microsoft:
learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/troubleshoot-ssh-connection
If you try to restore files or folders from partition or file system that are not supported by Snapshot Manager, then you get the following error in the restore job.
Error nbcs (pid=<processs id>) Failed to restore file(s) and folder(s) from snapshot for asset <asset name>
Workaround:
If you want to avoid listing files or folder while browsing for single file restore, which are not supported by Snapshot Manager, then enable CP DISKMAP check by setting below flag in bp.conf
file of NetBackup master server.
CP_DISKMAP_CHECK = true/yes
Backup from restore operation is partially successful when disk is full on the selected target directory. Following messages are displayed:
Dec 29, 2022 2:57:51 PM - Info nbcs (pid=2244) Granular restore(SFR) is completed Dec 29, 2022 2:57:51 PM - Info nbcs (pid=2244) Summary of SFR Operation - Success files/folders count: 0 , Failed files/folders count: 1 , Warning files/folders count: 0, Skipped files/folders count: 0 Dec 29, 2022 2:57:51 PM - Info nbcs (pid=2244) Detailed restore summary report is available on recovery target host at location: /var/log/flexsnap/restore/granular-restore-09b4d44d . . Dec 29, 2022 2:57:51 PM - Warning bprd (pid=1977) Granular Restore from backup completed with error. Copy the files manually from live access mount: ip-10-239-185-241:/mnt/vpfs_shares/vmfiles/8fcc/8fcc132b-a202-49a8-b654-81ff242a718a/livemount Dec 29, 2022 2:57:51 PM - end Restore; elapsed time 0:01:51 the requested operation was partially successful(1)
For backup from restore if live mount is created successfully then even though there is other error reported apart from ASSET_NOT_FOUND, its considered as partial success. If there is no network devices/filesystem mounted on target location and disk is full then below messages can be seen in job details:
Jan 02, 2023 12:11:16 AM - Error nbcs (pid=13934) 187776K space required for file/folder restore while 20K is total available space on /disk1
In this case other network devices/file system must have been mounted on the target path, hence Snapshot Manager agent considers the free space on device/file system. But once it tries to copy the file it fails with space error logged into summary report. For example,
/var/log/flexsnap/restore/granular-restore-09b4d44d in above Job details log
Workaround:
Check the summary report on target host location. For example,
/var/log/flexsnap/restore/granular-restore-09b4d44d [root@ip-10-239-187-148 granular-restore-09b4d44d]# cat root-error.log Dec 29 09:27:44: ERROR - FILE: /disk1/dl380g9-149-vm15_package.zip [Error 28] IOError: No space left on device
If the file copy operation failed due to disk space, then create some space and copy the file from live mount.
The live mount path details can be found in the job details as follows:
Dec 29, 2022 2:57:51 PM - Warning bprd (pid=1977) Granular Restore from backup completed with error. Copy the files manually from live access mount: ip-10-239-185-241:/mnt/vpfs_shares/vmfiles/8fcc/8fcc132b-a202-49a8-b654-81ff242a718a/livemount
Partial recovery could be due to the following reasons:
If the target virtual machine is disconnected (no connectivity through agent).
If any failure is seen while copying files/folders on target virtual machine.
When a window virtual machine content is restored on Linux target virtual machine.
In such partial recovery cases, the created instance access will not be deleted and will be available for next 24 hours.
The instance access retention interval is configurable by setting bp.conf
file. (Default value is 24hrs).
Workaround:
User can perform manual steps to access the Instant access share on target host and then manfully copy the required files/folders.
(Copy files over NFS) To restore Linux image content on Linux host:
To mount an NFS share on a Linux system, install the NFS client package using the following command:
$ sudo yum install nfs-utils
Using the following mount command, mount the Instant access on target Linux host:
# Create a directory say /mnt/restore $ mkdir -p /mnt/restore # Mount the instant access $ mount -t nfs <InstantAccessServer:InstantAccessPath> /mnt/restore
Instant access path can be retrieved from activity manager logs which is in the following format:
<InstantAccessServer>:/mnt/vpfs_shares/vmfiles/<id>/<InstantAccessId>/livemount
(SMB access) To restore Windows image contents (with ACL) on Windows target host:
SMB credentials of MSDP storage server of source virtual machine image must be added to the Windows credential manager.
Use the given live mount to access virtual hard disks by navigating to
.The virtual hard disks are listed under the folder with vhd_ as the prefix.
On the Disk Management dialog box under the
tab, attach the required virtual hard disk and click on .On the Add Drive Letter or Path dialog box, select
option to assign the letter to Virtual Disk to browse data and click on .Navigate to the assigned drive in the above step and copy the data manually.
(Live mount) To restore windows image contents on Linux target host:
Linux must have the cifs package. Obtain the packages using the # yum install cifs-utils command.
Create the mount directory using the # mkdir <my_mount_dir> command.
Mount exported path using Samba username and password as follows:
mount -t cifs -o username=<sambauser> //<InstantAccessServer>/<InstantAccessPath> <my_mount_dir>
Copy the files using the following command:
# cp <my_mount_dir>/<file_path> <target_dir_path>
Issue/Error | Description | Workaround |
---|---|---|
Log path to check | For information related to restore details on target host, check the following logs:
| To resolve the failures or exceptions occurred during single file restore on Snapshot Manager, refer to the following logs on Snapshot Manager host:
|
Pre-recovery check fails | When restoring files and folders to disconnected target virtual machine, the pre-recovery check fails with the following error: Target VM state: Target VM <vm_name> has no agent configured If recovery is started, restore operation would be partially successful. | Ensure that the target virtual machine is in connected state with agent configured for successful restore. |
Partial recovery for source Linux VM to target Windows VM (no NFS client) | When restoring files and folders from source Linux VM to target Windows VM without installing NFS client on Windows target machine, then the recovery is partially successful with the following error displayed: Error nbcs (pid=42513) Invalid operation for asset: <asset_id> Warning bprd (pid=42045) Granular Restore from backup completed with error. Copy the files manually from live access mount: <livemount_path>. Note that live access mount is available only for 24 hrs. | Install NFS client on window target machine before you perform restore from Linux VM to Windows VM. |
Restore job fails for deleted target VM | Restore job fails with the following error when restoring files and folders on target VM which is deleted from cloud environment: Error nbcs (pid=44859) Target VM not found, asset_id <asset_id> | Select a different target VM. |
Create Instant access fails | Creation of Instant access fails during restore job if Instant access is not enabled on MSDP storage server. | Verify if Instant access is supported on MSDP media server by using the following pre-check script: /usr/openv/pdde/vpfs/bin/ia_byo_precheck.sh |
Target VM does not have free drives to attach Virtual Disk | If the number of volumes containing the selected files are more than the number of free available drives on target host then the operation fails. | Restore files and folders from less number of volumes. |
Not enough space: '*\\driverMapping.json | If media server where MSDP is configured has FIPS enabled. | Disable FIPS on the media server where MSDP is installed or add domain user samba credentials to target VM. |
If one of VM's disks is not initialized, downloading or restoring VM files using Instant access fails with the following error:
Jan 24, 2023 11:58:47 AM - Error NBWMC (pid=3716) Internal Error: ('failed to find operation system information, please check the source VM', ('Failed to expose VMDK', 1006), None) Failed to create the instant access mount. (4001)
Libguestfs is a third party tool used by Instant access to retrieve files from VM backup. If a disk is not initialized, libguestfs will not be able to handle it.
Workaround:
Initialize the disk, backup the VM and try to download or restore VM files using Instant access again.