Best practices and performance tuning for optimized duplication to MSDP-Cloud cloud storage if the data is very large

Problem

If the data is very large, which means there are many images which belong to a parent job, and the total size of the images is more than 100TB, the opt-dup (optimized duplication) from local MSDP to cloud storage may encounter a performance issue which causes the duplication jobs to need more time than the original operations.

Error Message

No error message, but the job time could be very different.

For example, an Oracle backup policy can generate more than 5000 images in one FULL database-only backup, and the average size of image is 30GB. The time needed for a duplication job can be changed from 25 minutes to several hours, with one image needing 2 hours:

<NBU>   9:08:12 - Info bpdm (pid=32438) Waiting for storage server storageserver, disk volume PureDiskVolume
<NBU>  10:56:26 - Info bpdm (pid=32438) Transferred 32083545174 bytes, to storage server storageserver, disk volume PureDiskVolume
<NBU>  11:05:27 - end writing; write time: 1:57:16

Cause

When backups or duplications to cloud storage are performed under MSDP-Cloud, the MSDP global cache is not used and a local fingerprint cache will be used instead. The MSDP-Cloud will try to load the cache from LastImages, but if LastImages is too large, there can cause the performance issue when the cache is loading.

Solution

A. If possible, split the policy if possible to reduce the total number and size of the images. Ideally, it is suggested to have less than 200 images for each policy in a FULL backup.

B. Apply the following changes to attempt to balance the local and cloud cache size to improve the performance:

1. Apply the latest MSDP EEB bundle for the installed version of NetBackup.

2. Add the following parameter to the [Cache] section of the contentrouter.cfg file:

• CloudBits=27

 3. Change the parameters below in contentrouter.cfg:

•  MaxCacheSize=50% to 35%
•  MaxCloudCacheSize=20% to 35%

The contentrouter.cfg file resides in the following directories:

• (UNIX) storage_path/etc/puredisk
• (Windows) storage_path\etc\puredisk