Invalid UTF-8 Character Conversion

Problem

During the upgrade to NetBackup 10.2 or greater, string fields in one or more tables in the NetBackup database contain invalid UTF-8 characters. These errors are encountered during the NetBackup database migration although the NetBackup upgrade finishes successfully.

Error Message

WARNING: There were some invalid UTF-8 characters found while migrating the NetBackup database.
                   For details on what to do, please refer to: https://www.veritas.com/support/en_US/article.100055347

Windows:

Linux:

NOTE: The hostnames used in the screenshots are of lab machines for demonstration purpose. 

Cause

The database string field contains non UTF-8 characters in the columns.

Solution

Review the migration logs to ensure proper NetBackup behavior.

1.  If upgrading a Flex NetBackup instance or Linux cluster active node to 10.2 then skip steps 1 and 2 and go to step 3.  If upgrading to a version greater than 10.2 then continue with these steps.
   
   Go to the NetBackup installation temporary location:
   
   Linux: /usr/openv/tmp
   Windows: install_path\NetBackup\temp
   
   Then change to the migrate subdirectory in the NetBackup temp location:
   
   Linux: /usr/openv/tmp/migrate
   Windows: install_path\NetBackup\temp\migrate

2. Check if a file with the name pgsql_utf8_error.txt exists in the migrate directory, and in the NBAZDB and BMRDB subdirectories. The pgsql_utf8_error.txt in the migrate directory is for the NBDB database while the pgsql_utf8_error.txt file in the subdirectories is for corresponding databases NBAZDB and BMRDB.

3.  If upgrading a Flex NetBackup instance or Linux cluster active node to version 10.2 then look for pgsql_utf8_error.txt in the following locations.  If upgrading to a version greater than 10.2 then skip this step.
   
   Flex will have a link for /usr/openv/db/staging on the primary server that is similar to the following:
   
   lrwxrwxrwx. 1 nbsvcusr nbsvcusr   32 Mar 18 22:34 staging ->
/mnt/nbdata/usr/openv/db/staging
   
   Go to the symbolic link target directory, e.g. /mnt/nbdata/usr/openv/db/staging. Check if a file pgsql_utf8_error.txt exists on that directory or NBAZDB or BMRDB subdirectories.

   On a Linux cluster active node check if pgsql_utf8_error.txt can be found on the cluster's shared disk db/staging directory or NBAZDB or NBDB or BMRDB subdirectories.
    
4. The pgsql_utf8_error.txt file contains the names of the columns where the invalid byte sequence for the UTF-8 encoding were found and the name of the table that contains the corresponding the columns. For example:
   
   DBM_MAIN.DBM_DRMedia, FragmentID
   
   Here, DBM_MAIN.DBM_DRMedia is a table name and FragmentID is a column name.
    

5. All the details about each invalid UTF-8 occurrence are recorded in the nbdb log. These details include the table name, the column name, and the actual invalid sequences.
   
   On Windows:   install_path\NetBackup\logs\nbdb or install_path\NetBackup\Temp\Install\nbdb_mmddyy_*
   On Linux:         /usr/openv/netbackup/logs/nbdb or /usr/openv/tmp
   
   Search for the keywords "invalid byte sequence" in nbdb log file to find all the invalid UTF-8 occurrences.

6. All the invalid UTF-8 characters found, are converted to valid UTF-8 characters during the NetBackup database migration. At this point, Veritas recommends to contact Technical Support to review the changes and evaluate if the NetBackup database is affected.