Problem
An SAP backup can be restored to the same host, but fails with status 5 when restored to an alternate/redirected client. None of the files in the file list are restored and backint reports that an invalid copyId is detected.
Error Message
The brrestore output shows that backint failed with status 2 and that none of the file could be located for restore.
BR386E File /oracle/SID/sapdata10/btabd_7/btabd.data7 reported as not found by backup utility
BR386E File /oracle/SID/sapdata11/btabd_11/btabd.data11 reported as not found by backup utility
********************************************************************************
BR280I Time stamp 2012-01-25 16:53:11
BR279E Return code from '/usr/sap/SID/SYS/exe/run/backint -u SID -f restore -i /oracle/SID/sapbackup/.rehsoukg.lst -t file -p /oracle/SID/102/dbs/initSID.utl -c': 2
BR374E 0 of 80 files restored by backup utility
BR231E Backup utility call failed
BR406I End of file restore: rehsoukg.rsb 2012-01-25 16:53:11
BR280I Time stamp 2012-01-23 12.29.45
BR404I BRRESTORE terminated with errors
The backint debug log shows the image lookup processing. An internal data structure is built for each file to be restored and then populated with information from the master server. This is the initial processing for the first file to be restored. brrestore provides the BID and file pathname. backint converts the BID to a ctime, confirms the data structure for the file is not populated, and then queues an image list request to the master server. Notice that the restore is being run by user/group of oradba/sapsys.
16:52:03.401 [26574] <2> initFileQueryDescriptor: Backup id is <VXF1327455057>
16:52:03.401 [26574] <2> initFileQueryDescriptor: Image time of this object is 1327455057
16:52:03.401 [26574] <4> bsa_QueryFile: Query: FILE_BACKUP - /oracle/SID/sapdata10/btabd_7/btabd.data7
16:52:03.401 [26574] <4> bsa_findfile: INF - entering bsa_findfile, BackupId<NULL> FilePath<NULL> FileId</oracle/SID/sapdata10/btabd_7/btabd.data7>
16:52:03.401 [26574] <4> bsa_QueryFile: INF - Object not found in the file list.
16:52:03.402 [26574] <4> bsa_bplist: entering bsa_bplist
16:52:03.408 [26574] <2> logconnections: BPRD CONNECT FROM 3.3.3.3.41408 TO 1.1.1.1.1556 fd = 8
16:52:03.408 [26574] <2> bsa_bplist: start_date = Tue Jan 24 19:30:57 2012
16:52:03.408 [26574] <2> bsa_bplist: end_date = Tue Jan 24 19:30:57 2012
16:52:03.408 [26574] <2> bsa_bplist: Request = oradba sapsys myclient myclient myclient ... 17 ...
16:52:03.408 [26574] <4> bsa_bplist: Filepath = /
The master server returned the following output to backint. Notice that these image entries are owned by user/group of root/root and the permissions allow user/group/other to read each one. But, there should have been two lines of output for each file name, one for the physical location and one for the symbolic location relative to the SAP home directory. Notice that a match for btabd.data7 was not indicated and #NOTEFOUND was returned to brrestore.
16:52:05.013 [26574] <2> dbc_get_string: Output = 0 0 0 0 0 0 31457289 -1 9 -r--r--r-- root root 0 1327455057 1327455856 41 /oracle/SID/sapdata10/btabd_7/btabd.data7
16:52:05.013 [26574] <2> dbc_get_string: Output = 0 0 0 0 0 0 39845902 -1 9 -r--r--r-- root root 0 1327455057 1327456076 43 /oracle/SID/sapdata11/btabd_12/btabd.data12
...snip...
16:52:05.017 [26574] <2> dbc_get_string: Output = EXIT STATUS 0
...snip...
16:52:05.021 [26574] <2> logconnections: BPRD CONNECT FROM 3.3.3.3.41409 TO 1.1.1.1.1556 fd = 8
16:52:38.368 [26574] <4> bsa_QueryFile: INF - None of the objects returned matched the query file: /oracle/SID/sapdata10/btabd_7/btabd.data7
16:52:38.369 [26574] <4> do_file_restore: No object matched the specified predicate.
16:52:38.369 [26574] <4> do_file_restore: #NOTFOUND /oracle/SID/sapdata10/btabd_7/btabd.data7
...snip...
After failing to lookup another file or two, backint fails the entire restore and reports an invalid copyId. The copyId is one of the fields that would have been in the missing lines that were not returned from the master server.
16:53:10.777 [26574] <16> VxBSAGetObject: ERR - Invalid copyId detected - 0.1327455057
16:53:10.777 [26574] <4> do_file_restore: VxBSAGetObject: VxBSA_RC_SUCCESS
16:53:10.777 [26574] <4> do_restore: Exit.
The image database on the master server shows that the BID ctime matches the ctime of this image.
$ cd /usr/openv/netbackup/db/images/myclient/1327000000
mypolicy_1327455057_UBAK
mypolicy_1327455057_UBAK.f
The cat_convert output shows that two lines are actually present in the image for each file that was backed up. The first line is for the physical file and is owned by sapdba/dba with modes that differ from the second line. The second line is for the symbolic link, which does not have restrictive permissions so modes ugo+r are always used.
$ cat_convert -dump mypolicy_1327455057_UBAK.f
...snip...
1 1073741824 41 60 7 1 0 60129542158 65915 /oracle/SID/sapdata10/btabd_7/btabd.data7 33184 sapdba dba 1073741824 1327455015 1327455015 1327455015
0 0 41 48 31457289 9 0 0 -1 /oracle/SID/sapdata10/btabd_7/btabd.data7 292 root root 0 1327455856 1327455856 1327455856
2 1073741824 43 60 31457292 1 0 12884901891 65916 /oracle/SID/sapdata11/btabd_12/btabd.data12 33184 sapdba dba 1073741824 1327455424 1327455015 1327455015
0 0 43 48 39845902 9 0 0 -1 /oracle/SID/sapdata11/btabd_12/btabd.data12 292 root root 0 1327456076 1327456076 1327456076
...snip...
Cause
If the backup is run using the wrong user, then the image lines with modes ugo+r are returned, but the other lines are not because the permissions do not match.
If the backup is run using the correct user (sapdba/dba), then both lines are returned per the modes as shown in this example from a successful restore. Notice also that the match for btabd.data7 is correctly detected.
13:28:10.723 [22040] <2> logconnections: BPRD CONNECT FROM 3.3.3.3.40951 TO 1.1.1.1.1556 fd = 8
13:28:10.724 [22040] <2> bsa_bplist: start_date = Tue Jan 24 19:30:57 2012
13:28:10.724 [22040] <2> bsa_bplist: end_date = Tue Jan 24 19:30:57 2012
13:28:10.724 [22040] <2> bsa_bplist: Request = sapdba dba myclient myclient myclient NONE 11 sap_oracle* 3 999 1327455057 1327455057 4 4 1 0 1 0 17 0 1 3 0 C C C C C 0 2 0 0 0
13:28:10.724 [22040] <4> bsa_bplist: Filepath = /
13:28:11.355 [22040] <2> dbc_get_string: Output = 0 0 0 0 0 0 1 -1 9 -r--r--r-- root root 0 1327455057 1327455064 13 /PROXY_BACKUP
13:28:11.355 [22040] <2> dbc_get_string: Output = 1073741824 14 0 14 1073741824 1 7 65915 1 -rw-r----- sapdba dba 15728640K 1327455057 1327455015 41 /oracle/SID/sapdata10/btabd_7/btabd.data7
13:28:11.356 [22040] <2> dbc_get_string: Output = 0 0 0 0 0 0 31457289 -1 9 -r--r--r-- root root 0 1327455057 1327455856 41 /oracle/SID/sapdata10/btabd_7/btabd.data7
13:28:11.356 [22040] <4> bsa_QueryFile: INF - Object found: (1.1327455057) /oracle/SID/sapdata10/btabd_7/btabd.data7
13:28:11.356 [22040] <2> dbc_get_string: Output = 1073741824 3 0 3 1073741824 2 31457292 65916 1 -rw-r----- sapdba dba 4194304K 1327455057 1327455015 43 /oracle/SID/sapdata11/btabd_12/btabd.data12
13:28:11.356 [22040] <2> dbc_get_string: Output = 0 0 0 0 0 0 39845902 -1 9 -r--r--r-- root root 0 1327455057 1327456076 43 /oracle/SID/sapdata11/btabd_12/btabd.data12
Solution
NetBackup does not allow an arbitrary [non-root] user to restore the files belonging to another user. The restoring user must either be the same as the backup user or the files being backed up must have modes that explicitly allow access to the requesting user.
Perform the restore as an appropriate user that has permissions to access the files.
The bplist command can be run from the client host as the appropriate user to determine if the permissions are valid. Run the command as root on the master server (or as the backup user on the source client) to review the permissions that must be matched.
altclient$ id
uid=198(oradba) gid=198(sapsys)
$ bplist -C myclient -t 17 -l -R /
EXIT STATUS 227: no entity was found
vs
altclient$ id
uid=199(sapdba) gid=199(dba)
$ bplist -C myclient -t 17 -l -R /
-rw-r----- sapdba dba 15728640K Jan 24 19:30 /oracle/SID/sapdata10/btabd_7/btabd.data7
-rw-r----- sapdba dba 4194304K Jan 24 19:30 /oracle/SID/sapdata11/btabd_12/btabd.data12
-rw-r----- sapdba dba 9549800K Jan 24 19:30 /oracle/SID/sapdata5/btabd_9/btabd.data9
-rw-r----- sapdba dba 6291456K Jan 24 19:30 /oracle/SID/sapdata6/btabd_17/btabd.data17
...snip...
Applies To
NetBackup 7.1, but can affect any version