V-79-57344-65072 Backup or Restore job fails with error "The connection to the target system has been lost" and/or causes the Remote Agent service to terminate.

Problem

The connection to the target system has been lost and/or the Backup Exec Remote Agent for Windows Servers (RAWS) terminates during a backup or restore job.

Error Message

One of the following can be observed in the failed backup/restore job log:

When troubleshooting this error on a Windows Machine, first check the System and Application Event Viewer Log on the machine the job failed on for any entries similar to the following:

Source: Service Control Manager
Event ID: 7031
Description: The Backup Exec Remote Agent for Windows Servers service terminated unexpectedly. It has done this x time(s). The following corrective action will be taken in 0 milliseconds: No action

Note: "x" may be of value 1, 2, or 3, etc. 

Cause

The above mentioned event ID and its description indicates a process failure, outdated device driver or mismatched version of the Remote Agent between the media server and remote machine which may cause the Remote Agent service to crash during the backup or restore operation.

This issue may also be caused due to environmental problems like port conflicts, network connectivity, etc.

Solution

To troubleshoot and resolve this issue, check the following conditions and perform one (or more) of the below mentioned steps:

Solution 1:

Remote Agent version mismatch or RAWS service is stopped

1. Ensure Backup Exec media server is patched up to date with all the service packs / hotfixes.
   For more information see: How to get updates for Backup Exec - Feature Pack, Service Pack, Hotfix, and Patches (veritas.com)
    
2. Make sure the version of Remote Agent(s) on the remote machine(s) is at the exact same version as the Remote Agent on the media server.
    
3. Make sure the Backup Exec Remote Agent service on the remote machine(s) is started.

Solution 2:

Other application using/blocking one of the ports required by Backup Exec or any 3rd party/native backup application(s) installed or running

1. Make sure no other application(s) is using/blocking the following ports:
   • 10000 - Used by Backup Exec Remote Agent for Windows (process name - beremote.exe)
   • 6101 - Used by Backup Exec Agent Browser (process name - benetns.exe)
   • 3527 - Used by Backup Exec Server (process name - beserver.exe)
     
     Note 1: If Backup Exec is being used in firewall environments, see How to configure Backup Exec with Firewalls.
     Note 2: To change the default NDMP port used by Backup Exec (i.e. Port 10000) see How to change the default port used by the Backup Exec Remote Agent for Windows Servers (RAWS).
      
2. Make sure no other backup or open file technology applications, like Legato, ArcServe, NetBackup, St. Bernards Open Transaction Manager, etc are installed on the machine.
    
3. If a database application is being backed up when the Remote Agent's connection is terminated, make sure the application (i.e. Exchange, Lotus Domino, Oracle, SharePoint or SQL) is not running a native backup (Example: SQL Dump or Exchange backup using Windows Backup) or any other type of maintenance.
    
4. Also, make sure the anti-virus scanner software is not improperly scanning the file system at any time or attempting to run a scan or update at the same time the backup is running.
   See Virus scanning recommendations for Enterprise computers that are running currently support versions of Windows.

Solution 3:

The account used for Backup Exec services and the System account does not have sufficient rights within DCOM

Please perform the following steps to resolve this issue:

1. At Start | Run, type DCOMCNFG, and press <Enter>
2. Expand Component Services and then Computers
3. Right-click on My Computer, and select Properties
4. Click on the Security tab.
5. Select Launch and Activation Permissions
6. Add in the account that is specified for the Backup Exec services, along with the System account, and give them Allow Access rights.
7. Click OK, and then OK to close the application.
8. Start the Backup Exec services with the user account.

Solution 4:

Environmental issues like network connectivity, excessive drive fragmentation, low/insufficient physical memory (RAM) or Windows automatic updates

1. Confirm the Network Card Drivers are the latest version on both the media server and each remote machine being backed up. Make sure also that the drivers/firmware are up to date on all of each machine's hardware (i.e.: SCSI/Raid Controller, Host Bus Adapter, Motherboard, etc...).T
    
2. Run a disk defragment operation and the Windows Error Checking Utility (CHKDSK /F).
   Also see How to correct slow backups and agent initialization problems on fragmented Windows Server hard disk partitions.
3. Make sure the media server and each remote machine has enough physical RAM (Random Access Memory) available to accommodate all the applications running.
   • Backup is a very I/O Intensive Operation on a system, even if the machine is just a member server or workstation in a domain and has no other applications installed on it.
   • Also, installing the Advanced Open File Option (AOFO), anti-virus scanning software, transactional database applications, and/or promoting a server to a domain controller all require more memory for all the applications to function properly, in addition to the backup software (i.e.: media server or Remote Agents).
   • A restore operation is also more I/O intensive than a backup, because of the need to both read and write the data, verses just writing the data to a media during a backup.
   • Confirm also that the memory is not mis-matched (i.e.: different makes/model) and is what the server hardware manufacturer recommends/supports.
      
4. Confirm the Windows Automatic Updates are not set to automatically download or install patches at a time when a backup could possibly be running.

Note: If the Backup Exec Remote Agent Service does not terminate during a backup or restore but the "connection to the target system has been lost" error still occurs, then the issue is most likely occurring because of the network connectivity equipment or configuration (misconfigured DNS, Security Credentials, etc...) between the media server and remote machine(s).

Solution 5:

Hard disk compression, if enabled, may also result in backup/restore jobs to run slow and/or fail

Confirm disk compression is not enabled on the hard disks.
See How hard disk compression can affect backup and restore jobs on a Windows NT File System partition.

Solution 6:

If backing up to a tape hardware (not attached to fibre) with manually modified tape drive configuration settings in Backup Exec

Make sure that the Preferred Configuration is set to the default and that none of the modes are selected unless the tape hardware is attached to fibre.
For more information on this see Tape Drive Configuration Settings in Backup Exec for Windows Server (BEWS)

Solution 7:

A higher number of CPU cycles are used by the Remote Agent while in backup operation.

The priority of the Remote Agent can be changed for a particular job. Lowering the priority may be necessary when the Remote Agent fails to initialize. The Set Remote Agent priority option allows the modification of the number of CPU cycles the media server will use to maintain optimal server performance while Remote Agent Backups are running.

To change the Remote Agent Priority, see Changing the priority for a scheduled job

Solution 8:

Restore job failure: The target machine may not have enough free disk space available.

If the connection is terminated during a restore operation:

Confirm the target machine has enough free disk space available. Some restore operations require there be 1.5 to 2 times the amount of free disk space to be available than what the amount of data being restored is.
Note: More current versions of database applications will also record some type of useful diagnostic error or warning to the Application Event Viewer Log, so review it too.