Backup - Exchange - 0xe000fea9 - The Backup Exec data store encountered a problem during the operation

Exchange GRT backup fails with error: 0xe000fea9 - The Backup Exec data store encountered a problem during the operation.

Issue:

Exchange Granular Restore Technology (GRT) enabled backups are failing when running to Backup to Disk (B2D)/Deduplication folder/Tape

Error Message:

Final error: 0xe000fea9 - The Backup Exec data store encountered a problem during the operation. See the job log for details.

V-79-57344-65193 - The backup selection '\\Exchangeservername\Microsoft Information Store\DATABASE' was not successfully processed for Granular Recovery Technology (GRT). The Database Recovery failed with Error '*Null*'. You will not be able to restore individual items for this backup selection.

Possible Causes:

1. Exchange and server Operating system may not compatible

2. The GRT thread has detached from the disk prior to completing the GRT catalog operation.

3. ESE.DLL dependencies fail to load due to pre-requisites missing from the server, ESE.DLL has dependencies on VC and C++ redistributable packages.

4. Anti-Malware /Anti-virus software to monitor the Recovery path, Dedupe location and BE Process locations.

5. Recovery location doesn't have enough space to process the GRT.

6. Database corruption/Corrupted transaction logs not truncated.

7. Issue with the target storage device.

Solution:

· Check the Backup Exec Software compatible list.

· Backup Exec and remote agent is must be updated with latest patches.

· Install the Microsoft Visual C++ package on Backup Exec Server. Extract the VCredist folder from the Backup Exec installation package and install the updated or missing C++ redistributable package.

· If the Backup Job utilizing GRT continues to fail, Dependency Walker must be run on ESE.DLL. ESE.DLL can be found in the GRT temporary staging location, typically X:\TEMP\Backup Exec. This will reveal which Dependencies fail to load.

· The backup job chooses the volume which has greatest amount of free space. This issue will come up if the temp location selected by the backup job resides on a mount point

HKEY_LOCAL_MACHINE\SOFTWARE\veritas\Backup Exec For Windows\Backup Exec\Engine\Exchange, Create new String Value, "OnHostTemp" then set the value X:\Temp

· The GRT thread should not get detached from the disk prior to completing the GRT catalog operation.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VirtFile , Create new DWORD Value called Detach Timeout ,then set the value as 10000 (decimal) .

Above two registry values based on the storage.

IF the backup to the disk (B2D) - Registry value should add in BE server

IF the backup to the Tape/OST/cloud - Registry value should add in Exchange server

Restart the Backup Exec / Backup Exec Remote Agent for Windows service.

· GRT process may affected because the consistency check being performed as part of the Exchange Mailbox Database backup can be affected by an inconsistency in the database page size. So "OutProc Consistency Check" registry key can be set to alter the way Backup Exec calls the consistency check to be performed.

In Exchange server, HKEY_LOCAL_MACHINE\SOFTWARE\Symantec\Backup Exec For Windows\Backup Exec\Engine\Exchange Create a Dword "OutProc Consistency Check", then set the value as 1.

Restart the Backup Exec Remote Agent for Windows service.

· Create the Exchange GRT backup as per Backup Exec recommendation as mentioned in https://www.veritas.com/support/en_US/doc/59226269-99535599-0/v53932922-99535599

· Check the Exchange permissions and BESA account should have the full control permissions to the Recovery location.

· Run Windows Update and ensure that the media server has all the latest hotfixes from Microsoft.

· Target the backup job to different disk storage if suspect there is an issue with storage.

· Check the event viewer for the database related issues.

· Check the health of the databases (repair as needed with the assistance of Microsoft).

· Clear the snapshots and reclaim the free space.

· Open a command prompt as administrator and run the command.

"vssadmin delete shadows /for=c: /all" Please refer https://www.veritas.com/support/en_US/article.100022885

· Run Non-GRT backup for the affected databases.

· Install the same version of Exchange management tools in the Backup Exec server for passing required pre-requisites when the backup job to the disk (B2D).

Note: Recommended a complete backup of the registry prior to making any registry changes, and patch updates of the server required reboot so plan down time accordingly.