V-79-57344-34028 - Optimized Duplication fails with Read Write errors

Problem

The optimized duplication fails randomly with read write error

Error Message

Error in Opt Dupe Job Log :

UMI: V-79-57344-34028 - A backup storage read/write error has occurred.

Error in the SGMON debug (with verbose device and media debug) :

BENGINE:  [12/28/15 12:35:20] [4484]     [server]             | Status for: 'JOB NAME' updated
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: STS: Error: STS [DELL:] stsp_copy_extent, return code 2060019
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: ostutil:Opdup Copy extent 0x2ad50000 bytes copied from BEOST_00000676 to BEOST_00000010 in 5.553600 seconds, rc=2060019
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - TF_OSTIncludeInTempImage(): Execute( ADAMM_MOVER_EXECUTE_CONTROL_OST_IMAGE - OST_IMAGE_COMMAND_INCLUDE_IN_TEMP_IMAGE ) failed!  mvrRet = 0x00000017
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - OST ===> Include In Temp Image returning 0xe00084ec.
BENGINE:  [12/28/15 12:35:22] [6396]     [loops]              - OptDupeSetCopyBSDProcessor::ProcessBSD: Calling TF_OSTCopyTempImage because m_isLastBSD is true.
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - OST ===> Copy Temp Image.
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: ostutil: sts_get_image_prop_byname BEOST_00000010  0 0.000000 seconds
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - OST ===> Stats for copy temp:
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - OST ===> Closing image.
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: STS: Error: STS [DELL:] stsp_write_image failed, return code 2060019
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: STS: Error: STS [DELL:] Error c000000200000020 on image 0000/BEOST_00000010 while copying
BENGINE:  [12/28/15 12:35:22] [6396]                          | ADAMM mover: DeviceIo: STS: Error: STS [DELL:] stsp_close_image failed, return code 2060019
BENGINE:  [12/28/15 12:35:22] [6396]     [tpfmt]              - Execute( ADAMM_MOVER_EXECUTE_CLOSE_OST_IMAGE ) failed!  mvrRet = 0x00000017


#define STS_ESOCKET                    2060019      "error occurred on network socket"

Error in the OST Plugin Log :

[3744:1ab0]000000000079: 2016-12-28 12:35:22 [0]: [3744] ERROR: Error c000000200000020 from o3e replicate API. Source Image 0000/BEOST_00000676 offset 131072 len 1327235072 
[3744:1ab0]000000000080: 2016-12-28 12:35:22 [0]: [3744] ERROR: Error c000000200000020 replicating image through o3e (Broken pipe) 
[3744:1ab0]000000000081: 2016-12-28 12:35:22 [0]: [3744] ERROR: stsp_copy_extent, return code 2060019 
[3744:1ab0]000000000082: 2016-12-28 12:35:22 [0]: [3744] ERROR: stsp_write_image failed, return code 2060019 
[3744:1ab0]000000000083: 2016-12-28 12:35:22 [0]: [3744] ERROR: Error c000000200000020 on image 0000/BEOST_00000010 while copying 
[3744:1ab0]000000000084: 2016-12-28 12:35:22 [0]: [3744] ERROR: stsp_close_image failed, return code 2060019 

 

Cause

Network Issues

Solution

Please contact your Network Team to resolve any network Issues. In one of the case where this was seen, the Issue was diagnosed to be with the Internal Gateway Intrusion Prevention System blocking the connection between the OST Appiances.

UMI : V-79-57344-34028