Date: 2021-06-18

OS: Windows/Linux

Errors/Problems Fixed:

CFT-3780 - O365 Audits failing    
CFT-3781 - OneDrive token isn't refreshed            
CFT-3815 - TLS Handshake fails to initiate when f5 appliance is configured between the Collector and the SP Farm.        
CFT-3816 - Provide MSU level property to disable reconfirm flag    
CFT-3817 - Provide Safeguard Settings configuration at device level    
CFT-3818 - Logging enhancement for Controlpoint exe

This hotfix provides fixes for following issues:
1) O365 auditing failed while comparing MSU name from audit file because of case sensitivity.
2) OneDrive token isn't refreshed only in case of access token expire and we are using refresh tokens. 
3) SharePoint binaries were not TLS enabled, due to that client binaries were not able to communicate with Web services hosted on TLS enabled SharePoint server. 
4) Ability to turn on or off reconfirm functionality at MSU level. Previously it was available only at node level.
5) Ability to configure safeguard settings for NetApp Cluster at device level, previously it was available only at node and global level.

 
Applies to: Data Insight Management Server, Data Insight Collector worker node and Data Insight Indexer worker node.

Install/Uninstall Instructions:

Apply this hotfix to Data Insight Management Server, Data Insight Collector worker node and Data Insight Indexer worker node.

This hotfix can be applied to Data Insight version 6.2 only.

FILE AFFECTED BY THIS PATCH:

Windows:
INSTALL_ROOT\bin\sharepoint_scannerclient.exe
INSTALL_ROOT\bin\sharepoint_auditclient.exe
INSTALL_ROOT\bin\sharepoint_utilclient.exe
INSTALL_ROOT\bin\sharepoint_users.exe
INSTALL_ROOT\bin\fpolicycmod.exe
INSTALL_ROOT\bin\controlpoint.exe
INSTALL_ROOT\pdb\sharepoint_scannerclient.pdb
INSTALL_ROOT\pdb\sharepoint_auditclient.pdb
INSTALL_ROOT\pdb\sharepoint_utilclient.pdb
INSTALL_ROOT\pdb\sharepoint_users.pdb
INSTALL_ROOT\pdb\fpolicycmod.pdb
INSTALL_ROOT\pdb\controlpoint.pdb
INSTALL_ROOT\connectors\onedrive\dcConnectorOneDrive.jar
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$1.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$2.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$3.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$4.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$5.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$6.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils$ConnectorService.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\connectors\util\ConnectorUtils.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$1.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$2.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$3.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$4.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$5.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread$1.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread$2.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread$3.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread$4.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread$5.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexerThread.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexJob.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter$IndexMode.class
INSTALL_ROOT\tomcat\webapps\ROOT\WEB-INF\classes\com\symc\matrix\commd\scheduler\IndexWriter.class

Linux:
INSTALL_ROOT/bin/controlpoint
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$1.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$2.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$3.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$4.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$5.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread$1.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread$2.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread$3.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread$4.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread$5.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexerThread.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexJob.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter$IndexMode.class
INSTALL_ROOT/tomcat/webapps/ROOT/WEB-INF/classes/com/symc/matrix/commd/scheduler/IndexWriter.class

By default, value of INSTALL_ROOT on Windows is "C:\Program Files\DataInsight".
By default, value of INSTALL_ROOT on Linux is "/opt/DataInsight".

Steps:

Log into Data Insight Server and perform the following steps:
    
    1. Unzip the patch files to a temporary folder. In this folder, locate the
    "patches" folder. This folder contains all files for help application.
    
    Example: If you unzip hotfix to C:\temp, it will create the following folder
    under C:\temp:
    C:\TEMP\
    +---README
    +---patches
        +---RHEL6 
        +---RHEL7
        +---Windows    
    
    2. For Windows:
    
        a. Start the 64-bit version of either Windows PowerShell or Windows PowerShell ISE with the "Run as administrator" option.
        The version of PowerShell must be 3.0 or later. You can determine the version that you are running by typing the following at the PowerShell prompt:
        
        $PSVersionTable.PSVersion
        
        b. Run PowerShell script DIHFInstaller.ps1. This script does the following:
    
            i. Stops all Data Insight services.
            ii. Takes backup of existing binaries.
            iii. Installs required binaries.        
            iv. Brings the Data Insight services online.
        
        c. It is strongly recommended to use PowerShell script for installation of this hotfix.
        
            i. Run PowerShell cmdlet which enables all Scripts need to be signed for execution: 
            
                Set-ExecutionPolicy AllSigned
            
            ii. Check Execution Policy is set to AllSigned using below cmdlet:
            
                Get-ExecutionPolicy
                
            iii. Run Powershell Script: 
            
                .\DIHFInstaller.ps1
        
    3. For Linux:
        
        a. Stop all Data Insight services:
    
            INSTALL_ROOT/bin/DataInsightComm stop
            INSTALL_ROOT/bin/DataInsightConfig stop
            INSTALL_ROOT/bin/DataInsightWatchdog stop
        
        b. Install required binaries using following steps:
        
        i. For RHEL6 Linux, if you are patching Data Insight node installed at "/usr/local/DataInsight":        
            
            Copy and replace all the files from "patches/RHEL6/bin/" to "/usr/local/DataInsight/bin/"
            
            Copy and replace all the files from "patches/RHEL6/tomcat/webapps/ROOT/.../matrix/commd/scheduler/" to "/usr/local/DataInsight/tomcat/webapps/ROOT/.../matrix/commd/scheduler/"
            
        ii. For RHEL7 Linux, if you are patching Data Insight node installed at "/opt/DataInsight":
            
            Copy and replace all the files from "patches/RHEL7/bin/" to "/opt/DataInsight/bin/"
            
            Copy and replace all the files from "patches/RHEL7/tomcat/webapps/ROOT/.../matrix/commd/scheduler/" to "/opt/DataInsight/tomcat/webapps/ROOT/.../matrix/commd/scheduler/"
            
        c. Start Data Insight services:
   
            INSTALL_ROOT/bin/DataInsightComm start
            INSTALL_ROOT/bin/DataInsightConfig start
            INSTALL_ROOT/bin/DataInsightWatchdog start

Additional Notes for "Provide MSU level property to disable reconfirm flag":
Command Line to Turn "Reconfirm" flag ON/OFF at MSU Level:
    1. configdb -o -T share -k <msu_id> -J "job.IndexWriterJob.writer.reconfirm" -j True
    2. configdb -o -T share -k <msu_id> -J "job.IndexWriterJob.writer.reconfirm" -j False

    
Additional Notes for "Provide Safeguard Settings configuration at device level":
1. If the device level safegaurd property is not set for any of the below mentioned properties,then DI will pick the default value. Default values are mentioned below.
2. If the safegaurd property "events.disconn.fpolicy.cmod.enabled" is set to false for device at device level then none of the safegaurd settings(device/node/global level) will work for that device, as safeguard itself is disabled.
3. If device level properties were set for device and afterwards user wants that device to use node or global safegaurd properties and not the device level, then delete "events.disconn.fpolicy.cmod.enabled" property for that device. Do not turn it off, this will just disable safeguard itself.
   Command to delete configdb entry: 
   C:\Program Files\DataInsight\bin>sqlite3.exe "C:\DataInsight\data\conf\config.db.<latest_version>"
   delete from obj_attributes where name='events.disconn.fpolicy.cmod.enabled' and obj_id=<device_id>

Safeguard Properties for NetAppCmode:
1. Number of FPolicy Cluster Mode sample values to calculate the average latency : 
    events.disconn.fpolicy.cmod.samples
2. Minimum time to wait before reconnecting to the FPolicy Cluster Mode Server (after disconnecting) (min) :
    events.disconn.fpolicy.cmod.quiet_time
3. Disconnect FPolicy Cluster Mode Server when CIFS Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.cifs_latency
4. Connect FPolicy Cluster Mode Server when CIFS Latency falls below (ms):
    events.disconn.fpolicy.cmod.watermark.low.cifs_latency
5. Disconnect FPolicy Cluster Mode Server when CIFS Write Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.cifs_write_latency
6. Connect FPolicy Cluster Mode Server when CIFS Write Latency falls below (ms) :
    events.disconn.fpolicy.cmod.watermark.low.cifs_write_latency
7. Disconnect FPolicy Cluster Mode Server when CIFS Read Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.cifs_read_latency
8. Connect FPolicy Cluster Mode Server when CIFS Read Latency falls below (ms) :
    events.disconn.fpolicy.cmod.watermark.low.cifs_read_latency
9. Disconnect FPolicy Cluster Mode Server when NFS Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.nfsv3_latency
10. Connect FPolicy Cluster Mode Server when NFS Latency falls below (ms):
    events.disconn.fpolicy.cmod.watermark.low.nfsv3_latency
11. Disconnect FPolicy Cluster Mode Server when NFS Write Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.nfsv3_write_latency
12. Connect FPolicy Cluster Mode Server when NFS Write Latency falls below (ms) :
    events.disconn.fpolicy.cmod.watermark.low.nfsv3_write_latency
13. Disconnect FPolicy Cluster Mode Server when NFS Read Latency exceeds (ms):
    events.disconn.fpolicy.cmod.watermark.high.nfsv3_read_latency
14. Connect FPolicy Cluster Mode Server when NFS Read Latency falls below (ms) :
    events.disconn.fpolicy.cmod.watermark.low.nfsv3_read_latency

Default values of safegaurd settings:
1. Number of FPolicy Cluster Mode sample values to calculate the average latency : default value 10
2. Minimum time to wait before reconnecting to the FPolicy Cluster Mode Server (after disconnecting) (min) : default value 5
3. Disconnect FPolicy Cluster Mode Server when CIFS Latency exceeds (ms): default value 10
4. Connect FPolicy Cluster Mode Server when CIFS Latency falls below (ms): default value 2
5. Disconnect FPolicy Cluster Mode Server when CIFS Write Latency exceeds (ms): default value 10
6. Connect FPolicy Cluster Mode Server when CIFS Write Latency falls below (ms) : default value 2
7. Disconnect FPolicy Cluster Mode Server when CIFS Read Latency exceeds (ms): default value 10 
8. Connect FPolicy Cluster Mode Server when CIFS Read Latency falls below (ms) : default value 2
9. Disconnect FPolicy Cluster Mode Server when NFS Latency exceeds (ms): default value 10
10. Connect FPolicy Cluster Mode Server when NFS Latency falls below (ms): default value 2
11. Disconnect FPolicy Cluster Mode Server when NFS Write Latency exceeds (ms): default value 10
12. Connect FPolicy Cluster Mode Server when NFS Write Latency falls below (ms) : default value 2
13. Disconnect FPolicy Cluster Mode Server when NFS Read Latency exceeds (ms): default value 10
14. Connect FPolicy Cluster Mode Server when NFS Read Latency falls below (ms) : default value 2

Command line to set safeguard properties for device id 1:
#Property to enable/disable safeguard feature for device 1 at device level
1. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.enabled" -j true
2. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.enabled" -j false 
#Safeguard properties
1. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.samples" -j 2
2. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.quiet_time" -j 2
3. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.cifs_latency" -j 1
4. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.cifs_latency" -j 1
5. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.cifs_write_latency" -j 1
6. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.cifs_write_latency" -j 1
7. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.cifs_read_latency" -j 1
8. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.cifs_read_latency" -j 1
9. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.nfsv3_latency" -j 1
10. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.nfsv3_latency" -j 1
11. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.nfsv3_write_latency" -j 1
12. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.nfsv3_write_latency" -j 1
13. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.high.nfsv3_read_latency" -j 1
14. configdb -o -T filer -k 1 -J "events.disconn.fpolicy.cmod.watermark.low.nfsv3_read_latency" -j 1

Additional Notes for Hotfix:
1. Apply this hotfix to Data Insight 6.2 only.
2. If a new Data Insight node is added later, this hotfix needs to be applied.