InfoScale™ 9.0 Disaster Recovery Implementation Guide - Linux

Last Published:
Product(s): InfoScale & Storage Foundation (9.0)
Platform: Linux
  1. Section I. Introducing Storage Foundation and High Availability Solutions for disaster recovery
    1. About supported disaster recovery scenarios
      1.  
        About disaster recovery scenarios
      2. About campus cluster configuration
        1.  
          VCS campus cluster requirements
        2.  
          How VCS campus clusters work
        3.  
          Typical VCS campus cluster setup
      3. About replicated data clusters
        1.  
          How VCS replicated data clusters work
      4. About global clusters
        1.  
          How VCS global clusters work
        2.  
          User privileges for cross-cluster operations
        3. VCS global clusters: The building blocks
          1.  
            Visualization of remote cluster objects
          2.  
            About global service groups
          3. About global cluster management
            1.  
              About the wide-area connector process
            2.  
              About the wide-area heartbeat agent
            3.  
              Sample configuration for the wide-area heartbeat agent
          4. About serialization - The Authority attribute
            1.  
              About the Authority and AutoStart attributes
          5.  
            About resiliency and "Right of way"
          6.  
            VCS agents to manage wide-area failover
          7.  
            About the Steward process: Split-brain in two-cluster global clusters
          8.  
            Secure communication in global clusters
      5.  
        Disaster recovery feature support for components in the Veritas InfoScale product suite
      6.  
        Virtualization support for InfoScale 9.0 products in replicated environments
    2. Planning for disaster recovery
      1. Planning for cluster configurations
        1.  
          Planning a campus cluster setup
        2.  
          Planning a replicated data cluster setup
        3.  
          Planning a global cluster setup
      2. Planning for data replication
        1.  
          Data replication options
        2.  
          Data replication considerations
  2. Section II. Implementing campus clusters
    1. Setting up campus clusters for VCS and SFHA
      1. About setting up a campus cluster configuration
        1.  
          Preparing to set up a campus cluster configuration
        2.  
          Configuring I/O fencing to prevent data corruption
        3.  
          Configuring VxVM disk groups for campus cluster configuration
        4.  
          Configuring VCS service group for campus clusters
        5.  
          Setting up campus clusters for VxVM and VCS using Veritas InfoScale Operations Manager
      2.  
        Fire drill in campus clusters
      3.  
        About the DiskGroupSnap agent
      4. About running a fire drill in a campus cluster
        1.  
          Configuring the fire drill service group
        2.  
          Running a successful fire drill in a campus cluster
    2. Setting up campus clusters for SFCFSHA, SFRAC
      1.  
        About setting up a campus cluster for disaster recovery for SFCFSHA or SF Oracle RAC
      2.  
        Preparing to set up a campus cluster in a parallel cluster database environment
      3.  
        Configuring I/O fencing to prevent data corruption
      4.  
        Configuring VxVM disk groups for a campus cluster in a parallel cluster database environment
      5.  
        Configuring VCS service groups for a campus cluster for SFCFSHA and SF Oracle RAC
      6.  
        Tuning guidelines for parallel campus clusters
      7.  
        Best practices for a parallel campus cluster
  3. Section III. Implementing replicated data clusters
    1. Configuring a replicated data cluster using VVR
      1. About setting up a replicated data cluster configuration
        1.  
          About typical replicated data cluster configuration
        2.  
          About setting up replication
        3.  
          Configuring the service groups
        4.  
          Configuring the service group dependencies
      2. About migrating a service group
        1.  
          Switching the service group
      3.  
        Fire drill in replicated data clusters
    2. Configuring a replicated data cluster using third-party replication
      1.  
        About setting up a replicated data cluster configuration using third-party replication
      2.  
        About typical replicated data cluster configuration using third-party replication
      3.  
        About setting up third-party replication
      4.  
        Configuring the service groups for third-party replication
      5.  
        Fire drill in replicated data clusters using third-party replication
  4. Section IV. Implementing global clusters
    1. Configuring global clusters for VCS and SFHA
      1.  
        Installing and Configuring Cluster Server
      2. Setting up VVR replication
        1.  
          About configuring VVR replication
        2.  
          Best practices for setting up replication
        3. Creating a Replicated Data Set
          1. Creating a Primary RVG of an RDS
            1.  
              Prerequisites for creating a Primary RVG of an RDS
            2.  
              Example - Creating a Primary RVG containing a data volume
            3.  
              Example - Creating a Primary RVG containing a volume set
          2. Adding a Secondary to an RDS
            1.  
              Best practices for adding a Secondary to an RDS
            2.  
              Prerequisites for adding a Secondary to an RDS
          3. Changing the replication settings for a Secondary
            1. Setting the mode of replication for a Secondary
              1.  
                Example - Setting the mode of replication to asynchronous for an RDS
              2.  
                Example - Setting the mode of replication to synchronous for an RDS
            2.  
              Setting the latency protection for a Secondary
            3.  
              Setting the SRL overflow protection for a Secondary
            4.  
              Setting the network transport protocol for a Secondary
            5. Setting the packet size for a Secondary
              1.  
                Example - Setting the packet size between the Primary and Secondary
            6. Setting the bandwidth limit for a Secondary
              1.  
                Example: Limiting network bandwidth between the Primary and the Secondary
              2.  
                Example: Disabling Bandwidth Throttling between the Primary and the Secondary
              3.  
                Example: Limiting network bandwidth used by VVR when using full synchronization
        4. Synchronizing the Secondary and starting replication
          1. Methods to synchronize the Secondary
            1.  
              Using the network to synchronize the Secondary
            2.  
              Using block-level tape backup to synchronize the Secondary
            3.  
              Moving disks physically to synchronize the Secondary
          2. Using the automatic synchronization feature
            1.  
              Notes on using automatic synchronization
          3.  
            Example for setting up replication using automatic synchronization
          4.  
            About SmartMove for VVR
          5.  
            About thin storage reclamation and VVR
          6.  
            Determining if a thin reclamation array needs reclamation
        5. Starting replication when the data volumes are zero initialized
          1.  
            Example: Starting replication when the data volumes are zero initialized
      3.  
        Setting up third-party replication
      4. Configuring clusters for global cluster setup
        1.  
          Configuring global cluster components at the primary site
        2.  
          Installing and configuring VCS at the secondary site
        3.  
          Securing communication between the wide-area connectors
        4.  
          Configuring remote cluster objects
        5.  
          Configuring additional heartbeat links (optional)
        6.  
          Configuring the Steward process (optional)
      5. Configuring service groups for global cluster setup
        1.  
          Configuring VCS service group for VVR-based replication
        2.  
          Configuring a service group as a global service group
      6.  
        Fire drill in global clusters
    2. Configuring a global cluster with Storage Foundation Cluster File System High Availability, Storage Foundation for Oracle RAC, or Storage Foundation for Sybase CE
      1.  
        About global clusters
      2.  
        About replication for parallel global clusters using Storage Foundation and High Availability (SFHA) Solutions
      3.  
        About setting up a global cluster environment for parallel clusters
      4.  
        Configuring the primary site
      5. Configuring the secondary site
        1.  
          Configuring the Sybase ASE CE cluster on the secondary site
      6.  
        Setting up replication between parallel global cluster sites
      7.  
        Testing a parallel global cluster configuration
    3. Configuring global clusters with VVR and Storage Foundation Cluster File System High Availability, Storage Foundation for Oracle RAC, or Storage Foundation for Sybase CE
      1.  
        About configuring a parallel global cluster using Volume Replicator (VVR) for replication
      2. Setting up replication on the primary site using VVR
        1.  
          Creating the data and SRL volumes on the primary site
        2.  
          Setting up the Replicated Volume Group on the primary site
      3. Setting up replication on the secondary site using VVR
        1.  
          Creating the data and SRL volumes on the secondary site
        2.  
          Editing the /etc/vx/vras/.rdg files
        3.  
          Setting up IP addresses for RLINKs on each cluster
        4.  
          Setting up the disk group on secondary site for replication
      4.  
        Starting replication of the primary site database volume to the secondary site using VVR
      5. Configuring Cluster Server to replicate the database volume using VVR
        1.  
          Modifying the Cluster Server (VCS) configuration on the primary site
        2.  
          Modifying the VCS configuration on the secondary site
        3.  
          Configuring the Sybase ASE CE cluster on the secondary site
      6.  
        Replication use cases for global parallel clusters
  5. Section V. Reference
    1. Appendix A. Sample configuration files
      1. Sample Storage Foundation for Oracle RAC configuration files
        1.  
          sfrac02_main.cf file
        2.  
          sfrac07_main.cf and sfrac08_main.cf files
        3.  
          sfrac09_main.cf and sfrac10_main.cf files
        4.  
          sfrac11_main.cf file
        5.  
          sfrac12_main.cf and sfrac13_main.cf files
        6.  
          Sample fire drill service group configuration
      2. About sample main.cf files for Storage Foundation (SF) for Oracle RAC
        1.  
          Sample main.cf for Oracle 10g for CVM/VVR primary site
        2.  
          Sample main.cf for Oracle 10g for CVM/VVR secondary site
      3. About sample main.cf files for Storage Foundation (SF) for Sybase ASE CE
        1.  
          Sample main.cf for a basic Sybase ASE CE cluster configuration under VCS control with shared mount point on CFS for Sybase binary installation
        2.  
          Sample main.cf for a basic Sybase ASE CE cluster configuration with local mount point on VxFS for Sybase binary installation
        3.  
          Sample main.cf for a primary CVM VVR site
        4.  
          Sample main.cf for a secondary CVM VVR site

Modifying the Cluster Server (VCS) configuration on the primary site

The following are the tasks required to modify the existing VCS configuration on the primary site:

Table: Tasks for modifying the VCS configuration on the primary site

Task

Description

Configure two service groups:

  • A log owner group including the RVGLogowner resource.

  • A Replicated Volume Group (RVG) group including the RVGShared resource replication objects.

See “To modify VCS on the primary site”.

See Modifying the VCS configuration on the secondary site.

Add the RVGSharedPri resource to the existing database service group and define this group as a global group by setting the ClusterList and ClusterFailOverPolicy attributes.

See “To modify VCS on the primary site”.

See Modifying the VCS configuration on the secondary site.

Move the CVMVolDg resource from the existing database service group to the newly created RVG group.

See “To modify VCS on the primary site”.

See Modifying the VCS configuration on the secondary site.

To view the sample main.cf files on your system:

See “To view sample configuration files for SF Oracle RAC”.

See “To view sample configuration files for SF Sybase CE”.

To view sample configuration files for SF Oracle RAC

  1. Change directories to the find the sample main.cfs:
    # cd /etc/VRTSvcs/conf/sample_rac
  2. Enter:
    # ls *sfrac*
    sfrac07_main.cf sfrac08_main.cf

    The following files include CVM/VVR configuration examples:

    • For the primary: sfrac_07_main.cf

    • For the secondary: sfrac_08_main.cf

    See Sample Storage Foundation for Oracle RAC configuration files.

To view sample configuration files for SF Sybase CE

  1. Change directories to the find the sample main.cfs:
    # cd /etc/VRTSagents/ha/conf/Sybase/
  2. Enter:
    # ls cvmvvr*

    The following sample main.cf files illustrate the VCS configuration changes after setting up an existing database for replication:

    See About sample main.cf files for Storage Foundation (SF) for Sybase ASE CE.

To modify VCS on the primary site

  1. Log into one of the nodes on the primary cluster.
  2. Use the following command to save the existing configuration to disk, and make the configuration read-only while you make changes:
      # haconf -dump -makero
  3. Use the following command to make a backup copy of the main.cf file:
    # cd /etc/VRTSvcs/conf/config
    # cp main.cf main.orig
  4. Use vi or another text editor to edit the main.cf file. Review the sample configuration file after your product installation.

    Add a failover service group using the appropriate values for your cluster and nodes. Include the following resources:

    • In a shared disk group environment, currently only the cvm master node should be assigned the logowner role.

    • IP resource

    • NIC resources

    The following are examples of RVGLogowner service group for the different platforms.

    Example for Oracle RAC:

    Please see the section "CVM master node needs to assume the logowner role for VCS managed VVR resources" in the Cluster Server Bundled Agents Reference Guide for more info.

    group rlogowner ( 
        SystemList = { sys1 = 0, sys2 = 1 } 
        AutoStartList = { sys1, sys2 } 
        ) 
    
        IP logowner_ip (
            Device = eth0
            Address = "10.10.9.101"  
            NetMask = "255.255.255.0"  
            )
        NIC nic (
            Device = eth0
            NetworkType = ether
            NetworkHosts = "10.10.8.1"  
            ) 
    RVGLogowner logowner (  
         RVG = dbdata_rvg  
         DiskGroup = dbdatadg  
         )
    requires group RVGgroup online local firm
    logowner requires logowner_ip
    logowner_ip requires nic
  5. Add the RVG service group using the appropriate values for your cluster and nodes.

    Example RVGgroup service group:

    group RVGgroup ( 
         SystemList = { sys1 = 0, sys2 = 1 } 
         Parallel = 1 
         AutoStartList = { sys1, sys2 } 
         )
    RVGShared dbdata_rvg ( 
         RVG = dbdata_rvg 
         DiskGroup = dbdatadg 
         ) 
         CVMVolDg dbdata_voldg (  
             CVMDiskGroup = dbdatadg  
             CVMActivation = sw
             CVMVolume = { dbvol, dbdata_srl }  
             )
    requires group cvm online local firm
    dbdata_rvg requires dbdata_voldg
  6. Modify the database service group using the appropriate values for your cluster and nodes:

    • Define the database service group as a global group by specifying the clusters on the primary and secondary sites as values for the ClusterList group attribute. See the bolded attribute in the example that follows.

    • Add the ClusterFailOverPolicy cluster attribute. Veritas recommends using the Manual value. See the bolded attribute in the example.

    • Add the RVGSharedPri resource to the group configuration.

    • Remove the CVMVolDg resource, if it has been configured in your previous configuration. This resource is now part of the RVG service group.

    • Specify the service group (online, local, firm) to depend on the RVG service group.

    • Remove the existing dependency of the Database service group on the CVM service group. Remove the line:

      requires group CVM online local firm
    • Remove the existing dependency between the CFSMount for the database and the CVMVoldg for the database. Remove the line:

      dbdata_mnt requires dbdata_voldg
    See configuration examples below.
  7. Save and close the main.cf file.
  8. It is advisible to modify "OnlineRetryLimit" & "OfflineWaitLimit" attribute of IP resource type to 1 on both the clusters:
    # hatype -modify IP  OnlineRetryLimit  1
    # hatype -modify IP  OfflineWaitLimit  1
  9. Use the following command to verify the syntax of the /etc/VRTSvcs/conf/config/main.cf file:
    # hacf -verify /etc/VRTSvcs/conf/config
  10. Stop and restart VCS.
    # hastop -all -force

    Wait for port h to stop on all nodes, and then restart VCS with the new configuration on all primary nodes:

    # hastart

Storage Foundation Cluster File System High Availability (SFCFS HA) example of a database service group configured for replication:

group database_grp ( 
    SystemList = { sys1 = 0, sys2 = 1 } 
    ClusterList = { clus1 = 0, clus2 = 1 } 
    Parallel = 1 
    ClusterFailOverPolicy = Manual 
    Authority = 1 
    AutoStartList = { sys1,sys2 } 
				OnlineRetryLimit = 3
				TriggerResStateChange = 1
				OnlineRetryInterval = 120 
    ) 

    CFSMount dbdata_mnt (  
        MountPoint = "/dbdata"  
        BlockDevice = "/dev/vx/dsk/dbdatadg/dbdata_vol"  
        ) 

				Process vxfend (
				PathName = "/sbin/vxfend"
				Arguments = "-m sybase -k /tmp/vcmp_socket"
				)

    RVGSharedPri dbdata_vvr_shpri (  
       RvgResourceName = dbdata_rvg  
       OnlineRetryLimit = 0  
       )

requires group RVGgroup online local firm
oradata_mnt requires dbdata_vvr_shpri

Storage Foundation (SF) for Oracle RAC example of a database service group configured for replication:

group database_grp ( 
    SystemList = { sys1 = 0, sys2 = 1 } 
    ClusterList = { clus1 = 0, clus2 = 1 } 
    Parallel = 1 
    ClusterFailOverPolicy = Manual 
    Authority = 1 
    AutoStartList = { sys1,sys2 } 
    ) 

    CFSMount oradata_mnt (  
        MountPoint = "/oradata"  
        BlockDevice = "/dev/vx/dsk/oradatadg/racdbdata_vol"  
        ) 

    RVGSharedPri ora_vvr_shpri (  
       RvgResourceName = racdata_rvg  
       OnlineRetryLimit = 0  
       )

Oracle rac_db (  
    Sid @sys1 = vrts1  
    Sid @sys2 = vrts2  
    Owner = Oracle  
    Home = "/oracle/orahome"
    StartUpOpt = SRVCTLSTART
    ShutDownOpt = SRVCTLSTOP
    )
requires group RVGgroup online local firm
oradata_mnt requires ora_vvr_shpri
rac_db requires oradata_mnt

Storage Foundation (SF) for Sybase ASE CE example of a database service group configured for replication:

group database_group ( 
    SystemList = { sys1 = 0, sys2 = 1 } 
    ClusterList = { clus1 = 0, clus2 = 1 } 
    Parallel = 1 
    ClusterFailOverPolicy = Manual 
    Authority = 1 
    AutoStartList = { sys1,sys2 }
				OnlineRetryLimit = 3
				TriggerResStateChange = 1
				OnlineRetryInterval = 120 
    ) 

    CFSMount sybdata_101_syb_vol_mnt (  
        MountPoint = "/sybdata"  
        BlockDevice = "/dev/vx/dsk/sybdata_101/syb_vol"  
        ) 

    RVGSharedPri syb_vvr_shpri (  
       RvgResourceName = sybdata_rvg  
       OnlineRetryLimit = 0  
       )

				Process vxfend (
								PathName = "/sbin/vxfend"
								Arguments = "-m sybase -k /tmp/vcmp_socket"
								)

Syabase ase (  
    Sid @sys1 = ase1  
    Sid @sys1 = ase2  
    Owner = sybase  
    Home = "/sybase" 
				Version = 15
    SA = sa
	   Quorum_dev = "/quorum/q.dat"  
    )
requires group RVGgroup online local firm
sybdata_101_syb_vol_mnt requires syb_vvr_shpri
ase requires vxfend
ase requires sybdata_101_syb_vol_mnt
ase requires quorum_101_quorumvol_mnt
quorum_101_quorumvol_mnt requires quorum_101_voldg