InfoScale™ 9.0 Cluster Server Administrator's Guide - Windows

Last Published:
Product(s): InfoScale & Storage Foundation (9.0)
Platform: Windows
  1. Section I. Clustering concepts and terminology
    1. Introducing Cluster Server
      1. About Cluster Server
        1.  
          How VCS detects failure
        2. How VCS ensures application availability
          1.  
            About switchover and failover
      2. About cluster control guidelines
        1.  
          Defined start, stop, and monitor procedures
        2.  
          Ability to restart the application in a known state
        3.  
          External data storage
        4.  
          Licensing and host name issues
      3. About the physical components of VCS
        1.  
          About VCS nodes
        2.  
          About shared storage
        3.  
          About networking
      4. Logical components of VCS
        1.  
          About resources and resource dependencies
        2.  
          Categories of resources
        3.  
          About resource types
        4.  
          About service groups
        5. Types of service groups
          1.  
            About failover service groups
          2.  
            About parallel service groups
          3.  
            About hybrid service groups
        6.  
          About the ClusterService group
        7.  
          About agents in VCS
        8.  
          About agent functions
        9. Agent classifications
          1.  
            About bundled agents
          2.  
            About enterprise agents
          3.  
            About custom agents
        10.  
          VCS agent framework
        11. About cluster control, communications, and membership
          1.  
            About the high availability daemon (HAD)
          2.  
            About Group Membership Services and Atomic Broadcast (GAB)
          3.  
            About Low Latency Transport (LLT)
        12. About security services
          1.  
            Digital certification structure
          2.  
            Components for secure communication
          3.  
            Restriction after failed user login attempts
        13.  
          Components for administering VCS
      5.  
        Putting the pieces together
    2. About cluster topologies
      1. Basic failover configurations
        1.  
          Asymmetric or active / passive configuration
        2.  
          Symmetric or active / active configuration
        3.  
          About N-to-1 configuration
      2. About advanced failover configurations
        1.  
          About the N + 1 configuration
        2.  
          About the N-to-N configuration
      3. Cluster topologies and storage configurations
        1.  
          About basic shared storage cluster
        2.  
          About campus, or metropolitan, shared storage cluster
        3.  
          About shared nothing clusters
        4.  
          About replicated data clusters
        5.  
          About global clusters
    3. VCS configuration concepts
      1.  
        About configuring VCS
      2.  
        VCS configuration language
      3. About the main.cf file
        1.  
          About the SystemList attribute
        2.  
          Initial configuration
        3.  
          Including multiple .cf files in main.cf
      4.  
        About the types.cf file
      5. About VCS attributes
        1.  
          About attribute data types
        2.  
          About attribute dimensions
        3.  
          About attributes and cluster objects
        4.  
          Attribute scope across systems: global and local attributes
        5.  
          About attribute life: temporary attributes
        6.  
          Size limitations for VCS objects
      6.  
        VCS keywords and reserved words
      7.  
        VCS environment variables
  2. Section II. Administration - Putting VCS to work
    1. About the VCS user privilege model
      1. About VCS user privileges and roles
        1.  
          VCS privilege levels
        2.  
          User roles in VCS
        3.  
          Hierarchy in VCS roles
        4.  
          User privileges for CLI commands
        5.  
          User privileges for cross-cluster operations
        6.  
          User privileges for clusters that run in secure mode
      2.  
        How administrators assign roles to users
      3.  
        User privileges for OS user groups for clusters running in secure mode
      4.  
        VCS privileges for users with multiple roles
      5. Restricted user privileges
        1.  
          Restricting user privileges
        2.  
          Limitations on restricting user privileges
    2. Getting started with VCS
      1. Configuring the cluster using the Cluster Configuration Wizard
        1.  
          Configuring notification
        2.  
          Configuring Wide-Area Connector process for global clusters
      2. About configuring a cluster from the command line
        1. About preparing for a silent configuration
          1.  
            About configuring a non-secure cluster
          2.  
            About configuring a secure cluster
          3.  
            About deleting a non-secure cluster
          4.  
            About deleting a secure cluster
          5.  
            About element attributes values
          6.  
            About sample XML configuration
        2.  
          Running the silent configuration utility
    3. Administering the cluster from the command line
      1. About administering VCS from the command line
        1.  
          Symbols used in the VCS command syntax
        2.  
          How VCS identifies the local system
        3.  
          About specifying values preceded by a dash (-)
        4.  
          About the -modify option
        5.  
          Encrypting VCS passwords
        6. Encrypting agent passwords
          1.  
            Generating a security key
          2.  
            Encrypting the agent password
          3.  
            Changing the security key
      2.  
        Starting VCS
      3. Stopping the VCS engine and related processes
        1.  
          About stopping VCS without the -force option
        2.  
          About stopping VCS with options other than the -force option
        3.  
          About controlling the hastop behavior by using the EngineShutdown attribute
        4.  
          Additional considerations for stopping VCS
      4. About managing VCS configuration files
        1.  
          About the hacf utility
        2.  
          About multiple versions of .cf files
        3.  
          Verifying a configuration
        4.  
          Scheduling automatic backups for VCS configuration files
        5.  
          Saving a configuration
        6.  
          Setting the configuration to read or write
        7.  
          Displaying configuration files in the correct format
      5. About managing VCS users from the command line
        1.  
          Adding a user
        2.  
          Assigning and removing user privileges
        3.  
          Modifying a user
        4.  
          Deleting a user
        5.  
          Displaying a user
      6. About querying VCS
        1.  
          Querying service groups
        2.  
          Querying resources
        3.  
          Querying resource types
        4.  
          Querying agents
        5.  
          Querying systems
        6.  
          Querying clusters
        7.  
          Querying status
        8.  
          Querying log data files (LDFs)
        9.  
          Using conditional statements to query VCS objects
      7. About administering service groups
        1.  
          Adding and deleting service groups
        2. Modifying service group attributes
          1.  
            Modifying the SystemList attribute
        3.  
          Bringing service groups online
        4.  
          Taking service groups offline
        5.  
          Switching service groups
        6.  
          Freezing and unfreezing service groups
        7.  
          Enabling and disabling priority based failover for a service group
        8.  
          Enabling and disabling service groups
        9.  
          Clearing faulted resources in a service group
        10.  
          Linking and unlinking service groups
      8.  
        Administering agents
      9. About administering resources
        1.  
          About adding resources
        2.  
          Adding resources
        3.  
          Deleting resources
        4.  
          Adding, deleting, and modifying resource attributes
        5.  
          Defining attributes as local
        6.  
          Linking and unlinking resources
        7.  
          Bringing resources online
        8.  
          Taking resources offline
        9.  
          Probing a resource
        10.  
          Clearing a resource
      10. About administering resource types
        1.  
          Adding, deleting, and modifying resource types
        2.  
          Overriding resource type static attributes
      11.  
        Administering systems
      12. About administering clusters
        1.  
          Retrieving version information
      13.  
        Using the -wait option in scripts that use VCS commands
      14.  
        About administering simulated clusters from the command line
    4. Configuring resources and applications in VCS
      1. About configuring resources and applications
        1.  
          Considerations for Windows Server systems
      2. About Virtual Business Services
        1.  
          Features of Virtual Business Services
        2.  
          Sample virtual business service configuration
      3. About Intelligent Resource Monitoring (IMF)
        1.  
          VCS changes to support IMF
        2.  
          VCS agents that support IMF
        3.  
          How IMF works
        4.  
          How to enable IMF
        5.  
          How to disable IMF
        6. Recommended settings
          1.  
            Modify the MountV resource attributes
          2.  
            Modify the attribute values for SQL Analysis Service and SQL Server Agent resources
          3.  
            Modify the NumThreads attribute for MountV and VMDg
      4. About fast failover
        1.  
          VCS changes for fast failover
        2.  
          Enabling fast failover for disk groups
      5. How VCS monitors storage components
        1.  
          Shared storage - if you use NetApp filers
        2.  
          Shared storage - if you use SFW to manage cluster dynamic disk groups
        3.  
          Shared storage - if you use Windows LDM to manage shared disks
        4.  
          Non-shared storage - if you use SFW to manage dynamic disk groups
        5.  
          Non-shared storage - if you use Windows LDM to manage local disks
        6.  
          Non-shared storage - if you use VMware storage
      6. About storage configuration
        1. About managing storage using Windows Logical Disk Manager
          1.  
            Reserving disks (if you use Windows LDM)
          2.  
            Creating volumes (if you use Windows LDM)
          3.  
            Mounting volumes (if you use Windows LDM)
          4.  
            Unassigning a drive letter
          5.  
            Releasing disks (if you use Windows LDM)
          6.  
            Configuration tasks
        2. About managing storage in a Network Appliance storage environment
          1.  
            Configuring Microsoft iSCSI Initiator
          2.  
            Connecting virtual disks to the cluster node
          3.  
            Disconnecting virtual disks from the cluster nodes
        3. About managing shared storage using Storage Foundation for Windows
          1.  
            Using SFW with VCS
          2.  
            Before you configure shared storage using SFW
          3.  
            Configuring shared storage
        4.  
          Managing storage
      7. About configuring network resources
        1. About configuring IP addresses on the systems
          1.  
            Before you configure IP addresses on the systems
          2.  
            Disabling DHCP
          3.  
            Configuring IP addresses on the systems
        2. About configuring virtual computer names
          1.  
            Before you configure virtual computer names
          2.  
            Configuring virtual computer names
      8. About configuring file shares
        1.  
          Before you configure a file share service group
        2.  
          Configuring file shares using the wizard
        3.  
          Modifying a file share service group using the wizard
        4.  
          Deleting a file share service group using the wizard
        5.  
          Creating non-scoped file shares configured with VCS
        6.  
          Making non-scoped file shares accessible while using virtual server name or IP address if NetBIOS and WINS are disabled
      9. About configuring IIS sites
        1.  
          Before you configure an IIS service group
        2.  
          Fixing the IPv6 address configuration for FTP sites
        3.  
          Installing IIS on Windows Server Core
        4.  
          Configuring an IIS service group using the wizard
        5.  
          Modifying an IIS service group using the wizard
        6.  
          Deleting an IIS service group using the wizard
      10. About configuring services
        1.  
          About configuring a service using the GenericService agent
        2. Before you configure a service using the GenericService agent
          1.  
            Changing a service startup type
          2.  
            Configuring a service to run in a user context
        3.  
          Configuring a service using the GenericService agent
        4.  
          About configuring a service using the ServiceMonitor agent
        5.  
          Before you configure a service using the ServiceMonitor agent
        6.  
          Configuring a service using the ServiceMonitor agent
      11. About configuring processes
        1.  
          Before you configure processes
        2.  
          Configuring processes using the Process agent
      12. About configuring Microsoft Message Queuing (MSMQ)
        1.  
          Before you configure the MSMQ service group
        2.  
          Configuring the MSMQ resource using the command-line utility
        3.  
          Configuring the MSMQ service group using the wizard
        4.  
          Modifying an MSMQ service group using the wizard
        5.  
          Configuring MSMQ agent to check port bindings more than once
        6.  
          Binding an MSMQ instance to the correct IP address
        7.  
          Checking whether MSMQ is listening for messages
      13. About configuring the infrastructure and support agents
        1.  
          About configuring notification
        2. Configuring registry replication
          1.  
            About registry hive abbreviations
          2.  
            About excluding keys
          3.  
            About ignoring subkeys
          4.  
            About additional considerations for using IgnoreSubKeys
        3.  
          Configuring a proxy resource
        4.  
          Configuring a phantom resource
        5.  
          Configuring file resources
        6.  
          Configuring a RemoteGroup resource
      14. About configuring applications using the Application Configuration Wizard
        1.  
          Before you configure service groups using the Application Configuration wizard
        2. Adding resources to a service group
          1.  
            Configuring a GenericService resource
          2.  
            Configuring processes
          3.  
            Configuring a ServiceMonitor resource
          4.  
            Configuring VCS components
        3.  
          Configuring service groups using the Application Configuration Wizard
        4.  
          Modifying an application service group
        5.  
          Deleting resources from a service group
        6.  
          Deleting an application service group
      15. About application monitoring on single-node clusters
        1.  
          Configuring application monitoring on a single-node cluster
        2.  
          Verifying whether application monitoring is enabled on the single-node cluster
        3.  
          Unconfiguring application monitoring on a single-node cluster
        4.  
          About reviewing and troubleshooting the configuration and monitoring activities
      16. Configuring the service group in a non-shared storage environment
        1.  
          Setting the timeout duration for which the VMNSDg agent waits for all the disks to arrive before importing the disk group
      17. About the VCS Application Manager utility
        1.  
          Managing applications in virtual server context
      18. About testing resource failover using virtual fire drills
        1.  
          About virtual fire drills
        2.  
          About infrastructure checks and fixes for supported agents
        3.  
          About running a virtual fire drill
    5. Modifying the cluster configuration
      1.  
        About modifying the cluster configuration
      2.  
        Adding nodes to a cluster
      3.  
        Removing nodes from a cluster
      4.  
        Reconfiguring a cluster
      5.  
        Configuring single sign-on for the cluster manually
      6. Configuring the ClusterService group
        1.  
          Configuring notification
        2.  
          Configuring the wide-area connector process for global clusters
      7.  
        Deleting a cluster configuration
  3. Section III. Administration - Beyond the basics
    1. Controlling VCS behavior
      1. VCS behavior on resource faults
        1.  
          Critical and non-critical resources
        2. VCS behavior diagrams
          1.  
            Example scenario 1: Resource with critical parent faults
          2.  
            Example scenario 2: Resource with non-critical parent faults
          3.  
            Example scenario 3: Resource with critical parent fails to come online
      2. About controlling VCS behavior at the service group level
        1.  
          About the AutoRestart attribute
        2.  
          About controlling failover on service group or system faults
        3.  
          About defining failover policies
        4.  
          About system zones
        5.  
          Load-based autostart
        6.  
          About freezing service groups
        7.  
          About controlling Clean behavior on resource faults
        8.  
          Clearing resources in the ADMIN_WAIT state
        9.  
          About controlling fault propagation
        10. Customized behavior diagrams
          1.  
            Example scenario: Resource with a critical parent and ManageFaults=NONE
          2.  
            Example scenario: Resource with a critical parent and FaultPropagation=0
        11. VCS behavior for resources that support the intentional offline functionality
          1.  
            About the IntentionalOffline attribute
          2.  
            About the ExternalStateChange attribute
      3. About controlling VCS behavior at the resource level
        1. Resource type attributes that control resource behavior
          1.  
            About the RestartLimit attribute
          2.  
            About the OnlineRetryLimit attribute
          3.  
            About the ConfInterval attribute
          4.  
            About the ToleranceLimit attribute
          5.  
            About the FaultOnMonitorTimeouts attribute
        2. How VCS handles resource faults
          1.  
            VCS behavior when an online resource faults
          2.  
            VCS behavior when a resource fails to come online
        3.  
          VCS behavior after a resource is declared faulted
        4. About disabling resources
          1.  
            When to disable a resource
          2.  
            Limitations of disabling resources
          3.  
            Additional considerations for disabling resources
          4.  
            How disabled resources affect group states
      4.  
        Changing agent file paths and binaries
      5. Service group workload management
        1.  
          About enabling service group workload management
        2. System capacity and service group load
          1.  
            Static load versus dynamic load
          2.  
            About overload warning
        3.  
          System limits and service group prerequisites
        4.  
          About capacity and limits
      6. Sample configurations depicting workload management
        1.  
          System and Service group definitions
        2. Sample configuration: Basic four-node cluster
          1.  
            About AutoStart operation
          2.  
            About the failure scenario
          3.  
            About the cascading failure scenario
        3. Sample configuration: Complex four-node cluster
          1.  
            About the AutoStart operation
          2.  
            About the normal operation
          3.  
            About the failure scenario
          4.  
            About the cascading failure scenario
        4. Sample configuration: Server consolidation
          1.  
            About the AutoStart operation
          2.  
            About the normal operation
          3.  
            About the failure scenario
          4.  
            About the cascading failure scenario
    2. The role of service group dependencies
      1. About service group dependencies
        1. About dependency links
          1.  
            Dependency categories
          2.  
            Dependency location
          3.  
            Dependency rigidity
        2.  
          About dependency limitations
      2. Service group dependency configurations
        1. About failover parent / failover child
          1.  
            About failover parent / parallel child
          2.  
            About parallel parent / failover child
          3.  
            About parallel parent / parallel child
      3.  
        Frequently asked questions about group dependencies
      4.  
        About linking service groups
      5. VCS behavior with service group dependencies
        1.  
          Online operations in group dependencies
        2.  
          Offline operations in group dependencies
        3.  
          Switch operations in group dependencies
    3. VCS event notification
      1. About VCS event notification
        1.  
          Event messages and severity levels
        2.  
          About persistent and replicated message queue
        3.  
          How HAD deletes messages
      2. Components of VCS event notification
        1. About the notifier process
          1.  
            Example of notifier command
        2. About the hanotify utility
          1.  
            Example of hanotify command
      3. About VCS events and traps
        1.  
          Events and traps for clusters
        2.  
          Events and traps for agents
        3.  
          Events and traps for resources
        4.  
          Events and traps for systems
        5.  
          Events and traps for service groups
        6.  
          SNMP-specific files
        7. Trap variables in VCS MIB
          1.  
            About severityId
          2.  
            EntityType and entitySubType
          3.  
            About entityState
      4. About monitoring aggregate events
        1.  
          How to detect service group failover
        2.  
          How to detect service group switch
      5.  
        About configuring notification
    4. VCS event triggers
      1.  
        About VCS event triggers
      2.  
        Using event triggers
      3. List of event triggers
        1.  
          About the dumptunables trigger
        2.  
          About the injeopardy event trigger
        3.  
          About the loadwarning event trigger
        4.  
          About the nofailover event trigger
        5.  
          About the postoffline event trigger
        6.  
          About the postonline event trigger
        7.  
          About the preonline event trigger
        8.  
          About the resadminwait event trigger
        9.  
          About the resfault event trigger
        10.  
          About the resnotoff event trigger
        11.  
          About the resrestart event trigger
        12.  
          About the resstatechange event trigger
        13.  
          About the sysoffline event trigger
        14.  
          About the unable_to_restart_agent event trigger
        15.  
          About the unable_to_restart_had event trigger
        16.  
          About the violation event trigger
  4. Section IV. Cluster configurations for disaster recovery
    1. Connecting clusters–Creating global clusters
      1.  
        How VCS global clusters work
      2. VCS global clusters: The building blocks
        1.  
          Visualization of remote cluster objects
        2.  
          About global service groups
        3. About global cluster management
          1.  
            About the wide-area connector process
          2.  
            About the wide-area heartbeat agent
        4. About serialization - The Authority attribute
          1.  
            About the Authority and AutoStart attributes
        5.  
          About resiliency and "Right of way"
        6.  
          VCS agents to manage wide-area failover
        7.  
          About the Steward process: Split-brain in two-cluster global clusters
        8.  
          Secure communication in global clusters
      3. Prerequisites for global clusters
        1.  
          Prerequisites for cluster setup
        2.  
          Prerequisites for application setup
        3.  
          Prerequisites for wide-area heartbeats
        4.  
          Prerequisites for ClusterService group
        5.  
          Prerequisites for replication setup
      4. Setting up a global cluster
        1.  
          Preparing the application for the global environment
        2.  
          Configuring the ClusterService group
        3. Configuring replication resources in VCS
          1.  
            About the prerequisites for configuring replication resources in VCS
        4.  
          Linking the application and replication service groups
        5.  
          Configuring the second cluster
        6.  
          Linking clusters
        7.  
          Configuring the Steward process (optional)
        8.  
          Stopping the Steward process
        9.  
          Configuring the global service group
      5. About IPv6 support with global clusters
        1.  
          Prerequisites for configuring a global cluster to support IPv6
        2.  
          Migrating an InfoScale Availability cluster from IPv4 to IPv6 when Virtual IP (ClusterAddress) is configured
        3.  
          Migrating an InfoScale Availability cluster to IPv6 in a GCO deployment
      6. About cluster faults
        1.  
          About the type of failure
        2.  
          Switching the service group back to the primary
      7. About setting up a disaster recovery fire drill
        1. About creating and configuring the fire drill service group manually
          1.  
            Creating the fire drill service group
          2.  
            Linking the fire drill and replication service groups
          3.  
            Adding resources to the fire drill service group
          4.  
            Configuring the fire drill service group
          5.  
            Enabling the FireDrill attribute
      8.  
        Multi-tiered application support using the RemoteGroup agent in a global environment
      9. Test scenario for a multi-tiered environment
        1.  
          About the main.cf file for cluster 1
        2.  
          About the main.cf file for cluster 2
        3.  
          About the main.cf file for cluster 3
        4.  
          About the main.cf file for cluster 4
    2. Administering global clusters from Cluster Manager (Java console)
      1.  
        About global clusters
      2.  
        Adding a remote cluster
      3.  
        Deleting a remote cluster
      4. Administering global service groups
        1.  
          Converting local and global groups
        2.  
          Bringing a service group online in a remote cluster
        3.  
          Taking a service group offline in a remote cluster
        4.  
          Switching a service group to a remote cluster
      5. Administering global heartbeats
        1.  
          Adding a global heartbeat
        2.  
          Modifying a global heartbeat
        3.  
          Deleting a global heartbeat
    3. Administering global clusters from the command line
      1.  
        About administering global clusters from the command line
      2. About global querying in a global cluster setup
        1.  
          Querying global cluster service groups
        2.  
          Querying resources across clusters
        3.  
          Querying systems
        4.  
          Querying clusters
        5.  
          Querying status
        6.  
          Querying heartbeats
      3.  
        Administering global service groups in a global cluster setup
      4.  
        Administering resources in a global cluster setup
      5. Administering clusters in global cluster setup
        1.  
          Managing cluster alerts in a global cluster setup
        2.  
          Changing the cluster name in a global cluster setup
      6.  
        Administering heartbeats in a global cluster setup
    4. Setting up replicated data clusters
      1.  
        About replicated data clusters
      2.  
        How VCS replicated data clusters work
      3.  
        About setting up a replicated data cluster configuration
  5. Section V. Troubleshooting and performance
    1. VCS performance considerations
      1. How cluster components affect performance
        1.  
          How kernel components (GAB and LLT) affect performance
        2.  
          How the VCS engine (HAD) affects performance
        3. How agents affect performance
          1.  
            Monitoring resource type and agent configuration
        4.  
          How the VCS graphical user interfaces affect performance
        5.  
          If the network adapters cannot ping each other, the cluster nodes may not get GAB membership
      2. How cluster operations affect performance
        1.  
          VCS performance consideration when booting a cluster system
        2.  
          VCS performance consideration when a resource comes online
        3.  
          VCS performance consideration when a resource goes offline
        4.  
          VCS performance consideration when a service group comes online
        5.  
          VCS performance consideration when a service group goes offline
        6.  
          VCS performance consideration when a resource fails
        7.  
          VCS performance consideration when a system fails
        8.  
          VCS performance consideration when a network link fails
        9. VCS performance consideration when a system panics
          1.  
            About GAB client process failure
          2.  
            About GAB client registration monitoring
          3.  
            About network failure and GAB IOFENCE message
          4.  
            About quick reopen
        10.  
          VCS performance consideration when a service group switches over
        11.  
          VCS performance consideration when a service group fails over
      3.  
        Monitoring CPU usage
      4. VCS agent statistics
        1.  
          Tracking monitor cycle times
        2.  
          VCS attributes enabling agent statistics
      5.  
        About VCS performance with non-HA products
      6.  
        About VCS performance with SFW
    2. Troubleshooting and recovery for VCS
      1. VCS message logging
        1.  
          VCW logs
        2.  
          VCWsilent logs
        3.  
          Solutions wizard logs
        4.  
          Message catalogs
      2. Handling network failure
        1.  
          Disabling failover
        2. Example of how VCS handles network failure
          1.  
            Jeopardy scenario: link failure
          2.  
            Jeopardy scenario: link and node failure
          3.  
            Jeopardy scenario: failure of all links
        3.  
          Network partitioning
        4.  
          When VCS shuts down a system
        5.  
          Pre-existing network partitions
        6.  
          Seeding of VCS clusters
        7.  
          Reconnecting the private network
      3. Troubleshooting VCS startup
        1. Low Latency Transport (LLT)
          1.  
            Common LLT directives
        2.  
          Group Membership Atomic Broadcast (GAB)
        3. Verifying LLT, GAB, and cluster operation
          1.  
            Verifying LLT
          2.  
            Setting the checksum option
          3.  
            Verifying GAB
          4.  
            Verifying HAD
          5.  
            Verifying the cluster
        4. VCS startup errors
          1.  
            Seeding the cluster
        5.  
          Cluster ID is not unique over a network
      4.  
        Troubleshooting secure clusters
      5. Troubleshooting service groups
        1.  
          ClusterService group configuration
      6.  
        Troubleshooting resources
      7.  
        Troubleshooting notification
      8. Troubleshooting and recovery for global clusters
        1.  
          Disaster declaration
        2.  
          Lost heartbeats and the inquiry mechanism
        3. VCS alerts
          1.  
            Types of alerts
          2.  
            Managing alerts
          3.  
            Actions associated with alerts
          4.  
            Negating events
      9.  
        Troubleshooting the steward process
      10. VCS utilities
        1. The getcomms utility
          1.  
            getcomms options
          2.  
            Log location
        2. The hagetcf utility
          1.  
            Log location
          2.  
            Options for the hagetcf utility
        3.  
          The NICTest utility
        4.  
          The VCSRegUtil utility
        5. The havol utility
          1.  
            Using the -getdrive option
          2.  
            Using the -scsitest option
          3.  
            Retrieving the disk number
          4.  
            The -scsitest command options
        6.  
          The vmgetdrive utility
        7. Configuring the VCS Helper service manually
          1.  
            Command syntax
          2.  
            Command options
  6. Section VI. Appendixes
    1. Appendix A. VCS user privileges—administration matrices
      1.  
        About administration matrices
      2. Administration matrices
        1.  
          Agent Operations (haagent)
        2.  
          Attribute Operations (haattr)
        3.  
          Cluster Operations (haclus, haconf)
        4.  
          Service group operations (hagrp)
        5.  
          Heartbeat operations (hahb)
        6.  
          Log operations (halog)
        7.  
          Resource operations (hares)
        8.  
          System operations (hasys)
        9.  
          Resource type operations (hatype)
        10.  
          User operations (hauser)
    2. Appendix B. Cluster and system states
      1. Remote cluster states
        1.  
          Examples of cluster state transitions
      2. System states
        1.  
          Examples of system state transitions
    3. Appendix C. VCS attributes
      1.  
        About attributes and their definitions
      2.  
        Resource attributes
      3.  
        Resource type attributes
      4.  
        Service group attributes
      5.  
        System attributes
      6.  
        Cluster attributes
      7.  
        Heartbeat attributes (for global clusters)
      8.  
        Remote cluster attributes
    4. Appendix D. Configuring LLT over UDP
      1.  
        About configuring LLT over UDP
      2.  
        When to use LLT over UDP
      3. LLT over UDP configuration
        1.  
          The link command in the llttab file
        2.  
          The set-addr command in the llttab file
        3.  
          Selecting UDP ports
      4.  
        Sample configuration: Direct-attached links
      5.  
        Sample configuration: Links crossing IP routers
      6. Issues and limitations
        1.  
          VCW does not support configuring broadcasting for UDP
        2.  
          If the network adapters are unable to ping each other, the cluster nodes may not get GAB membership
    5. Appendix E. Handling concurrency violation in any-to-any configurations
      1.  
        About handling concurrency violation
      2.  
        Concurrency violation scenario
      3.  
        About the vcsgensvc.vbs script
      4. Sample configuration to handle concurrency violation
        1.  
          Notes for using scripts with the Process agent
    6. Appendix F. Accessibility and VCS
      1.  
        About accessibility in VCS
      2. Navigation and keyboard shortcuts
        1.  
          Navigation in the Java Console
        2.  
          Navigation in the Web console
      3.  
        Support for accessibility settings
      4.  
        Support for assistive technologies
    7. Appendix G. Executive Order logging
      1.  
        Executive Order logging
      2.  
        Log formatting
      3.  
        Enabling cluster server logs
      4.  
        Log forwarding

Cluster attributes

Table: Cluster attributes lists the cluster attributes.

Table: Cluster attributes

Cluster Attributes

Definition

AdministratorGroups

(user-defined)

List of operating system user account groups that have administrative privileges on the cluster. This attribute applies to clusters running in secure mode.

  • Type and dimension: string-keylist

  • Default: ""

Administrators

(user-defined)

Contains list of users with Administrator privileges.

  • Type and dimension: string-keylist

  • Default: ""

AuthorizationControl

(user-defined)

This attribute applies to clusters running in secure mode. It defines how VCS assigns cluster privileges to operating system (OS) users that have local or domain administrator privileges. The OS users must be defined as VCS users before modifying the attribute's default value.

The attribute can take the following values:

  • DEFAULT: Assigns cluster administrator privileges to users with local administrator and domain administrator privileges.

  • NONE: Does not assign cluster administrator privileges to users with local administrator and domain administrator privileges. Windows services running under local system accounts get cluster administrator privileges when they connect to VCS.

    You can override this setting by defining privileges for local and domain administrators in the VCS user list.

  • LOCAL : Assigns cluster administrator privileges to local administrators, but not to domain administrators.

    You can override this setting by defining privileges for domain administrators in the VCS user list.

  • GLOBAL : Assigns cluster administrator privileges to domain administrators, but not to local administrators.

    You can override this setting by defining privileges for local administrators in the VCS user list.

AutoClearQ

(System use only)

Lists the service groups scheduled to be auto-cleared. It also indicates the time at which the auto-clear for the group will be performed.

AutoStartTimeout

(user-defined)

If the local cluster cannot communicate with one or more remote clusters, this attribute specifies the number of seconds the VCS engine waits before initiating the AutoStart process for an AutoStart global service group.

  • Type and dimension: integer-scalar

  • Default: 150 seconds

AutoAddSystemtoCSG

(user-defined)

Indicates whether the newly joined or added systems in cluster become part of the SystemList of the ClusterService service group if the service group is configured. The value 1 (default) indicates that the new systems are added to SystemList of ClusterService. The value 0 indicates that the new systems are not added to SystemList of ClusterService.

  • Type and dimension: integer-scalar

  • Default: 1

BackupInterval

(user-defined)

Time period in minutes after which VCS backs up the configuration files if the configuration is in read-write mode.

The value 0 indicates VCS does not back up configuration files. Set this attribute to at least 3.

  • Type and dimension: integer-scalar

  • Default: 0

ClusState

(system use only)

Indicates the current state of the cluster.

  • Type and dimension: integer-scalar

  • Default: Not applicable.

ClusterAddress

(user-defined)

Specifies the cluster's virtual IP address (used by a remote cluster when connecting to the local cluster).

  • Type and dimension: string-scalar

  • Default: ""

ClusterLocation

(user-defined)

Specifies the location of the cluster.

  • Type and dimension: string-scalar

  • Default: ""

ClusterName

(user-defined)

The name of cluster.

  • Type and dimension: string-scalar

  • Default: ""

ClusterOwner

(user-defined)

This attribute used for VCS notification. VCS sends notifications to persons designated in this attribute when an event occurs related to the cluster. Note that while VCS logs most events, not all events trigger notifications.

Make sure to set the severity level at which you want notifications to be sent to ClusterOwner or to at least one recipient defined in the SmtpRecipients attribute of the NotifierMngr agent.

  • Type and dimension: string-scalar

  • Default: ""

  • Example: "jdoe@example.com"

ClusterRecipients

(user-defined)

This attribute is used for VCS email notification. VCS sends email notification to persons designated in this attribute when events related to the cluster occur and when the event's severity level is equal to or greater than the level specified in the attribute.

Make sure to set the severity level at which you want notifications to be sent to ClusterRecipients or to at least one recipient defined in the SmtpRecipients attribute of the NotifierMngr agent.

  • Type and dimension: string-association

  • email id: The e-mail address of the person registered as a recipient for notification.

    severity: The minimum level of severity at which notifications must be sent.

ClusterTime

(system use only)

The number of seconds since January 1, 1970. This is defined by the lowest node in running state.

  • Type and dimension: string-scalar

  • Default: Not applicable

CompareRSM

(system use only)

Indicates if VCS engine is to verify that replicated state machine is consistent. This can be set by running the hadebug command.

  • Type and dimension: integer-scalar

  • Default: 0

ConnectorState

(system use only)

Indicates the state of the wide-area connector (wac). If 0, wac is not running. If 1, wac is running and communicating with the VCS engine.

  • Type and dimension: integer-scalar

  • Default: Not applicable.

CounterInterval

(user-defined)

Intervals counted by the attribute GlobalCounter indicating approximately how often a broadcast occurs that will cause the GlobalCounter attribute to increase.

The default value of the GlobalCounter increment can be modified by changing CounterInterval. If you increase this attribute to exceed five seconds, consider increasing the default value of the ShutdownTimeout attribute.

  • Type and dimension: integer-scalar

  • Default: 5

CounterMissAction

(user-defined)

Specifies the action that must be performed when the GlobalCounter is not updated for CounterMissTolerance times the CounterInterval. Possible values are LogOnly and Trigger. If you set CounterMissAction to LogOnly, the system logs the message in Engine Log and Syslog. If you set CounterMissAction to Trigger, the system invokes a trigger which has default action of collecting the comms tar file.

  • Type and dimension: string-scalar

  • Default: LogOnly

CounterMissTolerance

(user-defined)

Specifies the time interval that can lapse since the last update of GlobalCounter before VCS reports an issue. If the GlobalCounter does not update within CounterMissTolerance times CounterInterval, VCS reports the issue. Depending on the CounterMissAction.value, appropriate action is performed.

  • Type and dimension: integer-scalar

  • Default: 20

CredRenewFrequency

(user-defined)

The number of days after which the VCS engine renews its credentials with the authentication broker. For example, the value 5 indicates that credentials are renewed every 5 days; the value 0 indicates that credentials are not renewed.

  • Type and dimension: integer-scalar

  • Default = 0

DumpingMembership

(system use only)

Indicates that the engine is writing or dumping the configuration to disk.

  • Type and dimension: vector

  • Default: Not applicable.

EnableFFDC

(user-defined)

Enables or disables FFDC logging. By default, FFDC logging is enabled.

  • Type and dimension: boolean-scalar

  • Default: 1

EnableVMAutoDiscovery

(user-defined)

Enables or disables auto discovery of virtual machines. By default, auto discovery of virtual machines is disabled.

  • Type and dimension: integer-scalar

  • Default: 0

EnablePBF

(user-defined)

Enables or disables priority based failover. When set to 1 (one), VCS gives priority to the online of high priority service group, by ensuring that its Load requirement is met on the system.

  • Type and dimension: boolean-scalar

  • Default: 0 (zero)

EnginePriority

(user-defined)

The priority in which HAD runs. Generally, a greater priority value indicates higher scheduling priority. A range of priority values is assigned to each scheduling class. For more information on the range of priority values, see the operating system documentation.

  • Type and dimension: string-scalar

  • Default: ""

EngineShutdown

(user-defined)

Defines the options for the hastop command. The attribute can assume the following values:

Enable - Process all hastop commands. This is the default behavior.

Disable - Reject all hastop commands.

DisableClusStop - Do not process the hastop -all command; process all other hastop commands.

PromptClusStop - Prompt for user confirmation before running the hastop -all command; process all other hastop commands.

PromptLocal - Prompt for user confirmation before running the hastop -local command; reject all other hastop commands.

PromptAlways - Prompt for user confirmation before running any hastop command.

  • Type and dimension: string-scalar

  • Default: Enable

GlobalCounter

(system use only)

This counter increases incrementally by one for each counter interval. It increases when the broadcast is received.

VCS uses the GlobalCounter attribute to measure the time it takes to shut down a system. By default, the GlobalCounter attribute is updated every five seconds. This default value, combined with the 600-second default value of the ShutdownTimeout attribute, means if system goes down within 120 increments of GlobalCounter, it is treated as a fault. Change the value of the CounterInterval attribute to modify the default value of GlobalCounter increment.

  • Type and dimension: integer-scalar

  • Default: Not applicable

Guests

(user-defined)

List of operating system user accounts that have Guest privileges on the cluster.

This attribute is valid clusters running in secure mode.

  • Type and dimension: string-keylist

  • Default: ""

GuestGroups

(user-defined)

List of operating system user groups that have Guest privilege on the cluster.

  • Type and dimension: string-keylist

  • Default: Not applicable

DefaultGuestAccess

(user-defined)

Indicates whether any authenticated user should have guest access to the cluster by default. The default guest access can be:

  • 0: Guest access for privileged users only.

  • 1: Guest access for everyone.

  • Type and dimension: boolean-scalar

  • Default: 0

GroupLimit

(user-defined)

Maximum number of service groups.

  • Type and dimension: integer-scalar

  • Default: 200

HacliUserLevel

(user-defined)

This attribute has two, case-sensitive values:

NONE - hacli is disabled for all users regardless of role.

COMMANDROOT - hacli is enabled for root only.

Note:

The command haclus -modify HacliUserLevel can be executed by root only.

  • Type and dimension: string-scalar

  • Default: NONE

HostAvailableMeters

(System use only)

Lists the meters that are available for measuring system resources. You cannot configure this attribute in the main.cf file.

  • Type and dimension: string-association

    Keys are the names of parameters and values are the names of meter libraries.

HostMeters

(user-defined)

  • Type and dimension: string-keylist

  • You can configure this attribute in the main.cf file. You cannot modify the value at run time.

LockMemory

(user-defined)

Controls the locking of VCS engine pages in memory. This attribute has the following values. Values are case-sensitive:

ALL: Locks all current and future pages.

CURRENT: Locks current pages.

NONE: Does not lock any pages.

  • Type and dimension: string-scalar

  • Default: ALL

LogClusterUUID

(user-defined)

Enables or disables logging of the cluster UUID in each log message. By default, cluster UUID is not logged.

  • Type and dimension: boolean-scalar

  • Default: 0

LogSize

(user-defined)

Indicates the size of engine log files in bytes.

Minimum value is = 65536 (equal to 64KB)

Maximum value = 134217728 (equal to 128MB)

  • Type and dimension: integer-scalar

  • Default: 134217728 (128 MB)

MeterControl

(user-defined)

Indicates the intervals at which metering and forecasting for the system attribute AvailableCapacity are done for the keys specified in HostMeters.

  • Type and dimension: integer-association

    This attribute includes the following keys:

  • MeterInterval

    Frequency in seconds at which metering is done by the HostMonitor agent. The value for this key can equal or exceed 30. The default value is 120 indicating that the HostMonitor agent meters available capacity and updates the System attribute AvailableCapacity every 120 seconds. The HostMonitor agent checks for changes in the available capacity for every monitoring cycle and when there is a change, the HostMonitor agent updates the values in the same monitoring cycle . The MeterInterval value applies only if Statistics is set to Enabled or MeterHostOnly.

  • ForecastCycle

    The number of metering cycles after which forecasting of available capacity is done. The value for this key can equal or exceed 1. The default value is 3 indicating that forecasting of available capacity is done after every 3 metering cycles. Assuming the default MeterInterval value of 120 seconds, forecasting is done after 360 seconds or 6 minutes. The ForecastCycle value applies only if Statistics is set to Enabled.

You can configure this attribute in main.cf. You cannot modify the value at run time. The values of MeterInterval and ForecastCycle apply to all keys of HostMeters.

MeterUnit

Represents units for parameters that are metered.

  • Type and dimension: string-association

You can configure this attribute in main.cf; if configured in main.cf, then it must contain units for all the keys as specified in HostMeters. You cannot modify the value at run time.

  • AvailableCapacity

  • HostAvailableForecast

  • Capacity

  • ReservedCapacity

The values of keys such as Mem and Swap can be represented in MB or GB, and CPU can be represented in CPU, MHz or GHz.

MeterWeight

(user-defined)

Indicates the default meter weight for the service groups in the cluster. You can configure this attribute in the main.cf file, but you cannot modify the value at run time. If the attribute is defined in the main.cf file, it must have at least one key defined. The weight for the key must be in the range of 0 to 10. Only keys from HostAvailableMeters are allowed in this attribute.

  • Type and dimension: integer-association

Notifier

(system use only)

Indicates the status of the notifier in the cluster; specifically:

State - Current state of notifier, such as whether or not it is connected to VCS.

Host - The host on which notifier is currently running or was last running. Default = None

Severity - The severity level of messages queued by VCS for notifier. Values include Information, Warning, Error, and SevereError. Default = Warning

Queue - The size of queue for messages queued by VCS for notifier.

  • Type and dimension: string-association

  • Default: Different values for each parameter.

OpenExternalCommunicationPort

(user-defined)

Indicates whether communication over the external communication port for VCS is allowed or not. By default, the external communication port for VCS is 14141.

  • Type and dimension: string-scalar

  • Valid values: YES, NO

  • Default: YES

  • YES: The external communication port for VCS is open.

  • NO: The external communication port for VCS is not open.

OperatorGroups

(user-defined)

List of operating system user groups that have Operator privileges on the cluster.

This attribute is valid clusters running in secure mode.

  • Type and dimension: string-keylist

  • Default: ""

Operators

(user-defined)

List of users with Cluster Operator privileges.

  • Type and dimension: string-keylist

  • Default: ""

PanicOnNoMem

(user-defined)

Indicate the action that you want VCS engine (HAD) to take if it cannot receive messages from GAB due to low-memory.

  • If the value is 0, VCS exits with warnings.

  • If the value is 1, VCS calls the GAB library routine to panic the system.

  • Default: 0

PrintMsg

(user-defined)

Enables logging TagM messages in engine log if set to 1.

  • Type and dimension: boolean-scalar

  • Default: 0

ProcessPriority

(user-defined)

The priority of processes created by the VCS engine. For example triggers.

  • Type and dimension: string-scalar

  • Default: ""

ProtocolNumber

(System use only)

A cluster-level attribute that displays the cluster protocol number for the cluster.

  • Type and dimension: Integer-scalar

  • Default: Not applicable

ReadOnly

(user-defined)

Indicates that cluster is in read-only mode.

  • Type and dimension: integer-scalar

  • Default: 1

ResourceLimit

(user-defined)

Maximum number of resources.

  • Type and dimension: integer-scalar

  • Default: 5000

SecInfo256

(user-defined)

Enables the creation of secure passwords when this attribute is added to the main.cf file with the security key as the value of the attribute.

  • Type and dimension: string-scalar

  • Default: ""

See Encrypting agent passwords.

IV256

(user-defined)

Enables the creation of secure passwords when this attribute is added to the main.cf file. This initialization vector (IV) is a fixed-size input to a cryptographic primitive that is typically required to be random. It adds randomness to the beginning of the encryption process.

  • Type and dimension: string-scalar

  • Default: ""

Note:

This attribute is used to encrypt or decrypt passwords when the cluster protocol version is 10000 or earlier.

See Encrypting agent passwords.

IV256List

(user-defined)

Enables the creation of secure passwords by maintaining a list of key-value pairs. Each encrypted password is saved as a key along with the IV that was used to encrypt the password.

  • Type and dimension: string-association

  • Default: ""

This attribute gets added to or update in the main.cf file at the following events:

  • When a user gets added to a cluster using the hauser -add command in case of a non-secure cluster.

  • When you use the vcsencrypt -vcs or the vcsencrypt -agent command to encrypt the VCS password or an agent password respectively.

If you use any of these commands, you do not need to run haconf -dump to make the configuration read-only again; this task is handled in the background.

You cannot modify IV256List by using the haconf - modify command. However, you can use the vcsencrypt -delkey encryptedPasswordValue command to delete stale entries from this attribute.

Note:

This attribute and the -delkey option of the vcsencrypt command are applicable only with cluster protocol version 11000 or later.

See Encrypting agent passwords.

SecInfoLevel

(user-defined)

Denotes the password encryption privilege level.

  • Type and dimension: string-scalar

  • Default: R

See Encrypting agent passwords.

SecureClus

(user-defined)

Indicates whether the cluster runs in secure mode. The value 1 indicates the cluster runs in secure mode. This attribute cannot be modified when VCS is running.

  • Type and dimension: boolean-scalar

  • Default: 0

SiteAware

(user-defined)

Indicates whether sites are configured for a cluster or not.

  • Type and dimension: boolean-scalar

  • Default: 0

    Possible values are:

    • 1: Sites are configured.

    • 0: Sites are not configured.

You can configure a site from Arctera InfoScale Operations Manager . This attribute will be automatically set to 1 when configured using Arctera InfoScale Operations Manager. If site information is not configured for some nodes in the cluster, those nodes are placed under a default site that has the lowest preference.

SourceFile

(user-defined)

File from which the configuration is read. Do not configure this attribute in main.cf.

Make sure the path exists on all nodes before running a command that configures this attribute.

  • Type and dimension: string-scalar

  • Default: Not applicable.

Statistics

(user-defined)

Indicates if statistics gathering is enabled and whether the FailOverPolicy can be set to BiggestAvailable.

  • Type and dimension: string-scalar

  • Default: Enabled

    You cannot modify the value at run time.

    Possible values are:

    • Enabled: The HostMonitor agent meters host utilization and forecasts the available capacity for the systems in the cluster. With this value set, FailOverPolicy for any service group cannot be set to Load.

    • MeterHostOnly: The HostMonitor agent meters host utilization but it does not forecast the available capacity for the systems in the cluster. The service group attribute FailOverPolicy cannot be set to BiggestAvailable.

    • Disabled: The HostMonitor agent is not started. Both metering of host utilization and forecasting of available capacity are disabled. The service group attribute FailOverPolicy cannot be set to BiggestAvailable.

See Service group attributes.

Stewards

(user-defined)

The IP address and hostname of systems running the steward process.

  • Type and dimension: string-keylist

  • {}

SystemRebootAction

(user-defined)

Determines whether frozen service groups are ignored on system reboot.

  • Type and dimension: string-keylist

  • Default: ""

If the SystemRebootAction value is IgnoreFrozenGroup, VCS ignores service groups that are frozen (TFrozen and Frozen) and takes the remaining service groups offline. If the frozen service groups have firm dependencies or hard dependencies on any other service groups which are not frozen, VCS gives an error.

If the SystemRebootAction value is "", VCS tries to take all service groups offline. Because VCS cannot be gracefully stopped on a node where a frozen service group is online, applications on the node might get killed.

Note:

The SystemRebootAction attribute applies only on system reboot and system shutdown.

TypeLimit

(user-defined)

Maximum number of resource types.

  • Type and dimension: integer-scalar

  • Default: 100

UserNames

(user-defined)

List of VCS users. The installer uses admin as the default user name.

  • Type and dimension: string-association

  • Default: ""

VCSFeatures

(system use only)

Indicates which VCS features are enabled. Possible values are:

0 - No features are enabled (VCS Simulator)

1 - L3+ is enabled

2 - Global Cluster Option is enabled

Even though the VCSFeatures is an integer attribute, when you query the value with the haclus -value command or the haclus -display command, it displays as the string L10N for value 1 and DR for value 2.

  • Type and dimension: integer-scalar

  • Default: Not applicable.

VCSMode

(system use only)

Denotes the mode for which VCS is licensed.

Even though the VCSMode is an integer attribute, when you query the value with the haclus -value command or the haclus -display command, it displays as the string UNKNOWN_MODE for value 0 and VCS for value 7.

  • Type and dimension: integer-scalar

  • Default: Not applicable

WACPort

(user-defined)

The TCP port on which the wac (Wide-Area Connector) process on the local cluster listens for connection from remote clusters. Type and dimension: integer-scalar

  • Default: 14155

More Information

Scheduling automatic backups for VCS configuration files

About VCS event notification