Search <book_title>...

InfoScale™ 9.0 Storage Foundation and High Availability Configuration and Upgrade Guide - Linux

Last Published: 2025-04-18

Product(s): InfoScale & Storage Foundation (9.0)

Platform: Linux

Section I. Introduction to SFHA
1. Introducing Storage Foundation and High Availability
Section II. Configuration of SFHA
Section III. Upgrade of SFHA
Section IV. Post-installation tasks
1. Performing post-installation tasks
  1. Switching on Quotas
  2. About configuring authentication for SFDB tools
    1. Configuring vxdbd for SFDB tools authentication
Section V. Adding and removing nodes
1. Adding a node to SFHA clusters
2. Removing a node from SFHA clusters
  1. Removing a node from a SFHA cluster
Section VI. Configuration and upgrade reference

About the LLT and GAB configuration files

Low Latency Transport (LLT) and Group Membership and Atomic Broadcast (GAB) are VCS communication services. LLT requires /etc/llthosts and /etc/llttab files. GAB requires /etc/gabtab file.

Table: LLT configuration files lists the LLT configuration files and the information that these files contain.

Table: LLT configuration files

File	Description
/etc/sysconfig/llt	This file stores the start and stop environment variables for LLT: LLT_START - Defines the startup behavior for the LLT module after a system reboot. Valid values include: 1 - Indicates that LLT is enabled to start up. 0 - Indicates that LLT is disabled to start up. LLT_STOP - Defines the shutdown behavior for the LLT module during a system shutdown. Valid values include: 1 - Indicates that LLT is enabled to shut down. 0 - Indicates that LLT is disabled to shut down. The installer sets the value of these variables to 1 at the end of SFHA configuration. If you manually configured VCS, make sure you set the values of these environment variables to 1. Assign the buffer pool memory for RDMA operations: LLT_BUFPOOL_MAXMEM - Maximum assigned memory that LLT can use for the LLT buffer pool. This buffer pool is used to allocate memory for RDMA operations and packet allocation, which are delivered to the LLT clients. The default value is calculated based on the total system memory, the minimum value is 1GB, and the maximum value is 10GB. You must specify the value in GB. Define the number of RDMA queue pairs per LLT link LLT_RDMA_QPS - Maximum number of RDMA queue pairs per LLT link. You can change the value of this parameter in the `/etc/sysconfig/llt` file. The default value is 4. However, you can extend this value to maximum 8 queue pairs per LLT link. Enable or disable the adaptive window feature: For performance reason, the adaptive window feature (LLT_ENABLE_AWINDOW ) is enabled by default for 5 (cfs) and port 24(cvm). You can disable the adaptive window feature by manually changing the value of the LLT_ENABLE_AWINDOW parameter to zero. To enable adaptive window for ports other than 5 and 24, add the port numbers in LLT_AW_PORT_LIST separated by comma. For example: LLT_AW_PORT_LIST=: '"5,24,0,1,5,14"'. If you want to disable the adaptive window feature for any of the ports, remove that specific port from this parameter. Example: LLT_AW_PORT_LIST='"5"' Configure LLT over TCP When you configure LLT over the Transmission Control Protocol (TCP) layer for clusters using wide-area networks and routers, you can use the LLT_TCP_CONNS parameter to control the number of TCP connections that can be established between peer nodes. The default value of this parameter is 8 connections. However, you can extend this value to maximum 64 connections.
/etc/llthosts	The file llthosts is a database that contains one entry per system. This file links the LLT system ID (in the first column) with the LLT host name. This file must be identical on each node in the cluster. A mismatch of the contents of the file can cause indeterminate behavior in the cluster. For example, the file /etc/llthosts contains the entries that resemble: 0 sys1 1 sys2
/etc/llttab	The file llttab contains the information that is derived during installation and used by the utility lltconfig(1M). After installation, this file lists the LLT network links that correspond to the specific system. For example, the file /etc/llttab contains the entries that resemble: set-node sys1 set-cluster 2 link eth1 eth1 - ether - - link eth2 eth2 - ether - - If you use aggregated interfaces, then the file contains the aggregated interface name instead of the eth-MAC_address. set-node sys1 set-cluster 2 link eth1 eth-00:04:23:AC:12:C4 - ether - - link eth2 eth-00:04:23:AC:12:C5 - ether - - The first line identifies the system. The second line identifies the cluster (that is, the cluster ID you entered during installation). The next two lines begin with the link command. These lines identify the two network cards that the LLT protocol uses. If you configured a low priority link under LLT, the file also includes a "link-lowpri" line. Refer to the llttab(4) manual page for details about how the LLT configuration may be modified. The manual page describes the ordering of the directives in the llttab file.

File

Description

/etc/sysconfig/llt

This file stores the start and stop environment variables for LLT:

LLT_START - Defines the startup behavior for the LLT module after a system reboot. Valid values include:
1 - Indicates that LLT is enabled to start up.
0 - Indicates that LLT is disabled to start up.
LLT_STOP - Defines the shutdown behavior for the LLT module during a system shutdown. Valid values include:
1 - Indicates that LLT is enabled to shut down.
0 - Indicates that LLT is disabled to shut down.

The installer sets the value of these variables to 1 at the end of SFHA configuration.

If you manually configured VCS, make sure you set the values of these environment variables to 1.

Assign the buffer pool memory for RDMA operations:

LLT_BUFPOOL_MAXMEM - Maximum assigned memory that LLT can use for the LLT buffer pool. This buffer pool is used to allocate memory for RDMA operations and packet allocation, which are delivered to the LLT clients.
The default value is calculated based on the total system memory, the minimum value is 1GB, and the maximum value is 10GB. You must specify the value in GB.

Define the number of RDMA queue pairs per LLT link

LLT_RDMA_QPS - Maximum number of RDMA queue pairs per LLT link. You can change the value of this parameter in the /etc/sysconfig/llt file. The default value is 4. However, you can extend this value to maximum 8 queue pairs per LLT link.

Enable or disable the adaptive window feature:

For performance reason, the adaptive window feature (LLT_ENABLE_AWINDOW ) is enabled by default for 5 (cfs) and port 24(cvm). You can disable the adaptive window feature by manually changing the value of the LLT_ENABLE_AWINDOW parameter to zero.
To enable adaptive window for ports other than 5 and 24, add the port numbers in LLT_AW_PORT_LIST separated by comma. For example: LLT_AW_PORT_LIST=: '"5,24,0,1,5,14"'.
If you want to disable the adaptive window feature for any of the ports, remove that specific port from this parameter. Example: LLT_AW_PORT_LIST='"5"'

Configure LLT over TCP

When you configure LLT over the Transmission Control Protocol (TCP) layer for clusters using wide-area networks and routers, you can use the LLT_TCP_CONNS parameter to control the number of TCP connections that can be established between peer nodes. The default value of this parameter is 8 connections. However, you can extend this value to maximum 64 connections.

/etc/llthosts

The file llthosts is a database that contains one entry per system. This file links the LLT system ID (in the first column) with the LLT host name. This file must be identical on each node in the cluster. A mismatch of the contents of the file can cause indeterminate behavior in the cluster.

For example, the file /etc/llthosts contains the entries that resemble:

    0      sys1
    1      sys2

/etc/llttab

The file llttab contains the information that is derived during installation and used by the utility lltconfig(1M). After installation, this file lists the LLT network links that correspond to the specific system.

For example, the file /etc/llttab contains the entries that resemble:

    set-node sys1
    set-cluster 2
    link eth1 eth1 - ether - -
    link eth2 eth2 - ether - -

If you use aggregated interfaces, then the file contains the aggregated interface name instead of the eth-MAC_address.

    set-node sys1
    set-cluster 2
    link eth1 eth-00:04:23:AC:12:C4 - ether - -
    link eth2 eth-00:04:23:AC:12:C5 - ether - -

The first line identifies the system. The second line identifies the cluster (that is, the cluster ID you entered during installation). The next two lines begin with the link command. These lines identify the two network cards that the LLT protocol uses.

If you configured a low priority link under LLT, the file also includes a "link-lowpri" line.

Refer to the llttab(4) manual page for details about how the LLT configuration may be modified. The manual page describes the ordering of the directives in the llttab file.

Table: GAB configuration files lists the GAB configuration files and the information that these files contain.

Table: GAB configuration files

File	Description
/etc/sysconfig/gab	This file stores the start and stop environment variables for GAB: GAB_START - Defines the startup behavior for the GAB module after a system reboot. Valid values include: 1 - Indicates that GAB is enabled to start up. 0 - Indicates that GAB is disabled to start up. GAB_STOP - Defines the shutdown behavior for the GAB module during a system shutdown. Valid values include: 1 - Indicates that GAB is enabled to shut down. 0 - Indicates that GAB is disabled to shut down. The installer sets the value of these variables to 1 at the end of SFHA configuration.
/etc/gabtab	After you install SFHA, the file /etc/gabtab contains a gabconfig(1) command that configures the GAB driver for use. The file /etc/gabtab contains a line that resembles: /sbin/gabconfig -c -nN The -c option configures the driver for use. The *-nN* specifies that the cluster is not formed until at least N nodes are ready to form the cluster. Arctera recommends that you set N to be the total number of nodes in the cluster. Note: Arctera does not recommend the use of the -c -x option for /sbin/gabconfig. Using -c -x can lead to a split-brain condition. Use the -c option for /sbin/gabconfig to avoid a split-brain condition.

File

Description

/etc/sysconfig/gab

This file stores the start and stop environment variables for GAB:

GAB_START - Defines the startup behavior for the GAB module after a system reboot. Valid values include:
1 - Indicates that GAB is enabled to start up.
0 - Indicates that GAB is disabled to start up.
GAB_STOP - Defines the shutdown behavior for the GAB module during a system shutdown. Valid values include:
1 - Indicates that GAB is enabled to shut down.
0 - Indicates that GAB is disabled to shut down.

The installer sets the value of these variables to 1 at the end of SFHA configuration.

/etc/gabtab

After you install SFHA, the file /etc/gabtab contains a gabconfig(1) command that configures the GAB driver for use.

The file /etc/gabtab contains a line that resembles:

    /sbin/gabconfig -c -nN

The -c option configures the driver for use. The -nN specifies that the cluster is not formed until at least N nodes are ready to form the cluster. Arctera recommends that you set N to be the total number of nodes in the cluster.

Note:

Arctera does not recommend the use of the -c -x option for /sbin/gabconfig. Using -c -x can lead to a split-brain condition. Use the -c option for /sbin/gabconfig to avoid a split-brain condition.