InfoScale™ 9.0 Storage Foundation Cluster File System High Availability Administrator's Guide - Linux
- Section I. Introducing Storage Foundation Cluster File System High Availability
- Overview of Storage Foundation Cluster File System High Availability
- About Veritas File System
- About Veritas Replicator
- How Dynamic Multi-Pathing works
- How Volume Manager works
- How Volume Manager works with the operating system
- How Volume Manager handles storage management
- Volume layouts in Veritas Volume Manager
- Online relayout
- Volume resynchronization
- Dirty region logging
- Volume snapshots
- FastResync
- How VxVM handles hardware clones or snapshots
- Volume encryption
- How Veritas File System works
- How Storage Foundation Cluster File System High Availability works
- About Storage Foundation Cluster File System High Availability architecture
- About Veritas File System features supported in cluster file systems
- About single network link and reliability
- About I/O fencing
- About preventing data corruption with I/O fencing
- About I/O fencing components
- About server-based I/O fencing
- About secure communication between the SFCFSHA cluster and CP server
- How Cluster Volume Manager works
- Overview of clustering
- Cluster Volume Manager (CVM) tolerance to storage connectivity failures
- Storage disconnectivity and CVM disk detach policies
- CVM initialization and configuration
- Dirty region logging in cluster environments
- Multiple host failover configurations
- About Flexible Storage Sharing
- Application isolation in CVM environments with disk group sub-clustering
- Overview of Storage Foundation Cluster File System High Availability
- Section II. Provisioning storage
- Provisioning new storage
- Advanced allocation methods for configuring storage
- Customizing allocation behavior
- Using rules to make volume allocation more efficient
- Understanding persistent attributes
- Customizing disk classes for allocation
- Specifying allocation constraints for vxassist operations with the use clause and the require clause
- Creating volumes of a specific layout
- Customizing allocation behavior
- Creating and mounting VxFS file systems
- Creating a VxFS file system
- Mounting a VxFS file system
- tmplog mount option
- ioerror mount option
- largefiles and nolargefiles mount options
- Resizing a file system
- Monitoring free space
- Extent attributes
- Section III. Administering multi-pathing with DMP
- Administering Dynamic Multi-Pathing
- Discovering and configuring newly added disk devices
- About discovering disks and dynamically adding disk arrays
- How to administer the Device Discovery Layer
- Administering DMP using the vxdmpadm utility
- Gathering and displaying I/O statistics
- Specifying the I/O policy
- Discovering and configuring newly added disk devices
- Dynamic Reconfiguration of devices
- Reconfiguring a LUN online that is under DMP control using the Dynamic Reconfiguration tool
- Manually reconfiguring a LUN online that is under DMP control
- Managing devices
- Displaying disk information
- Changing the disk device naming scheme
- Adding and removing disks
- Event monitoring
- Administering Dynamic Multi-Pathing
- Section IV. Administering Storage Foundation Cluster File System High Availability
- Administering Storage Foundation Cluster File System High Availability and its components
- Administering CFS
- About the mount, fsclustadm, and fsadm commands
- When the CFS primary node fails
- About Snapshots on SFCFSHA
- Administering VCS
- Administering CVM
- About setting cluster node preferences for master failover
- About changing the CVM master manually
- Importing disk groups as shared
- Administering Flexible Storage Sharing
- Administering ODM
- About administering I/O fencing
- About the vxfentsthdw utility
- Testing the coordinator disk group using the -c option of vxfentsthdw
- About the vxfenadm utility
- About the vxfenclearpre utility
- About the vxfenswap utility
- About administering the coordination point server
- About migrating between disk-based and server-based fencing configurations
- Migrating between fencing configurations using response files
- About the vxfentsthdw utility
- Administering SFCFSHA global clusters
- Enabling S3 server
- Using Clustered NFS
- Understanding how Clustered NFS works
- Configure and unconfigure Clustered NFS
- Administering Clustered NFS
- Samples for configuring a Clustered NFS
- Using Common Internet File System
- Deploying Oracle with Clustered NFS
- Administering sites and remote mirrors
- About sites and remote mirrors
- Fire drill - testing the configuration
- Changing the site name
- Administering the Remote Mirror configuration
- Failure and recovery scenarios
- Administering iSCSI with SFCFSHA
- Administering datastores with SFCFSHA
- Administering Storage Foundation Cluster File System High Availability and its components
- Section V. Optimizing I/O performance
- Veritas File System I/O
- Veritas Volume Manager I/O
- Managing application I/O workloads using maximum IOPS settings
- Section VI. Veritas Extension for Oracle Disk Manager
- Using Veritas Extension for Oracle Disk Manager
- About Oracle Disk Manager
- About Oracle Disk Manager and Oracle Managed Files
- Using Cached ODM
- Using Veritas Extension for Oracle Disk Manager
- Section VII. Using Point-in-time copies
- Understanding point-in-time copy methods
- When to use point-in-time copies
- About Storage Foundation point-in-time copy technologies
- Volume-level snapshots
- Storage Checkpoints
- About FileSnaps
- About snapshot file systems
- Administering volume snapshots
- Traditional third-mirror break-off snapshots
- Full-sized instant snapshots
- Creating instant snapshots
- Adding an instant snap DCO and DCO volume
- Controlling instant snapshot synchronization
- Creating instant snapshots
- Cascaded snapshots
- Adding a version 0 DCO and DCO volume
- Administering Storage Checkpoints
- Storage Checkpoint administration
- Administering FileSnaps
- Administering snapshot file systems
- Understanding point-in-time copy methods
- Section VIII. Optimizing storage with Storage Foundation Cluster File System High Availability
- Understanding storage optimization solutions in Storage Foundation Cluster File System High Availability
- About SmartMove
- Migrating data from thick storage to thin storage
- Maintaining Thin Storage with Thin Reclamation
- Reclamation of storage on thin reclamation arrays
- Identifying thin and thin reclamation LUNs
- InfoScale 4K sector device support solution
- Understanding storage optimization solutions in Storage Foundation Cluster File System High Availability
- Section IX. Maximizing storage utilization
- Understanding storage tiering with SmartTier
- Creating and administering volume sets
- Multi-volume file systems
- Features implemented using multi-volume file system (MVFS) support
- Adding a volume to and removing a volume from a multi-volume file system
- Volume encapsulation
- Load balancing
- Administering SmartTier
- About SmartTier
- Placement classes
- Administering placement policies
- File placement policy rules
- Multiple criteria in file placement policy rule statements
- Using SmartTier with solid state disks
- Sub-file relocation
- Administering hot-relocation
- How hot-relocation works
- Moving relocated subdisks
- Compressing files
- About compressing files
- Use cases for compressing files
- Section X. Administering and protecting storage
- Managing volumes and disk groups
- Rules for determining the default disk group
- Moving volumes or disks
- Monitoring and controlling tasks
- Performing online relayout
- Adding a mirror to a volume
- Encrypting existing volumes
- Managing disk groups
- Disk group versions
- Displaying disk group information
- Creating a disk group
- Importing a disk group
- Moving disk groups between systems
- Importing a disk group containing hardware cloned disks
- Handling conflicting configuration copies
- Destroying a disk group
- Backing up and restoring disk group configuration data
- Managing plexes and subdisks
- Erasure coding in Veritas InfoScale storage environments
- Erasure coding deployment scenarios
- Customized failure domain
- Decommissioning storage
- Rootability
- Encapsulating a disk
- Rootability
- Sample supported root disk layouts for encapsulation
- Encapsulating and mirroring the root disk
- Administering an encapsulated boot disk
- Quotas
- Using Veritas File System quotas
- File Change Log
- Support for protection against ransomware
- Non-modifiable storage checkpoints
- Soft WORM storage
- Secure file system
- Secure file system for Oracle Single Instance
- Secure file system for PostgreSQL database
- Managing volumes and disk groups
- Section XI. Reference
- Appendix A. Reverse path name lookup
- Appendix B. Tunable parameters
- Tuning the VxFS file system
- Methods to change Dynamic Multi-Pathing tunable parameters
- Tunable parameters for VxVM
- Methods to change Veritas Volume Manager tunable parameters
- About LLT tunable parameters
- About GAB tunable parameters
- About VXFEN tunable parameters
- Appendix C. Command reference
- Appendix D. Creating a starter database
- Appendix E. Executive Order logging
Using Stripe Group and Stripe Confined Group while creating erasure coded volume
While creating an erasure coded volume, user can specify both these constraints, however if nothing is specified the Stripe Group value will be the host:
Stripe Group (SG): Stripe Group defines the separation criteria for allocating the column, where
One column is allocated from one or more separation criteria members.
One stripe group member contributes to at max one column.
For example: Suppose that one stripe group has two disks, then these two disks contributes to a single column of a volume. However you cannot use first disk in one column and second disks in another column of a same volume.
Stripe Confine Group (SCG): Stripe Confine Group defines the confinement group for allocating column where,
One column is confined within a failure domain, specified in that stripe confined class.
One stripe confined group member may contain either one or more complete columns or none, however no partial column (s) is present on any of the SCG members.
Considering both the constraints, SCG is at a higher level of hierarchy than SG. So if SCG is a node, then SG can be specified as tray. However if SG is specified as node then SCG cannot be specified as a tray.
For example: Suppose 1 column is entirely within a particular SCG group member, then another SCG group member cannot have the same column.
Note:
When you create an erasure coded volume, logs are maintained by default for that volume. In this case if the stripe confined group constrain is not specified, the value is set as host by default.
To use SG and SCG while creating a volume
- Specify layout, number of data columns and parity columns.
See Creating erasure coded volumes on a single node with DAS or SAN storage.
- Specify the Stripe Group and Stripe Confine Group.
To specify SG, use the following command:
# stripe=dtag:<tag name>
To specify SCG, use the following command:
# Stripeconfine=dtag:<tag name>
- Create a volume with stripe group and stripe confine group using the following command:
# vxassist -g <dg_name> make <vol_name> <vol_size> layout=ecoded stripe=dtag:<tag name> stripeconfine=dtag:<tag name>
Now let us see few examples with different configurations for creating volume using custom failure domain
The following example erasure coded configuration uses 3 data columns and 1 parity column. Stripe group is a tray, and stripe confine group is a node. The configuration is such that one tray will have either one complete or one partial column on it but will not have 2 columns and one column will not span across two nodes.
In this case both Stripe Group (SG) and Stripe Confine Group is specified.
# vxassist -g ecdg make ecvol1 1g ncol=3 nparity=1 layout=ecoded stripe=dtag:tray stripeconfine=dtag:node
# vxprint -g ecdg ecvol1 TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg ecdg ecdg - - - - - - dm vmr720-18vm3_vmdk0_0 vmr720-18vm3_vmdk0_0 - 4128464 - - - - dm vmr720-18vm3_vmdk0_1 vmr720-18vm3_vmdk0_1 - 4128464 - - - - dm vmr720-18vm4_vmdk0_0 vmr720-18vm4_vmdk0_0 - 4128464 - REMOTE - - dm vmr720-18vm4_vmdk0_1 vmr720-18vm4_vmdk0_1 - 4128464 - REMOTE - - dm vmr720-18vm5_vmdk0_0 vmr720-18vm5_vmdk0_0 - 4128464 - REMOTE - - dm vmr720-18vm5_vmdk0_1 vmr720-18vm5_vmdk0_1 - 4128464 - REMOTE - - dm vmr720-18vm6_vmdk0_0 vmr720-18vm6_vmdk0_0 - 4128464 - REMOTE - - dm vmr720-18vm6_vmdk0_1 vmr720-18vm6_vmdk0_1 - 4128464 - REMOTE - - v ecvol1 fsgen ENABLED 2097408 - SYNC - - pl ecvol1-01 ecvol1 ENABLED 2097408 - ACTIVE - - sd vmr720-18vm3_vmdk0_0-02 ecvol1-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm3_vmdk0_0-01 ecvol1-01 ENABLED 699136 0 - - - sd vmr720-18vm4_vmdk0_0-02 ecvol1-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm4_vmdk0_0-01 ecvol1-01 ENABLED 699136 0 - - - sd vmr720-18vm5_vmdk0_0-02 ecvol1-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm5_vmdk0_0-01 ecvol1-01 ENABLED 699136 0 - - - sd vmr720-18vm6_vmdk0_0-02 ecvol1-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm6_vmdk0_0-01 ecvol1-01 ENABLED 699136 0 - - - dc ecvol1_dco ecvol1 - - - - - - v ecvol1_dcl gen ENABLED 67840 - ACTIVE - - pl ecvol1_dcl-01 ecvol1_dcl ENABLED 67840 - ACTIVE - - sd vmr720-18vm3_vmdk0_0-03 ecvol1_dcl-01 ENABLED 67840 0 - - - pl ecvol1_dcl-02 ecvol1_dcl ENABLED 67840 - ACTIVE - - sd vmr720-18vm4_vmdk0_0-03 ecvol1_dcl-02 ENABLED 67840 0 - - -
The following example erasure code volume configuration uses 3 columns and 1 parity, where node is a stripe group. Create a volume with columns separated by different nodes. The configuration is such that one node will have either one complete or one partial column on it but will not have 2 columns. However, one column can span across one or more nodes.
In this case only Stripe Group (SG) is specified.
# vxassist -g ecdg make ecvol2 1g ncol=3 nparity=1 layout=ecoded stripe=dtag:node
# vxprint -g ecdg ecvol2 TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 v ecvol2 fsgen ENABLED 2097408 - SYNC - - pl ecvol2-01 ecvol2 ENABLED 2097408 - ACTIVE - - sd vmr720-18vm5_vmdk0_1-02 ecvol2-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm5_vmdk0_1-01 ecvol2-01 ENABLED 699136 0 - - - sd vmr720-18vm6_vmdk0_1-02 ecvol2-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm6_vmdk0_1-01 ecvol2-01 ENABLED 699136 0 - - - sd vmr720-18vm3_vmdk0_1-02 ecvol2-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm3_vmdk0_1-01 ecvol2-01 ENABLED 699136 0 - - - sd vmr720-18vm4_vmdk0_1-02 ecvol2-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm4_vmdk0_1-01 ecvol2-01 ENABLED 699136 0 - - - dc ecvol2_dco ecvol2 - - - - - - v ecvol2_dcl gen ENABLED 67840 - ACTIVE - - pl ecvol2_dcl-01 ecvol2_dclENABLED 67840 - ACTIVE - - sd vmr720-18vm3_vmdk0_1-03 ecvol2_dcl-01 ENABLED 67840 0 - - - pl ecvol2_dcl-02 ecvol2_dclENABLED 67840 - ACTIVE - - sd vmr720-18vm4_vmdk0_1-03 ecvol2_dcl-02 ENABLED 67840 0 - - -
The following example, shows how the capacity is exhausted when the stripe group is a node. In this case an erasure code volume configuration uses 4 columns and 2 parity, where node is a stripe group. Create a volume with columns separated by different nodes and confined to node. The configuration is such that 6 (4+2) nodes are required and one node will have one column on it, however there are only 4 nodes available. So the creation of the volumes will fail.
# vxassist -g ecdg make ecvol3 1g ncol=4 nparity=2 layout=ecoded stripe=dtag:node
# VxVM vxassist ERROR V-5-1-15315 Cannot allocate space for 2097152 block volume: Not enough HDD devices that meet specification
The following example, shows that there is more capacity then the available trays when the stripe group is a tray. In this case an erasure code volume configuration uses 4 columns and 2 parity, where tray is a stripe group. Create a volume with columns separated by different trays. The configuration is such that 6 (4+2) trays are required and one tray will have one column on it, however there are 8 (4*2) trays available. While creating volume 6 trays are required however there are 8 trays available so the creation is successful.
# vxassist -g ecdg make ecvol3 1g ncol=4 nparity=2 layout=ecoded stripe=dtag:tray
# vxprint -g ecdg ecvol3 TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 v ecvol3 fsgen ENABLED 2097152 - SYNC - - pl ecvol3-01 ecvol3 ENABLED 2097152 - ACTIVE - - sd vmr720-18vm3_vmdk0_0-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm3_vmdk0_0-01 ecvol3-01 ENABLED 524288 0 - - - sd vmr720-18vm4_vmdk0_0-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm4_vmdk0_0-01 ecvol3-01 ENABLED 524288 0 - - - sd vmr720-18vm5_vmdk0_0-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm5_vmdk0_0-01 ecvol3-01 ENABLED 524288 0 - - - sd vmr720-18vm6_vmdk0_0-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm6_vmdk0_0-01 ecvol3-01 ENABLED 524288 0 - - - sd vmr720-18vm3_vmdk0_1-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm3_vmdk0_1-01 ecvol3-01 ENABLED 524288 0 - - - sd vmr720-18vm4_vmdk0_1-02 ecvol3-01 ENABLED 2097152 0 ECLOG - - sd vmr720-18vm4_vmdk0_1-01 ecvol3-01 ENABLED 524288 0 - - - dc ecvol3_dco ecvol3 - - - - - - v ecvol3_dcl gen ENABLED 67840 - ACTIVE - - pl ecvol3_dcl-01 ecvol3_dclENABLED 67840 - ACTIVE - - sd vmr720-18vm5_vmdk0_0-03 ecvol3_dcl-01 ENABLED 67840 0 - - - pl ecvol3_dcl-02 ecvol3_dclENABLED 67840 - ACTIVE - - sd vmr720-18vm6_vmdk0_0-03 ecvol3_dcl-02 ENABLED 67840 0 - - - pl ecvol3_dcl-03 ecvol3_dcl ENABLED 67840 - ACTIVE - - sd vmr720-18vm3_vmdk0_0-03 ecvol3_dcl-03 ENABLED 67840 0 - - -