InfoScale™ 9.0 Storage Foundation Cluster File System High Availability Administrator's Guide - Linux
- Section I. Introducing Storage Foundation Cluster File System High Availability
- Overview of Storage Foundation Cluster File System High Availability
- About Veritas File System
- About Veritas Replicator
- How Dynamic Multi-Pathing works
- How Volume Manager works
- How Volume Manager works with the operating system
- How Volume Manager handles storage management
- Volume layouts in Veritas Volume Manager
- Online relayout
- Volume resynchronization
- Dirty region logging
- Volume snapshots
- FastResync
- How VxVM handles hardware clones or snapshots
- Volume encryption
- How Veritas File System works
- How Storage Foundation Cluster File System High Availability works
- About Storage Foundation Cluster File System High Availability architecture
- About Veritas File System features supported in cluster file systems
- About single network link and reliability
- About I/O fencing
- About preventing data corruption with I/O fencing
- About I/O fencing components
- About server-based I/O fencing
- About secure communication between the SFCFSHA cluster and CP server
- How Cluster Volume Manager works
- Overview of clustering
- Cluster Volume Manager (CVM) tolerance to storage connectivity failures
- Storage disconnectivity and CVM disk detach policies
- CVM initialization and configuration
- Dirty region logging in cluster environments
- Multiple host failover configurations
- About Flexible Storage Sharing
- Application isolation in CVM environments with disk group sub-clustering
- Overview of Storage Foundation Cluster File System High Availability
- Section II. Provisioning storage
- Provisioning new storage
- Advanced allocation methods for configuring storage
- Customizing allocation behavior
- Using rules to make volume allocation more efficient
- Understanding persistent attributes
- Customizing disk classes for allocation
- Specifying allocation constraints for vxassist operations with the use clause and the require clause
- Creating volumes of a specific layout
- Customizing allocation behavior
- Creating and mounting VxFS file systems
- Creating a VxFS file system
- Mounting a VxFS file system
- tmplog mount option
- ioerror mount option
- largefiles and nolargefiles mount options
- Resizing a file system
- Monitoring free space
- Extent attributes
- Section III. Administering multi-pathing with DMP
- Administering Dynamic Multi-Pathing
- Discovering and configuring newly added disk devices
- About discovering disks and dynamically adding disk arrays
- How to administer the Device Discovery Layer
- Administering DMP using the vxdmpadm utility
- Gathering and displaying I/O statistics
- Specifying the I/O policy
- Discovering and configuring newly added disk devices
- Dynamic Reconfiguration of devices
- Reconfiguring a LUN online that is under DMP control using the Dynamic Reconfiguration tool
- Manually reconfiguring a LUN online that is under DMP control
- Managing devices
- Displaying disk information
- Changing the disk device naming scheme
- Adding and removing disks
- Event monitoring
- Administering Dynamic Multi-Pathing
- Section IV. Administering Storage Foundation Cluster File System High Availability
- Administering Storage Foundation Cluster File System High Availability and its components
- Administering CFS
- About the mount, fsclustadm, and fsadm commands
- When the CFS primary node fails
- About Snapshots on SFCFSHA
- Administering VCS
- Administering CVM
- About setting cluster node preferences for master failover
- About changing the CVM master manually
- Importing disk groups as shared
- Administering Flexible Storage Sharing
- Administering ODM
- About administering I/O fencing
- About the vxfentsthdw utility
- Testing the coordinator disk group using the -c option of vxfentsthdw
- About the vxfenadm utility
- About the vxfenclearpre utility
- About the vxfenswap utility
- About administering the coordination point server
- About migrating between disk-based and server-based fencing configurations
- Migrating between fencing configurations using response files
- About the vxfentsthdw utility
- Administering SFCFSHA global clusters
- Enabling S3 server
- Using Clustered NFS
- Understanding how Clustered NFS works
- Configure and unconfigure Clustered NFS
- Administering Clustered NFS
- Samples for configuring a Clustered NFS
- Using Common Internet File System
- Deploying Oracle with Clustered NFS
- Administering sites and remote mirrors
- About sites and remote mirrors
- Fire drill - testing the configuration
- Changing the site name
- Administering the Remote Mirror configuration
- Failure and recovery scenarios
- Administering iSCSI with SFCFSHA
- Administering datastores with SFCFSHA
- Administering Storage Foundation Cluster File System High Availability and its components
- Section V. Optimizing I/O performance
- Veritas File System I/O
- Veritas Volume Manager I/O
- Managing application I/O workloads using maximum IOPS settings
- Section VI. Veritas Extension for Oracle Disk Manager
- Using Veritas Extension for Oracle Disk Manager
- About Oracle Disk Manager
- About Oracle Disk Manager and Oracle Managed Files
- Using Cached ODM
- Using Veritas Extension for Oracle Disk Manager
- Section VII. Using Point-in-time copies
- Understanding point-in-time copy methods
- When to use point-in-time copies
- About Storage Foundation point-in-time copy technologies
- Volume-level snapshots
- Storage Checkpoints
- About FileSnaps
- About snapshot file systems
- Administering volume snapshots
- Traditional third-mirror break-off snapshots
- Full-sized instant snapshots
- Creating instant snapshots
- Adding an instant snap DCO and DCO volume
- Controlling instant snapshot synchronization
- Creating instant snapshots
- Cascaded snapshots
- Adding a version 0 DCO and DCO volume
- Administering Storage Checkpoints
- Storage Checkpoint administration
- Administering FileSnaps
- Administering snapshot file systems
- Understanding point-in-time copy methods
- Section VIII. Optimizing storage with Storage Foundation Cluster File System High Availability
- Understanding storage optimization solutions in Storage Foundation Cluster File System High Availability
- About SmartMove
- Migrating data from thick storage to thin storage
- Maintaining Thin Storage with Thin Reclamation
- Reclamation of storage on thin reclamation arrays
- Identifying thin and thin reclamation LUNs
- InfoScale 4K sector device support solution
- Understanding storage optimization solutions in Storage Foundation Cluster File System High Availability
- Section IX. Maximizing storage utilization
- Understanding storage tiering with SmartTier
- Creating and administering volume sets
- Multi-volume file systems
- Features implemented using multi-volume file system (MVFS) support
- Adding a volume to and removing a volume from a multi-volume file system
- Volume encapsulation
- Load balancing
- Administering SmartTier
- About SmartTier
- Placement classes
- Administering placement policies
- File placement policy rules
- Multiple criteria in file placement policy rule statements
- Using SmartTier with solid state disks
- Sub-file relocation
- Administering hot-relocation
- How hot-relocation works
- Moving relocated subdisks
- Compressing files
- About compressing files
- Use cases for compressing files
- Section X. Administering and protecting storage
- Managing volumes and disk groups
- Rules for determining the default disk group
- Moving volumes or disks
- Monitoring and controlling tasks
- Performing online relayout
- Adding a mirror to a volume
- Encrypting existing volumes
- Managing disk groups
- Disk group versions
- Displaying disk group information
- Creating a disk group
- Importing a disk group
- Moving disk groups between systems
- Importing a disk group containing hardware cloned disks
- Handling conflicting configuration copies
- Destroying a disk group
- Backing up and restoring disk group configuration data
- Managing plexes and subdisks
- Erasure coding in Veritas InfoScale storage environments
- Erasure coding deployment scenarios
- Customized failure domain
- Decommissioning storage
- Rootability
- Encapsulating a disk
- Rootability
- Sample supported root disk layouts for encapsulation
- Encapsulating and mirroring the root disk
- Administering an encapsulated boot disk
- Quotas
- Using Veritas File System quotas
- File Change Log
- Support for protection against ransomware
- Non-modifiable storage checkpoints
- Soft WORM storage
- Secure file system
- Secure file system for Oracle Single Instance
- Secure file system for PostgreSQL database
- Managing volumes and disk groups
- Section XI. Reference
- Appendix A. Reverse path name lookup
- Appendix B. Tunable parameters
- Tuning the VxFS file system
- Methods to change Dynamic Multi-Pathing tunable parameters
- Tunable parameters for VxVM
- Methods to change Veritas Volume Manager tunable parameters
- About LLT tunable parameters
- About GAB tunable parameters
- About VXFEN tunable parameters
- Appendix C. Command reference
- Appendix D. Creating a starter database
- Appendix E. Executive Order logging
Secure file system for Oracle Single Instance
SecureFS Oracle integration provides the ability to take application aware file system checkpoints for Oracle Single Instance for ransomware protection. The feature creates consistent online checkpoints for all the file systems used by Oracle to store data, redo logs, and archive logs. This makes application consistent recovery in case of ransomware scenario possible with minimum RTO.
Note:
Before configuring SecureFS, ensure that the Oracle instance is in running state.
Prerequisites
SecureFS for Oracle requires Oracle database to be running in ARCHIVELOG mode. Refer to the Oracle Database documentation for steps to enable ARCHIVELOG mode.
Ensure that all the directories used by Oracle for storing data and log files reside on a VxFS file system. This allows SecureFS to checkpoint all these file systems in application consistent manner.
To configure secure file system for Oracle Single Instance using CLI
- Run the command to ensure that vxfstaskd is in an active state.
# systemctl status vxfstaskd
- Enable and start the dbed service.
# systemctl enable vxdbdctrl.service # systemctl start vxdbdctrl.service
- If the Oracle application is managed through a InfoScale Cluster Server resource, create the new Process agent resource. Refer to the Cluster Server Administrator's Guide for details. If the Oracle application is not managed through the InfoScale Cluster Server (VCS) resource, make all the file systems used by the application primary on the node where the Oracle instance is running.
# /opt/VRTS/bin/fsclustadm setprimary <mount point>
Sample command:
# /opt/VRTS/bin/fsclustadm setprimary /dbmnt
- Create a secureFS config file on the data file mount point.
# /opt/VRTS/bin/vxschadm create <mount point>
Sample command:
# /opt/VRTS/bin/vxschadm create /dbmnt
- Enable Compliance (WORM) or Enterprise (SoftWORM) on data file mount point.
# /opt/VRTS/bin/fsadm -o <worm/softworm> <mount point>
Sample command:
# /opt/VRTS/bin/fsadm -o worm /dbmnt
- Configure DBED on the node where the Oracle instance is running.
# /opt/VRTS/bin/dbed_update -S <Oracle SID> -H <Oracle home>
Sample command:
# /opt/VRTS/bin/dbed_update -S testdb -H /u01/oracle/dbhome
- Add Oracle application in secureFS config file on data file mount point.
# /opt/VRTS/bin/vxschadm add application oracle <ora_user> <ora_home> <ora_sid> <mntpt>
Sample command:
# /opt/VRTS/bin/vxschadm add application oracle orauser /u01/oracle/dbhome testdb /dbmnt
When SecureFS is configured on the data file system, it identifies all the other file systems used by Oracle and applies the same checkpoint schedule for all such file systems. Manual intervention is not required.
Note:
WORM mode and audit log feature enabled on data file mount point, are applied to redo and archival log mount points.
- Enable the checkpoint feature on the data file mount point.
# /opt/VRTS/bin/vxschadm add checkpoint <interval in os cron format> <maximum number of checkpoints> <retention period of checkpoint> <time mode> <removable> <mount_point>
Time interval
Format : "mm hh dd MM ww" mm : minutes (0-59) or * hh : hours (0-23) or * dd : days (1-31) or * MM: months (1-12) or * ww: week (0-6) SUNDAY=0, SATURADAY=6
*: Refers 'per', for example per minute, per hour, and so on.
Time mode can be:
m : minutes h : hours d : days M : months y : years
Sample command and output:
# /opt/VRTS/bin/vxschadm add checkpoint "59 15 * * *" 1 h 0 1 /mnt1
Following are the sample output of the snapshot format:
Example 1: "30 15 * * *" Output: Every day at 1530 hours, the task is run.
Example 2: "30 15 * * 6" :Output: Every Saturday at 1530 hours, the task is run.
Caution: "* * * * *"
Every minute and hour options are not supported.
- If the Oracle Single Instance is under VCS control, you must mark the file system where the Oracle database (DB) resource is online as primary. You can mark the file system as primary by setting the Oracle resource
SetFSPrimary
attribute value to 1.The
SetFSPrimary
attribute for Oracle resource in VCS configures the file system as primary on the node where the Oracle DB resource is online. This attribute plays an important role when the Oracle database is set up as a VCS resource in the Oracle service group that includes both CFSmount and CVMVoldg resources.When the Oracle resource is online on a node, the InfoScale cluster file system (CFS) should be primary on that same node to ensure that the SecureFS feature functions correctly. To achieve this, set
SetFSPrimary
to 1 where the Oracle resource is present. WithSetFSPrimary
set to 1, the Oracle agent resource ensures that the underlying CFS mount is primary on that node.Note the following when you use this attribute in SecureFS configuration:
This attribute takes effect only when Cluster Server (VCS) manages the startup of the application.
If an application mount point is added after the VCS resource is online, then to mark the mount point as primary on the file system, you must restart the VCS application service group.
Use one of the following methods depending on the product version and the SecureFS primary configuration.
For a fresh InfoScale installation (not an upgrade case), run the following commands to enable the
SetFSPrimary
attribute:# /opt/VRTSvcs/bin/haconf -makerw # /opt/VRTSvcs/bin/hares -modify Oracleresourcename SetFSPrimary 1 # /opt/VRTSvcs/bin/haconf -dump -makero
If you have upgraded from InfoScale version 8.0.2 Update 2 to the current version of InfoScale, you must first remove the existing independent Process agent resource that is added in the Oracle service group. In InfoScale 8.0.2, this Process agent resource is used to mark the file system as primary. However, beginning with InfoScale 9.0 release, a new attribute named
SetFSPrimary
is introduced to provide the same functionality. Therefore, to set the file system as primary, you must first remove the Process agent resource and then enable the attribute.Run the following commands:
# /opt/VRTSvcs/bin/haconf -makerw
Take the existing Process agent resource in the Oracle service group offline gracefully:
# /opt/VRTSvcs/bin/hares -offline Processagentresourcename
Confirm that the Process agent resource is offline and then delete that resource from the Oracle service group:
# /opt/VRTSvcs/bin/hares -delete Processagentresourcename
Enable the
SetFSPrimary
attribute by setting its value to 1:# /opt/VRTSvcs/bin/hares -modify Oracleresourcename SetFSPrimary 1
Note:
The
SetFSPrimary
attribute value changes take effect only in the next online of the Oracle resource.Save the VCS configuration changes:
# /opt/VRTSvcs/bin/haconf -dump -makero
If Oracle Single Instance is not under VCS control, ensure FS primary of data file mount point is the system where Oracle Single Instance is up and running.
Interval of less than one hour is not supported. The checkpoints that get created are non-modifiable checkpoints which can only be mounted in read-only mode. If a new datafile or log directory is added to Oracle application which is on a new file system, such file system automatically becomes part of the SecureFS configuration on next checkpoint schedule. Manual intervention is not required.
All SecureFS configuration file operations that are done using # /opt/VRTS/bin/vxschadm must be on the data file mount point.
Note:
Operations performed on redo and archive file system can lead to undefined behavior. Previously taken checkpoints and modes that are WORM and aulog, remain as is, even after you delete the checkpoint feature.