Veritas InfoScale™ 7.3.1 Troubleshooting Guide - Solaris

Associating CVM commands issued from slave to master node

When you run commands on the CVM slave node that change the shared disk group configuration, CVM ships the commands to the CVM master node for execution.

For example, on the slave node, you run the following command, to create a volume in a shared disk group. CVM ships the command to the master node, and CVM executes the command on the master node.

# vxassist -g shareddg make shared-vol1 200M

On the CVM slave node, enter the following command to identify the shipped command from the transaction log (translog):

# egrep CMDSHIP_REQUEST translog

In this example, the following entry was recorded in the transaction log on slave node:

Thu Jul 15 06:30:16 2010
Clid = 5302, PID =  589906, Part = 0, Status = 0, Abort Reason = 0
        DG_SET_CURRENT_ID  shareddg
        DG_SET_CURRENT  shareddg
        DG_GETCFG_ID  0xdde49f shareddg
        DG_GETCFG_NAME  0xdde49f shareddg
        DG_SET_CURRENT_ID  shareddg
        DG_SET_CURRENT  shareddg
        DG_SET_CURRENT_ID  shareddg
        DG_SET_CURRENT  shareddg
        DG_GETCFG_ALL  0x420
        DG_GETCFG_ALL  0x420
        VOL_TRANS   ds4700-0_7 ds4700-0_3 ds4700-0_
        DG_GET_DEFAULT  <no request data>
        CMDSHIP_REQUEST  Command Shipped = /usr/sbin/vxassist -g 

shareddg make shared-vol1 200M
	Default dg = nodg
        DROPPED  <no request data>

To locate the utility that issued this request on the slave node, use this syntax:

# egrep -n PID cmdlog | egrep Clid

In this example, enter the following command:

# egrep -n 589906 cmdlog | egrep 5302
7310#: 5302, 589906, Thu Jul 15 06:30:14 2010 /usr/sbin/vxassist -g 

The output from the example shows a match at line 7310 in the command log. Examining lines 7310 and 7311 in the command log indicates that the vxassist make command was run on the shareddg disk group:

# sed -e '7310,7311!d' cmdlog

# 5302, 589906, Thu Jul 15 06:30:14 2010
/usr/sbin/vxassist -g shareddg make shared-vol1 200M

If the command uses disk access (DA) names, the shipped command converts the DA names to unique disk IDs (UDID) or Disk media (DM) names. On the CVM master node, the vxconfigd log shows the entry for the received command. To determine the commands received from slave nodes on the master, enter the command:

# egrep CMDSHIP_REQUEST /var/adm/messages

In this example, the following received command is recorded in the vxconfigd log on the master node:

07/15 06:29:02: V-5-1-0 receive_cmdship_message: 
CMDSHIP_REQUEST: Received command: 
Text - /usr/sbin/vxassist -g shareddg make shared-vol1 200M len = 53 
CLID = 5302 SlaveID = 0 Defaultdg = nodg 

From the above output on the master node, you can determine the slave from which the command is triggered based on the SlaveID. The SlaveID is the cluster monitor nodeid (CM nid) of the node in the cluster.

To determine the slave node from the command is triggered, enter the following command and find the slave node with the matching SlaveID:

# /etc/vx/bin/vxclustadm nidmap

For example:

bash-3.00# # /etc/vx/bin/vxclustadm nidmap

Name                   CVM Nid    CM Nid     State
pl9dom1                1          3          Joined: Master
pl9dom2                0          1          Joined: Slave
pl9dom3                2          2          Joined: Slave
pl9dom4                3          0          Joined: Slave
bash-3.00#

The CVM master node executes the command and sends the response to the slave node.

To find the response that the master node sent to the slave node, enter a command such as the following on the master node:

# egrep CMDSHIP_RESPONSE translog | egrep SlaveCLID

In this example, enter the following command to find the response that the master node sent:

# egrep CMDSHIP_RESPONSE translog | egrep 5302
Thu Jul 15 06:29:03 2010
Clid = 27741, PID = 475212, Part = 0, Status = 0, Abort Reason = 0
 CMDSHIP_RESPONSE  SlaveCLID = 5302 SlaveCMID = 0
 ExitCode = 12 Flags = 1 stdoutlen = 0 stderrlen = 98  Response = 

VxVM vxassist ERROR V-5-1-10127 creating volume shared-vol1:        

Record already exists in disk group        
DROPPED  <no request data>