How to force backups over multiple specific networks interfaces instead of all those available.

Problem

Normal network routing on a host may cause different outbound interfaces to be used when connecting to the same host at different times. In addition, there may be times when it is desired to restrict backup traffic to a specific NIC or subset of NICs.

The REQUIRED_INTERFACE and PREFERRED_NETWORK may seem to work in some configurations but not all.

Error Message

N/A

Cause

The hostname set for REQUIRED_INTERFACE can resolve to only one IP address and it may be desired to restrict traffic to more than one of the available interfaces.

REQUIRED_INTERFACE does not affect inbound connections, only outbound connections. The outbound TCP SYN sent from the host has the source IP of the REQUIRED_INTERFACE. As a result, it will affect the interface to which packets are returned, which may affect the outbound interface used on the remote host.

REQUIRED_INTERFACE allows the application to hint to the operating system which outbound interface to use, but the O/S can ignore the hint and use a different interface.

Regardless of the outbound interface used, the source IP will be the IP resolved from the REQUIRED_INTERFACE. As a result, outbound packets on the outbound connection will leave the host via the O/S selected NIC. But the return packets for the same connection will be addressed to the REQUIRED_INTERFACE and routed through the network to that NIC. The differing paths for the outbound vs returned packets is referred to as asymmetrical routing and may fail if both routes do not exist.

A less than careful analysis of a network trace might correctly note that the backup image is on the expected network segment, but fail to notice that the TCP ACKs and smaller amount of application data flowing in the other direction are on the wrong network segment. This condition will be much more obvious during a restore operation when the [large] backup image is traversing the wrong network segment and the TCP ACKs and smaller amount of returned application data are on the expected network.

Some security models such as the Windows 2003/2008 stronghost model may drop or reject connections where the IP address appears to be spoofed because it is not on the expected network segment.

The discussion above also applies to the PREFERRED_NETWORK option, both when PROHIBITing a local interface and also when specifying a source interface.

Solution

The only way to ensure the correct interfaces are used is to configure NetBackup with appropriate hostnames for the destinations (policy client, storage unit residence, storage server, etc), then ensure those hostnames resolve to the correct IP addresses and that the TCP stack on each host and the network segments between perform appropriate network routing. It may be necessary to configure IP static host routes on the local host, specifying the remote hosts and desired outbound network.

Example Configurations:

Example 1: Server with Four NICs

If I have a server with four NICs and we need to run backup over two of them, 10.1.1.1 and 10.2.2.2

• Master Server:
  • NIC 1: 10.1.1.1
  • NIC 2: 10.2.2.1
  • NIC 3: 192.168.1.1
  • NIC 4: 172.16.1.1
• Client 1:
  • Production NIC: 192.168.1.2
  • Backup NIC: 10.1.1.2
• Client 2
  • Production NIC: 172.16.1.2
  • Backup NIC: 10.2.2.2.2

In the above example, since each client has a NIC on an IP segment shared by the master server, the natural OS routing should be able to determine the best route because the IP segments are directly attached.

Example 2: Server with Routers

• Master Server:
  • NIC 1: 10.1.1.1
  • NIC 2: 10.2.2.1
  • NIC 3: 192.168.1.1
  • NIC 4: 172.16.1.1
  • Router: 10.1.1.2
  • Router: 10.2.2.2
• Client 1:
  • Production NIC: 192.168.1.2
  • Backup NIC: 142.10.1.2
  • Backup Router: 142.10.1.1
  • Production Router: 192.168.1.1
• Client 2:
  • Production NIC: 172.16.1.2
  • Backup NIC: 131.222.1.2
  • Backup Router: 131.222.1.1
  • Production Router: 172.16.1.1

In this example, since the production NICs and backup NICs are not directly attached, you can use IP route statements to statically define the best path for traffic.

Note: The router IP addresses are defined to show what the next hop address will be.

Routes back to the server's backup NIC could be done by using this type of routing statement.

Client1$ route add -n 10.1.1.1 142.10.1.1
Client2$ route add -n 10.2.2.1 131.222.1.1

On the server:

$ route add -n 142.10.1.2 10.1.1.2
$ route add -n 131.222.1.2 10.2.2.2

To view the routing table:

$ netstat -rn

Example from Client 1:

$ netstat -rn
Routing tables
Destination Gateway Flags Refcnt Use Interface
127.0.0.1 127.0.0.1 UH 1 132 lo0
192.168.1.0 192.168.1.1 UGHD 1 514 le0
default 192.168.1.1 UG 0 0 le0
10.1.1.1 142.10.1.1 UG 1 4904 le0

For hostname resolution, you can have more granular results by relying on local host files as opposed to DNS lookups. This way, you can resolve the server hostname to whatever IP address you want the client to use for backups. Then, in the cases where the client does not have a directly connected IP segment, adding a host route as seen in the details above would provide the client with the preferred path to the server.

Note: Microsoft has published details of the weakhost and stronghost models here...

Please note that this article is not owned or maintained by Veritas and, as such, Veritas is not responsible for the content portrayed on such sites, including any revisions to or deletions of content or third-party software on which this article relies. User is responsible for conducting all necessary due diligence prior to following the instructions described in this article.

3rd Party Reference: The Cable Guy Strong and Weak Host Models (learn.microsoft.com)

See the Related Articles for details on Preferred Network and weak host backup behavior.

Applies To

Master Server has multiple NICs and needs to be able to run backups over at least one, but not all network segments.

NetBackup 3.4 - 7.5