Page History

...

1. Have we received the peer's NI information? Two ways to get it:
   1. This node pinged the peer.
   2. The peer pushed this node.
2. Has the peer received the local node's NI information. Again two ways:
   1. This node pushed to the peer.
   2. The peer pinged this node.
3. Has local NI config changed from what the peer was told. Several ways this can happen:
   1. DLC update
   2. Interface hotplug

Peer State

The state of a peer is a combination of the following bits of information, where the flags can be found in the source code by prepending LNET_PEER_, so CONFIGURED becomes LNET_PEER_CONFIGURED.

...

The PING_REQUIRED flag indicates that a Ping is necessary to properly determine the state of a peer. Triggering a Ping is the mechanism by which discovery attempts to recover from any problems it may have encountered while processing a peer. Pings triggered by this flag happen even if discovery has been disabled.

• MULTI_RAIL: This flag indicates whether a peer is running a multi-rail aware version of Lustre.

If MULTI_RAIL is set, then lp_node_seqno contains the last ping source sequence number of the node that has been received by the peer. This is used to determine whether a mult-rail aware peer needs a Push in lnet_peer_needs_push(). The ping source sequence number is a 32 bit number that is updated whenever the source buffer for LNet ping is changed.

Peer NI State

We also keep some state related to discovery in the peer_ni data structures. These state flags are prefixed by LNET_PEER_NI_ in the code.

• CONFIGURED: This flag indicates that the peer_ni was configured using DLC.
• NON_MR_PREF: This flag indicates that the peer_ni has a implicit preferred local NI.

The NON_MR_PREF flag exists to track the fact that a peer_ni for a non-multi-rail node should always see the same local NI as the source of the traffic from the node. If the preferred local NI has not been explicitly set with DLC then the code picks one and sets this flag to indicate that this is the reason for the preferred NI. That way we know to clear this preferred NI if the peer_ni turns out to be part of a multi-rail aware peer.

State Changes

1. Configuring a peer with DLC
   1. DLC adds a local NI
      1. The ping source buffer is updated and its sequence number increased
   2. DLC deletes a local NI
      1. The ping source buffer is updated and its sequence number increased
   3. DLC creates a peer
   4. DLC adds a peer NI to a peer
   5. DLC deletes a peer from a peer
   6. DLC deletes a peer

The following discussion must be updated – OW

...