...
Depending on the network topology which the Lustre network is built on, it might be necessary to assign priorities to specific interfaces which are connected to optimized paths. In this way messages don't take more hops than necessary to get to the destination. As an example, in a dragonfly topology as diagrammed below, a node can have multiple interfaces on the same network, but some interfaces are not optimized to go directly to the destination group. So if the selection algorithm is operating without any rules, it could select a local interface which is less than optimal.
The clouds in the diagram below represents a group of LNet nodes on the o2ib network. The admin should know which node interfaces resolve to a direct path to the destination group. Therefore, giving priority for a local NID within a network is a way to ensure that messages always prefer the optimized paths.
| Gliffy Diagram | ||||||
|---|---|---|---|---|---|---|
|
The diagram above was inspired by: https://www.ece.tufts.edu/~karen/classes/final_presentation/Dragonfly_Topology_Long.pptx
Refer to the above power point for further discussion on the dragon-fly topology.
Preferred local/remote NID pairs
...
| Code Block |
|---|
/* * lustre_lnet_add_net_sel_pol * Add a net selection policy. If there already exists a * policy for this net it will be updated. * net - Network for the selection policy * priority - priority of the rule */ int lustre_lnet_add_net_sel_pol(char *net, int priority); /* * lustre_lnet_del_net_sel_pol * Delete a net selection policy. * net - Network for the selection policy * id - [OPTIONAL] ID of the policy. This can be retrieved via a show command. */ int lustre_lnet_del_net_sel_pol(char *net, int id); /* * lustre_lnet_show_net_sel_pol * Show configured net selection policies. * net - filter on the net provided. */ int lustre_lnet_show_net_sel_pol(char *net); /* * lustre_lnet_add_nid_sel_pol * Add a nid selection policy. If there already exists a * policy for this nid it will be updated. NIDs can be either * local NIDs or remote NIDs. * nid - NID for the selection policy * priority - priority of the rule */ int lustre_lnet_add_nid_sel_pol(char *nid, int priority); /* * lustre_lnet_del_nid_sel_pol * Delete a nid selection policy. * nid - NID for the selection policy * id - [OPTIONAL] ID of the policy. This can be retrieved via a show command. */ int lustre_lnet_del_nid_sel_pol(char *nid, int id); /* * lustre_lnet_show_nid_sel_pol * Show configured nid selection policies. * nid - filter on the NID provided. */ int lustre_lnet_show_nid_sel_pol(char *nid); /* * lustre_lnet_add_nid_sel_pol * Add a peer to peer selection policy. If there already exists a * policy for the pair it will be updated. * src_nid - source NID * dst_nid - destination NID * priority - priority of the rule */ int lustre_lnet_add_peer_sel_pol(char *src_nid, char *dst_nid, int priority); /* * lustre_lnet_del_peer_sel_pol * Delete a peer to peer selection policy. * src_nid - source NID * dst_nid - destination NID * id - [OPTIONAL] ID of the policy. This can be retrieved via a show command. */ int lustre_lnet_del_peer_sel_pol(char *src_nid, char *dst_nid, int id); /* * lustre_lnet_show_peer_sel_pol * Show peer to peer selection policies. * src_nid - [OPTIONAL] source NID. If provided the output will be filtered * on this value. * dst_nid - [OPTIONAL] destination NID. If provided the output will be filtered * on this value. */ int lustre_lnet_show_peer_sel_pol(char *src_nid, char *dst_nid); |
...
Data structures
IOCTL
Serialization/Deserialization
...