LNet Block Level Diagram
The diagram above represents the different functional blocks in LNet. A quick overview will help in understanding the code - When LNet starts up it reads various module parameters and configures itself based on these values.
- Further configuration can be added dynamically via
lnetctl utility. - The main APIs to request LNet to send messages are
LNetPut() and LNetGet() .
- When a message is sent a peer block is created to track messages to and from that peer.
- When a message is received a peer block is created to track messages to and from that peer.
- When sending messages LNet has to select the local and remote interfaces (IE the path the message will traverse to reach its destination). It does so through the selection algorithm.
- In that process it selects the local network interfaces and remote network interfaces for the destination peer.
- Each peer has its own set of credits used to rate limit messages to it. LNet checks and manages these credits before sending the message.
- When a message is sent a credit is consumed.
- When a message is received a credit is returned.
- If the destination peer is not on the same network as the node, then lookup a route to the final destination. If no route is present then the message can not be sent.
- If a node is acting as a router, then it can receive messages to which it is not the final destination. It then can forward these messages to the final destination.
- When a received message is to be forwarded then a router buffer is used to receive the message data. Router buffers have their own credits.
- A fault injection module can be activated for testing. That module will simulate message send/receive failures.
| 