| ID | Class | Version | Status | Description |
|---|
| REQUIRED | 1.0 | ACCEPTED | LNet shall maintain a health value per local NI |
| REQUIRED | 1.0 | ACCEPTED | LNet shall maintain a health value per peer NI |
| REQUIRED | 1.0 | ACCEPTED | The health value is a positive number between 0 - 1000. This range is chosen to allow enough granularity for decrementing and incrementing the health value. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall decrement the health value of an NI by the configured health sensitivity value whenever there is a error sending a message over or to the NI. The health value shall not be less than 0. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall increment the health value but not beyond a 1000, which represents a healthy NI.
|
| REQUIRED | 1.0 | ACCEPTED | LNet shall determine the NI to select based on the following ordered criteria: - NI health
- NUMA closeness
- NI available credits
- Round Robin
|
| REQUIRED | 1.0 | ACCEPTED | When LNet fails to send on a local NI or to a remote NI, it shall place that NI on a recovery queue. The NI shall be pinged or used for a ping periodically to determine if it has recovered. 1000 minus current health value pings must pass sequentially in order for an interface to be considered fully healthy. EX: if the NI's health value is 900, then 100 pings using that NI must be successful in order for the NI to be considered fully healthy. Each successful send will increment the NI's health value by 1. |
| REQUIRED | 1.0 | ACCEPTED | On a local, remote and network timeout LNet shall reselect a pair of local and peer NI to resend the message. - Another option is to be more granular when selecting the interfaces depending on the timeout that occurred. If local timeout then only re-select a different local NI. If remote timeout then re-select a peer NI and for a network timeout then select a new pair of local and peer NIs.
- This has the disadvantage of a more complicated implementation.
- There isn't a clear advantage of making the selection more granular.
- Since the health value of the NI in question is decremented based on the error encountered, then the selection algorithm will favor the NI with poor health less.
|
| REQUIRED | 1.0 | ACCEPTED | On any type of timeout, if the peer is non-MR capable, LNet shall retransmit the message from the same local NI. |
| DESIRED | 1.0 | ACCEPTED | For the routers LNet shall re-transmit a message over any of its local NIs. Routers are a special case since non-MR peers expect the same source NID of the final destination, but doesn't care about the router NIDs. The router NIDs are not passed up to ptlrpc or other ULPs. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall not attempt to resend a message on the following failure types: - Shutdown in progress
- Out of memory
- Discovery errors out with one of the errors on this list.
- An MD bind failure
-EINVAL-HOSTUNREACH
Invalid information given - Internal failure
The assumption is that any resend will encounter the same failure again. Let the upper layers deal with the failure. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall re-transmit messages no more than the retry count specified by the user. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall stop re-transmitting when one of the following criterion is satisfied - Message is sent successfully
- Retry count is reached
- Transaction timeout expires.
|
| REQUIRED | 1.0 | ACCEPTED | LNet shall default the transaction timeout to 5 seconds |
| REQUIRED | 1.0 | ACCEPTED | LNet shall timeout a message and send a failure event to the ULP if a message is not re-transmitted successfully |
| REQUIRED | 1.0 | ACCEPTED | LNet shall calculate the message timeout based on the ULP provided timeout if one is provided or the configured transaction timeout otherwise. message timeout = transaction timeout / number of retries. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall pass the message timeout to the LND and will rely on the LND to enforce the timeout. If the LND times out the message then it will notify the LNet layer which will attempt to re-transmit the message. |
| REQUIRED | 1.0 | ACCEPTED | LNet shall not attempt to re-transmit if the retry count is set to 0 |
| REQUIRED | 1.0 | ACCEPTED | LNet shall monitor the ACK/REPLY for a PUT/GET. It will send a timeout event for a PUT or a GET if the respective ACK/REPLY is not received within the transaction timeout/ |
| REQUIRED | 1.0 | ACCEPTED | LNet shall allow the callers of LNetGet() or LNetPut() to specify a different transaction timeout other than the one configured in the system. - EX: lnetctl ping can specify a shorter timeout than ptlrpc
|
| REQUIRED | 1.0 | ACCEPTED | LNet shall activate the transaction timeout only after a PUT which requires an ACK or a GET which requires a REPLY is successfully passed to the LND (IE lnd_send() returns successfully) For PUT which requires no ACK no timeout will be activated. |
| DESIRED | 1.0 | IN-PROGRESS | LNet shall use UDEV events to propagate errors detected on a local or peer NI. |
| DESIRED | 1.0 | IN-PROGRESS | LNet shall handle flapping of interfaces and will favor the interface less. |
