You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next » 

257 »·······if (tunables->lnd_map_on_demand == 1) {
258 »·······»·······/* don't make sense to create map if only one fragment */
259 »·······»·······tunables->lnd_map_on_demand = 2;
260 »·······}


/* lnd_concurrent_sends has no explicit default and since it's declared as static, I think it defaults to 0 */
 
#define LNET_MAX_IOV»···256
#define IBLND_MAX_RDMA_FRAGS LNET_MAX_IOV
 
/* if map_on_demand is specified and it's in the range of 1 to 32, then concurrent sends is double the lct_peer_tx_credits */
/* lct_peer_tx_credits is set either by the user or it defaults to 8. OPA it's set to 128 
   lct_peer_tx_credits can't be greater than the lct_max_tx_credits. Also set by the user and defaults to 256 (does it make sense to be more?*/
262 »·······if (tunables->lnd_concurrent_sends == 0) {
263 »·······»·······if (tunables->lnd_map_on_demand > 0 &&
264 »·······»·······    tunables->lnd_map_on_demand <= IBLND_MAX_RDMA_FRAGS / 8) {
265 »·······»·······»·······tunables->lnd_concurrent_sends =
266 »·······»·······»·······»·······»·······net_tunables->lct_peer_tx_credits * 2;
267 »·······»·······} else {
268 »·······»·······»·······tunables->lnd_concurrent_sends =
269 »·······»·······»·······»·······net_tunables->lct_peer_tx_credits;
270 »·······»·······}
271 »·······}
 
kiblnd_rdma_frags() assumes that the value of map_on_demand is the number of rdma_frags that can be transferred between peers. This is used during the negotiation process.
 
When creating a connection peer_ni->ibp_max_frags is set to kiblnd_cfg_rdma_frags(). This is a command function used by kiblnd_rdma_frags(). Basically, what's happening here is that a peer is created and the maximum number of fragments it can support is set to map_on_demand if set or IBLND_MAX_RDMA_FRAGS (256) if not set. During connection establishment the ibp_max_frags value is propagated in the connection parameters. in kiblnd_passive_connect() the value carried in the connection parameter is compared to the local value number of supported fragments. If the connection param value is greater than what can be supported locally, then reject the connection and provide the local number of fragments supported. The peer will retry if it has map_on_demand configured. It will attempt reconnection with the same num_of_frags given in the rejection message. 
 
If the connection param value is less than the local value, and fmr is not enabled, then peer can't negotiate. This will result in connection rejection, since the rejection message will get to the peer 
 
TODO
  • No labels