Page History

...

Network Interface Management

...

A connection is created per socket. This connection is added to the ksock_peer_ni list of connections.

The following TCP port range is defined for the outgoing connections (active): LNET_ACCEPTOR_MIN_RESERVED_PORT (512) to LNET_ACCEPTOR_MAX_RESERVED_PORT (1023). The requests are sent to the single predefined acceptor port on the other (passive) side. 

A hello message is sent by the active side of the connection. This hello message contains the list of IP addresses stored in the ksnd_data.ksock_interfaces to which we don't have routes yet. When the passive side receives the hello message it sends its own hello as a response. The active side will receive that hello which contains the list of the remote's peer IP addresses. It will then create additional routes to these interfaces, which we would create on demand when sending messages.

Connection creation management is unduly complex due to TCP bonding. In fact the purpose of the hello message appears to be primarily for passing around the IP addresses of the peer.

...

• Remove the storage of multiple IP addresses in the ksock_net 
• Remove all associated managment code of the multiple IP addresses
• Remove all the route constructs and the code which uses the route constructs
• Connections should be associated directly with the peer
• Hello message can be kept for backwards compatibility, however, they will always include only one IP address

Redesign Overview: re-purposing route construct

While removing multiple interface per ksock_net is straightforward, getting rid of route constructs is much more invasive to current socklnd design. 

In the current design, there are multiple route structures associated with each peer_ni. The association is not unique because routes may be shared, so separate reference counts are maintained. As mentioned above, deprecating tcp bonding means that only single route is needed per peer_ni. It is proposed to re-purpose the existing route construct as follows:

• Rename the route construct to "ksock_peer_conn_cb" or similar name. 
• Peer_ni shall reference single ksock_peer_conn_cb
• ksock_peer_conn_cb structure shall be the same as existing route, except there shall be no need for individual refcount
• It shall be ensured that ksock_peer_conn_cb is created and deleted at the same time as the owning peer_ni construct
• It shall be ensured that  ksock_peer_conn_cb won't be deleted before all of the connections it owns are deleted

This proposal simplifies both "the route structure removal" and "adding structure to encapsulate ksock_socket_conn" tasks by combining them.

Multiple Connections Per Peer

...