You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

Overview

LNet is a virtual networking layer which allows Lustre nodes to communicate with each other.

System Diagram

LNetSystemDiagram

  • lnetctl: User space utility used to configure and query LNet kernel module
  • DLC Library: User space library which communicates with LNet kernel module primarily via IOCTL
  • LNet IOCTL: Module which handles the IOCTLs and calls appropriate callbacks in the LNet kernel module
  • PTLRPC: Kernel module which implements an RPC protocol. It's the primary user of LNet.
  • LNet: Kernel module which implements the Lustre Networking communication protocol
  • o2iblnd: Verbs driver. It executes RDMA operations via verbs
  • socklnd: TCP/IP driver. It sends/receives TCP messages
  • gnilnd: Cray/HPE driver not maintained by us

LNet Block Level Diagram

LNetBlockDiagram

The diagram above represents the different functional blocks in LNet. A quick overview will help in understanding the code

  • When LNet starts up it reads various module parameters and configures itself based on these values
  • Further configuration can be added dynamically via lnetctl utility
  • The main

Useful Documentation


LNet Source Directory
  • lustre-release/lnet 
    • Top LNet director
  • include 
    • lnet
      • Internal includes
    • uapi 
      • include used by user space and other kernel modules
  • klnds 
    • gnilnd 
      • Cray LNet Driver (LND). Developed and tested by Cray/HPE
    • o2iblnd 
      • IB LND used by mellanox and Intel OmniPath. It uses the Verbs API
      • Only uses IBoIP for connection establishment
    • socklnd
      • Socket LND used for ethernet interfaces. It uses TCP/IP
  • lnet 
    • LNet kernel source directory
  • selftest 
    • LNet selftest tool. Generated RDMA traffic. Runs in knernel
  • utils 
    • User space tools including lnetctl and liblnetconfig.
    • lnetctl is a CLI used to configure lnet
    • liblnetconfig is the library used by lnetctl to communicate with the lnet kernel module

Presentations


Tasks


Medium
  • LNet Router Testing
    • We need to expand our testing of LNet. The link above lists a set of routing tests. We need to write LUTF scripts for them
      • Benefits:
        • Learn how to configure LNet routers
        • Learn how to use the LUTF
        • Learn how to test LNet
        • Learn the code
  • LU-12041
  • No labels