You are viewing an old version of this page. View the current version.
Compare with Current
View Page History
« Previous
Version 2
Next »
Overview
LNet is a virtual networking layer which allows Lustre nodes to communicate with each other.
System Diagram
- lnetctl: User space utility used to configure and query LNet kernel module
- DLC Library: User space library which communicates with LNet kernel module primarily via IOCTL
- LNet IOCTL: Module which handles the IOCTLs and calls appropriate callbacks in the LNet kernel module
- PTLRPC: Kernel module which implements an RPC protocol. It's the primary user of LNet.
- LNet: Kernel module which implements the Lustre Networking communication protocol
- o2iblnd: Verbs driver. It executes RDMA operations via verbs
- socklnd: TCP/IP driver. It sends/receives TCP messages
- gnilnd: Cray/HPE driver not maintained by us
LNet Block Level Diagram
The diagram above represents the different functional blocks in LNet. A quick overview will help in understanding the code
- When LNet starts up it reads various module parameters and configures itself based on these values
- Further configuration can be added dynamically via lnetctl utility
- The main
Useful Documentation
LNet Source Directory
lustre-release/lnet
include
lnet
uapi
- include used by user space and other kernel modules
klnds
gnilnd
- Cray LNet Driver (LND). Developed and tested by Cray/HPE
o2iblnd
- IB LND used by mellanox and Intel OmniPath. It uses the Verbs API
- Only uses IBoIP for connection establishment
- socklnd
- Socket LND used for ethernet interfaces. It uses TCP/IP
lnet
- LNet kernel source directory
selftest
- LNet selftest tool. Generated RDMA traffic. Runs in knernel
utils
- User space tools including lnetctl and liblnetconfig.
- lnetctl is a CLI used to configure lnet
- liblnetconfig is the library used by lnetctl to communicate with the lnet kernel module
General Tips and Tricks
Title | Creator | Modified |
---|
Useful Links | Serguei Smirnov | Jun 03, 2024 |
Frequently Asked Questions | Amir Shehata | Jun 03, 2024 |
MLX Info and Tips | Amir Shehata | Aug 09, 2023 |
Crash course on Crash | Amir Shehata | Aug 03, 2023 |
MR Cluster Setup | Amir Shehata | May 19, 2023 |
Adhoc Lustre Tips | Amir Shehata | Jan 04, 2023 |
GIT tips | Amir Shehata | Jan 04, 2023 |
Loading hfi1.conf parameters on boot | Amir Shehata | Nov 23, 2022 |
Useful Lustre commands | Amir Shehata | Aug 31, 2022 |
Kernel Debugging Misc | Amir Shehata | Jun 10, 2022 |
Installing MOFED | Amir Shehata | Oct 05, 2021 |
Installing debug symbols on Ubuntu | Amir Shehata | Jun 18, 2021 |
Virsh cheat sheet | Amir Shehata | May 20, 2021 |
Issues to look out for | Amir Shehata | May 04, 2021 |
Kernel GDB live Debugging with KVM | Amir Shehata | Apr 24, 2021 |
Lustre QoS | Amir Shehata | Apr 22, 2021 |
Setting up a Failover Pair with virsh/virt-manager | Amir Shehata | Jul 23, 2020 |
self-test template script | Amir Shehata | Jul 12, 2020 |
Building Lustre | Amir Shehata | Jun 22, 2020 |
Changing and Building the Linux Kernel | Amir Shehata | May 14, 2020 |
Presentations
Tasks
Medium
- LNet Router Testing
- We need to expand our testing of LNet. The link above lists a set of routing tests. We need to write LUTF scripts for them
- Benefits:
- Learn how to configure LNet routers
- Learn how to use the LUTF
- Learn how to test LNet
- Learn the code
- LU-12041