Purpose
This document describes the UDSP feature from the user perspective. The document will cover all Intended use cases, list expected behaviour and potential issues for each.
UDSP Use Cases
Local Peer Config
The following configuration is used for local testing.
| PeerA | PeerB |
|---|---|
Two nids on tcp, one nid on tcp1 | Two nids on tcp, one nid on tcp1 |
lnetctl net show
net:
- net type: lo
local NI(s):
- nid: 0@lo
status: up
- net type: tcp
local NI(s):
- nid: 192.168.122.110@tcp
status: up
interfaces:
0: eth0
- nid: 192.168.122.253@tcp
status: up
interfaces:
0: eth1
- net type: tcp1
local NI(s):
- nid: 192.168.122.238@tcp1
status: up
interfaces:
0: eth2 | lnetctl net show
net:
- net type: lo
local NI(s):
- nid: 0@lo
status: up
- net type: tcp
local NI(s):
- nid: 192.168.122.103@tcp
status: up
interfaces:
0: eth0
- nid: 192.168.122.154@tcp
status: up
interfaces:
0: eth1
- net type: tcp1
local NI(s):
- nid: 192.168.122.127@tcp1
status: up
interfaces:
0: eth2 |
Prioritize Source
Local Source NID
| UDSP | Behaviour |
|---|---|
| Prioritize one of PeerA NIDs on tcp | Prioritized nid is used when its net (tcp) is used on peerA Priority of the nid is 0 Every send toggles between the prioritized nid on tcp and the nid on tcp1 on the source. The remaining nid on tcp is not used |
lnetctl udsp add --src 192.168.122.110@tcp lnetctl udsp show
udsp:
- idx: 0
src: 192.168.122.110@tcp
dst: NA
rte: NA
action:
priority: 0
| PeerA: |
Local Source Net
| UDSP | Behaviour |
|---|---|
Prioritize tcp1 as source on PeerA Note that source NID rule is still in the list. | Prioritized tcp1 is used on PeerA Priority of tcp1 is 0 (highest) Every send on PeerA goes out tcp1. NID rule is overriden. |
lnetctl udsp add --src tcp1 lnetctl udsp show
udsp:
- idx: 0
src: 192.168.122.110@tcp
dst: NA
rte: NA
action:
priority: 0
- idx: 1
src: tcp1
dst: NA
rte: NA
action:
priority: 0 | PeerA: lnetctl net show -v 4
net:
- net type: lo
local NI(s):
- nid: 0@lo
status: up
statistics:
send_count: 0
recv_count: 0
drop_count: 0
udsp info:
net priority: -1
nid priority: -1
sent_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
received_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
dropped_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
health stats:
health value: 0
interrupts: 0
dropped: 0
aborted: 0
no route: 0
timeouts: 0
error: 0
tunables:
peer_timeout: 0
peer_credits: 0
peer_buffer_credits: 0
credits: 0
dev cpt: 0
tcp bonding: 0
CPT: "[0]"
- net type: tcp
local NI(s):
- nid: 192.168.122.110@tcp
status: up
interfaces:
0: eth0
statistics:
send_count: 9
recv_count: 9
drop_count: 0
udsp info:
net priority: -1
nid priority: 0
sent_stats:
put: 1
get: 8
reply: 0
ack: 0
hello: 0
received_stats:
put: 1
get: 2
reply: 6
ack: 0
hello: 0
dropped_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
health stats:
health value: 1000
interrupts: 0
dropped: 0
aborted: 0
no route: 0
timeouts: 0
error: 0
tunables:
peer_timeout: 180
peer_credits: 8
peer_buffer_credits: 0
credits: 256
dev cpt: -1
tcp bonding: 0
CPT: "[0]"
- nid: 192.168.122.253@tcp
status: up
interfaces:
0: eth1
statistics:
send_count: 2
recv_count: 2
drop_count: 0
udsp info:
net priority: -1
nid priority: -1
sent_stats:
put: 0
get: 2
reply: 0
ack: 0
hello: 0
received_stats:
put: 0
get: 1
reply: 1
ack: 0
hello: 0
dropped_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
health stats:
health value: 1000
interrupts: 0
dropped: 0
aborted: 0
no route: 0
timeouts: 0
error: 0
tunables:
peer_timeout: 180
peer_credits: 8
peer_buffer_credits: 0
credits: 256
dev cpt: -1
tcp bonding: 0
CPT: "[0]"
- net type: tcp1
local NI(s):
- nid: 192.168.122.238@tcp1
status: up
interfaces:
0: eth2
statistics:
send_count: 19
recv_count: 19
drop_count: 0
udsp info:
net priority: 0
nid priority: -1
sent_stats:
put: 4
get: 15
reply: 0
ack: 0
hello: 0
received_stats:
put: 1
get: 0
reply: 15
ack: 3
hello: 0
dropped_stats:
put: 0
get: 0
reply: 0
ack: 0
hello: 0
health stats:
health value: 1000
interrupts: 0
dropped: 0
aborted: 0
no route: 0
timeouts: 0
error: 0
tunables:
peer_timeout: 180
peer_credits: 8
peer_buffer_credits: 0
credits: 256
dev cpt: -1
tcp bonding: 0
CPT: "[0]"
|
Prioritize Destination
Local Destination NID
| UDSP | Behaviour |
|---|---|
On PeerA, prioritize one of PeerB's NIDs on tcp | Prioritized NID tcp NID on PeerA is used when tcp is selected on peerA Priority of the prioritized NID is 0 (lnetct peer show -v 4) Sends on PeerA alternate between tcp and tcp1. When tcp is used, send goes to the prioritized dest NID |
lnetctl udsp add --dst 192.168.122.103@tcp lnetctl udsp show
udsp:
- idx: 0
src: NA
dst: 192.168.122.103@tcp
rte: NA
action:
priority: 0 | PeerB: use "lnetct net show -v" to verify the receive count going up on the prioritized NID PeerA: use "lnetct peer show -v 4" to verify dest NID priority. |
Local Destination Net
| UDSP | Behaviour |
|---|---|
On PeerA, prioritize tcp1 as dest net | Prioritized dest net (tcp1) is used to send from PeerA Priority of the prioritized dest net is 0 (lnetct peer show -v 4) Sends on PeerA use tcp1. |
lnetctl udsp add --dst tcp1 lnetctl udsp show
udsp:
- idx: 0
src: NA
dst: tcp1
rte: NA
action:
priority: 0 | PeerB: use "lnetctl net show -v" to verify the receive count going up on the nids of the prioritized dest net PeerA: use "lnetctl peer show -v 4" to verify dest net priority. |
NID Pairing
Local Destination NID Pairing
| UDSP | Behaviour |
|---|---|
On PeerA, pair one of the tcp nids with one off the PeerB tcp nids | When paired nid on PeerA is selected, it always sends to its pair on peerB PeerA nid is listed as preferred NID for the peerB nid ("lnetctl net show -v") |
lnetctl udsp add --src 192.168.122.110@tcp --dst 192.168.122.103@tcp lnetctl udsp show
udsp:
- idx: 0
src: 192.168.122.110@tcp
dst: 192.168.122.103@tcp
rte: NA
| PeerA: use "lnetctl peer show -v 4" to verify nid pairing. |