You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Purpose

This document describes the UDSP feature from the user perspective. The document will cover all Intended use cases, list expected behaviour and potential issues for each.

UDSP Use Cases

Local Peer Config

LocalLayout1

The following configuration is used for local testing.

PeerAPeerB

Two nids on tcp, one nid on tcp1

Two nids on tcp, one nid on tcp1
lnetctl net show 
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.110@tcp
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.253@tcp
          status: up
          interfaces:
              0: eth1
    - net type: tcp1
      local NI(s):
        - nid: 192.168.122.238@tcp1
          status: up
          interfaces:
              0: eth2
lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.103@tcp
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.154@tcp
          status: up
          interfaces:
              0: eth1
    - net type: tcp1
      local NI(s):
        - nid: 192.168.122.127@tcp1
          status: up
          interfaces:
              0: eth2

Prioritize Source

Local Source NID 

UDSPBehaviour
Prioritize one of PeerA NIDs on tcp

Prioritized nid is used when its net (tcp) is used on peerA

Priority of the nid is 0. Other nids' priority is -1 (default, no priority)

Every send toggles between the prioritized nid on tcp and the nid on tcp1 on the source. The remaining nid on tcp is not used


lnetctl udsp add --src 192.168.122.110@tcp

lnetctl udsp  show 
udsp:
    - idx: 0
      src: 192.168.122.110@tcp
      dst: NA
      rte: NA
      action:
          priority: 0

PeerA:

lnetctl net show -v 4

>>>>
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.110@tcp
          status: up
          interfaces:
              0: eth0
....
          udsp info:
              net priority: -1
              nid priority: 0
<<<<

Local Source Net

 

UDSPBehaviour

Prioritize tcp1 as source on PeerA

Note that source NID rule is still in the list.

Prioritized tcp1 is used on PeerA

Priority of tcp1 is 0 (highest)

Every send on PeerA goes out tcp1. NID rule is overriden.

lnetctl udsp add --src tcp1

lnetctl udsp show
udsp:
    - idx: 0
      src: 192.168.122.110@tcp
      dst: NA
      rte: NA
      action:
          priority: 0
    - idx: 1
      src: tcp1
      dst: NA
      rte: NA
      action:
          priority: 0




PeerA:
lnetctl net show -v 4

>>>>
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.110@tcp
          status: up
          interfaces:
              0: eth0
....
          udsp info:
              net priority: -1
              nid priority: 0
<<<<
>>>>    
    - net type: tcp1
      local NI(s):
        - nid: 192.168.122.238@tcp1
          status: up
          interfaces:
              0: eth2
....
          udsp info:
              net priority: 0
              nid priority: -1
<<<<

Prioritize Destination

Local Destination NID

UDSPBehaviour

On PeerA, prioritize one of PeerB's NIDs on tcp 


Prioritized NID tcp NID on PeerA is used when tcp is selected on peerA

Priority of the prioritized NID is 0 (lnetct peer show -v 4)

Sends on PeerA alternate between tcp and tcp1. When tcp is used, send goes to the prioritized dest NID

lnetctl udsp add --dst 192.168.122.103@tcp

lnetctl udsp show
udsp:
    - idx: 0
      src: NA
      dst: 192.168.122.103@tcp
      rte: NA
      action:
          priority: 0

PeerB: use "lnetct net show -v" to verify the receive count going up on the prioritized NID 

PeerA: use "lnetct peer show -v 4" to verify dest NID priority.

Local Destination Net

UDSPBehaviour

On PeerA, prioritize tcp1 as dest net


Prioritized dest net (tcp1) is used to send from PeerA

Priority of the prioritized dest net is 0 (lnetct peer show -v 4)

Sends on PeerA use tcp1.

lnetctl udsp add --dst tcp1

lnetctl udsp show
udsp:
    - idx: 0
      src: NA
      dst: tcp1
      rte: NA
      action:
          priority: 0

PeerB: use "lnetctl net show -v" to verify the receive count going up on the nids of the prioritized dest net 

PeerA: use "lnetctl peer show -v 4" to verify dest net priority.

NID Pairing

Local Destination NID Pairing

UDSPBehaviour

On PeerA, pair one of the tcp nids with one off the PeerB tcp nids 


When paired nid on PeerA is selected, it always sends to its pair on peerB

PeerA nid is listed as preferred NID for the peerB nid ("lnetctl net show -v")

lnetctl udsp add --src 192.168.122.110@tcp --dst 192.168.122.103@tcp

lnetctl udsp show
udsp:
    - idx: 0
      src: 192.168.122.110@tcp
      dst: 192.168.122.103@tcp
      rte: NA

PeerA: use "lnetctl peer show -v 4" to verify nid pairing.



Remote Peer Config

The following configuration is used for testing that involves remote peers and routing:

PeerAPeerBGateway1Gateway2
two nids on tcptwo nids on tcp2two nids on tcp, two nids on tcp2two nids on tcp, two nids on tcp2
lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.103@tcp
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.154@tcp
          status: up
          interfaces:
              0: eth1
lnetctl route show
route:
    - net: tcp2
      gateway: 192.168.122.132@tcp
    - net: tcp2
      gateway: 192.168.122.102@tcp
lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp2
      local NI(s):
        - nid: 192.168.122.110@tcp2
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.253@tcp2
          status: up
          interfaces:
              0: eth1
lnetctl route show
route:
    - net: tcp
      gateway: 192.168.122.26@tcp2
    - net: tcp
      gateway: 192.168.122.200@tcp2
lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.102@tcp
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.252@tcp
          status: up
          interfaces:
              0: eth1
    - net type: tcp2
      local NI(s):
        - nid: 192.168.122.237@tcp2
          status: up
          interfaces:
              0: eth2
        - nid: 192.168.122.200@tcp2
          status: up
          interfaces:
              0: eth3
lnetctl net show
net:
    - net type: lo
      local NI(s):
        - nid: 0@lo
          status: up
    - net type: tcp
      local NI(s):
        - nid: 192.168.122.132@tcp
          status: up
          interfaces:
              0: eth0
        - nid: 192.168.122.150@tcp
          status: up
          interfaces:
              0: eth1
    - net type: tcp2
      local NI(s):
        - nid: 192.168.122.26@tcp2
          status: up
          interfaces:
              0: eth2
        - nid: 192.168.122.244@tcp2
          status: up
          interfaces:
              0: eth3

Gateway Preference For Source

Preferred Gateway For Source Net

UDSPBehaviour
On PeerB, designate Gateway1 as preferred on tcp2

All sends from PeerB use Gateway1

Use "lnetctl stats show" on Gateway1 and Gateway2. On every send from PeerB to PeerA, "route_count" should increase only on Gateway1. On PeerB, Gateway1 should be listed as preferred for every nid on tcp2:

lnetctl udsp add --src tcp2 --rte 192.168.122.102@tcp

lnetctl udsp show
udsp:
    - idx: 0
      src: tcp2
      dst: NA
      rte: 192.168.122.102@tcp

PeerB:

lnetctl net show -v 4

>>>>
    - net type: tcp2
      local NI(s):
        - nid: 192.168.122.110@tcp2
          status: up
          interfaces:
              0: eth0
....
          udsp info:
              net priority: -1
              nid priority: -1
              Preferred gateway NIDs:
                  NID-0: 192.168.122.102@tcp
<<<<
>>>>
        - nid: 192.168.122.253@tcp2
          status: up
          interfaces:
              0: eth1
....
          udsp info:
              net priority: -1
              nid priority: -1
              Preferred gateway NIDs:
                  NID-0: 192.168.122.102@tcp
>>>>

Preferred Gateway For Source NID

Current behaviour in this case is to make the specified gateway preferred for all nids on the same net as the specified nid.

Gateway Preference For Destination

Preferred Gateway For Destination NID

UDSPBehaviour

On PeerB, add UDSP that gives one of the PeerA nids highest priority

On PeerB, designate Gateway1 as preferred for the same PeerA nid.

All sends from PeerB go to the prioritized nid of PeerA through the preferred gateway 

Use "lnetctl stats show" on Gateway1 and Gateway2. On every send from PeerB to PeerA, "route_count" should increase only on Gateway1. On PeerA "send_count" and "recv_count" should increase only for the prioritised nid. On PeerB, Gateway1 should be listed as preferred for the specified nid of PeerA on tcp:

lnetctl udsp add --dst 192.168.122.103@tcp

lnetctl udsp add --dst 192.168.122.103@tcp --rte 192.168.122.102@tcp

lnetctl udsp show
udsp:
    - idx: 0
      src: NA
      dst: 192.168.122.103@tcp
      rte: NA
      action:
          priority: 0
    - idx: 1
      src: NA
      dst: 192.168.122.103@tcp
      rte: 192.168.122.102@tcp

PeerB:

lnetctl peer show -v 4

>>>>
        - nid: 192.168.122.103@tcp
          udsp info:
              net priority: -1
              nid priority: 0
              Preferred gateway NIDs:
                  NID-0: 192.168.122.102@tcp
<<<<


Preferred Gateway For Destination Net

UDSPBehaviour

On PeerB, designate Gateway1 as preferred for tcp remote network

 The preferred gateway for tcp is used for all traffic

Use "lnetctl stats show" on Gateway1 and Gateway2. On every send from PeerB to PeerA, "route_count" should increase only on Gateway1. On PeerB, Gateway1 should be listed as preferred for every nid on tcp:

lnetctl udsp add --dst tcp --rte 192.168.122.102@tcp

lnetctl udsp show
udsp:
    - idx: 0
      src: NA
      dst: tcp
      rte: 192.168.122.102@tcp

PeerB:

lnetctl peer show -v 4

>>>>
        - nid: 192.168.122.103@tcp
          udsp info:
              net priority: -1
              nid priority: -1
              Preferred gateway NIDs:
                  NID-0: 192.168.122.102@tcp
<<<<<
>>>>>
        - nid: 192.168.122.154@tcp
          udsp info:
              net priority: -1
              nid priority: -1
              Preferred gateway NIDs:
                  NID-0: 192.168.122.102@tcp

<<<<
  • No labels