Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

LUTF IDIssue

Unit Test Description

cfg-095cfg-160
test_udsp_basic_01pass
  • Setup: configure single network, 3 NIDs on the network 
  • Execute UDSP "Show" command 
  • Verify that the resulting YAML file doesn't contain any policy description
cfg-125, cfg-100cfg-115, cfg-125

test_udsp_basic_02 

it seems the tc only add rule to 1 network instead of NID, also not lower the priority

pass
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives two of the interfaces highest priority
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
  • Add UDSP rule that lowers the priority of one of the NIDs with highest priority back to default
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
cfg-110

test_udsp_basic_03

tc only delete rule on network, doesn't delete the rule on NID

pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • Add UDSP rule that gives the highest priority to one of the NIDs on the network with the highest priority
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected (for network and NID)
  • Delete UDSP rule that gives the highest priority to one of the NIDs
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected (for network)
  • Delete UDSP rule that gives the highest priority to one of the networks
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file doesn't contain any policy description
cfg-115

test_udsp_basic_04

tc doesn't delete rule

pass
  • Setup: configure single network, 3 NIDs on the network locally, 3 NIDs on the remote peer
  • Add UDSP rule that gives priority to one of the remote peer's NIDs
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
  • Delete UDSP rule that gives the highest priority to the remote peer's NID
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file doesn't contain any policy description
cfg-120

test_udsp_basic_05

tc doesn't delete rule

pass
  • Setup: configure single network, 3 NIDs on the network locally, 3 NIDs on the local peer
  • Add UDSP rule that gives priority to a pair of the local NID and peer's NID
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
  • Delete UDSP rule that gives the highest priority to the NID pair
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file doesn't contain any policy description



test_udsp_basic_06

tc doesn't delete rule

pass
  • Configure single local network with two NIDs on the network 
  • Configure two routers/gateways each providing access to a remote network
  • Configure remote peer to have access to the remote network with two NIDs
  • Add UDSP rule that designates a pair of a router and remote peer NID as preferred
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
  • Delete UDSP rule that gives the highest priority to the router/remote NID pair
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file doesn't contain any policy description
TODO: add N:1, 1:N, N:N mapping tests




TODO: update requirements to cover router-related rules




...

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

LUTF IDIssue

Unit Test Description

cfg-105
basic-cfg-err-001test_udsp_basic_err_01fail (return -22), the error seems known issue listed below
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives two of the interfaces highest priority
  • Execute UDSP "Show" command
  • Verify that the resulting YAML file contains policy description as expected
  • Execute UDSP "Delete" command providing policy index that doesn't exist
  • Verify that the command failed with an error


basic-cfg-err-002test_udsp_basic_err_02pass
  • Setup: configure single network, 3 NIDs on the network 
  • Verify that attempts to add UDSP rule for source or dst network fail: "0tcp", "0o2ib", "tcpp", "t", "tcp100000000000000000000000000000000000000000000000001"


basic-cfg-err-003case missing; manually tested
Jira
serverWhamcloud Community Jira
serverId8bba2dd1-4333-3006-bfcd-f35d4ebbd2ad
keyLU-15749
  • Setup: configure single network, 3 NIDs on the network 
  • Verify that attempts to add UDSP rule for source or dst NID fail: "@tcp", "1@tcp", "1.1.1@tcp", "1.1.1.1.1@tcp"

...

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

LUTF IDIssue

Unit Test Description




test_udsp_single_net_01pass
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives one of the local interfaces highest priority
  • Start traffic TODO: need to specify direction?
  • Stop traffic
  • Verify that the NID with the highest priority was used



test_udsp_single_net_02pass
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives two of the local interfaces highest priority
  • Start traffic
  • Stop traffic
  • Verify that two NIDs with the highest priority were used 



test_udsp_single_net_03pass
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives two of the interfaces highest priority
  • Start traffic
  • Stop traffic
  • Verify that two NIDs with the highest priority were used
  • Add UDSP that lowers the priority of one of the NIDs with highest priority back to default
  • Start traffic
  • Stop traffic
  • Verify that the remaining highest-priority NID was used



test_udsp_single_net_04
Jira
serverWhamcloud Community Jira
serverId8bba2dd1-4333-3006-bfcd-f35d4ebbd2ad
keyLU-15746
  • Setup: configure single network, 3 NIDs on the network 
  • Add UDSP rule that gives two of the interfaces highest priority
  • Start traffic
  • Stop traffic
  • Verify that two NIDs with the highest priority were used
  • Add UDSP that lowers the priority of both of the NIDs with the highest priority back to default
  • Start traffic
  • Stop traffic
  • Verify that all NIDs were used



test_udsp_single_net_05pass
  • Setup: configure single network, 3 NIDs on the network, 3 NIDs on the local peer
  • Add UDSP rule that gives the highest priority to one of the local peer NIDs
  • Start traffic
  • Stop traffic
  • Verify that local peer NID with the highest priority was used



test_udsp_single_net_06pass
  • Setup: configure single network, 3 NIDs on the network, 3 NIDs on the local peer
  • Add UDSP rule that gives one of the local NIDs highest priority
  • Add UDSP rule that pairs the highest priority local NID with a certain peer NID
  • Start traffic
  • Stop traffic
  • Verify that the paired peer NID was used


singlenet-00607?
Jira
serverWhamcloud Community Jira
serverId8bba2dd1-4333-3006-bfcd-f35d4ebbd2ad
keyLU-15747
  • Setup: configure single network, 3 NIDs on the network, 3 NIDs on the local peer
  • Add UDSP rule that creates 3 NID pairs such that one of the peer NIDs is not in any pair
  • Start traffic
  • Stop traffic
  • Verify that the peer NID that is not part of any pair was not used (less used?)
  • Delete UDSP rule
  • Start traffic
  • Stop traffic
  • Verify that all NIDs are used 

...

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

LUTF IDIssue

Unit Test Description




test_udsp_multi_net_01pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • Start traffic
  • Stop traffic
  • Verify that NIDs on the network with the highest priority were used



test_udsp_multi_net_02pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • Start traffic
  • Stop traffic
  • Verify that NIDs on the network with the highest priority were used
  • Add UDSP rule that reverses networks priorities
  • Start traffic
  • Stop traffic
  • Verify that NIDs on the new network that has the highest priority were used



test_udsp_multi_net_03pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • Add UDSP rule that gives the highest priority to one of the NIDs on the network with the highest priority
  • Start traffic
  • Stop traffic
  • Verify that the NID with highest priority on the network with highest priority was used



test_udsp_multi_net_04pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • Add UDSP rule that gives the highest priority to one of the NIDs on the network with the lowest priority
  • Start traffic
  • Stop traffic
  • Verify that the NIDs on the network with highest priority were used



test_udsp_multi_net_05pass
  • Setup: configure two networks, two NIDs on each network
  • Add UDSP rule that gives one of the networks highest priority
  • On the peer add UDSP rule that gives the highest priority to the other network
  • Start traffic
  • Stop traffic
  • Verify that the local network with highest priority was used

...

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

LUTF IDIssue

Unit Test Description




test_udsp_routed_net_01

the test itself passed, but the unconfigure code(at the end of the script) return error

error: "LUTFError:\n  node-name: mds_HOST\n  msg: lustre_rmmod failed\n  arg:\

          \ null\n  file name: /usr/lib64/lustre/tests/lutf/python/tests-infra/lnet.py\n\

          \  line number: 611\n  function: unconfigure\n"


if comment out the unconfigure_lnet(), test can pass

lutf>>> R()

Tests:

  - name: lutf-udsp

    description: auster lutf

    SubTests:

      - status: PASS

        duration: 8

        return_code: 0

        name: udsp_routed_net_01

    duration: 8

    status: PASS

    submission: Fri Apr 22 17:07:56 UTC 2022

  • Configure single local network with two NIDs on the network  (RemoteLayout1)
  • Configure two routers/gateways each providing access to a remote network
  • Configure remote peer to have access to the remote network with two NIDs
  • Add UDSP rule that designates a pair of a router and remote peer NIDs as preferred
  • Add UDSP rule that gives the same remote peer NID highest priority
  • Start traffic
  • Stop traffic
  • Verify that the preferred remote peer NID and router NID were used




pass - manually run
  • Configure single local network with two NIDs on the network (RemoteLayout1)
  • Configure two routers/gateways each providing access to a remote network
  • Configure remote peer to have access to the remote network with two NIDs
  • Assign higher route priority to the route using gateway 1
  • Add UDSP rule that designates a pair of a router (gateway 2) and remote peer NIDs as preferred
  • Start traffic
  • Stop traffic
  • Verify that the higher priority route was used (gateway 1)




fail - manually run

fail to set up the env as layout2, I can make the top path passed, but the bottom one wont work

  • Configure two local networks (net1, net2) with two NIDs on each network (RemoteLayout2)
  • Configure remote peer to have access to the remote network with two NIDs
  • On peerA add UDSP rule that designates gateway 1 as preferred for all NIDs on peerB
  • Start traffic (peerA to peerB)
  • Stop traffic
  • Verify that if net1 was used then the paired router was used 

Out-of-Range UT

Primary Requirement ID

Secondary Requirement ID

Unit Test ID

Issue

Unit Test Description




pass with only non-exist router since following error(manually run)

while udsp doesn't support add --src and --rte at the same time

[root@trevis-88vm4 lutf]# lnetctl udsp add --src 10.240.43.83@tcp --rte 10.240.43.80@tcp

add:

    - udsp:

          errno: -1

          descr: "The combination of src, dst and rte is not supported"

  • Configure single local network with two NIDs on the network  (RemoteLayout1)
  • Configure two routers/gateways each providing access to a remote network
  • Configure remote peer to have access to the remote network with two NIDs
  • Add UDSP rule that designates a pair of a router and remote peer NIDs as preferred
  • Add UDSP rule that gives the same remote peer NID highest priority
  • Start traffic
  • Stop traffic
  • Verify that the preferred remote peer NID and router NID were used

    • Configure single local network with two NIDs on the network (RemoteLayout1)network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDsAssign higher route priority to the route using gateway 1
    • Add UDSP rule that designates a pair of a router (gateway 2) and remote peer NIDs of a router that doesn't exist and a local network as preferred
    • Start traffic
    • Stop traffic
    • Verify that the higher priority route was used (gateway 1)both routers were used



    not valid case due to the error showed above. not support udsp rule with --src and --rte at the same time
    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote networkConfigure two local networks (net1, net2) with two NIDs on each network (RemoteLayout2)
    • Configure remote peer to have access to the remote network with two NIDs
    • On peerA add Add UDSP rule that designates gateway 1 as preferred for all NIDs on peerB
    • Start traffic (peerA to peerB)
    • Stop traffic
    • Verify that if net1 was used then the paired router was used 

    Out-of-Range UT

    Unit Test Description
    • a pair of a router and a local network that doesn't exist as preferred
    • Start traffic
    • Stop traffic
    • Verify that both routers were used



    pass - manually run

    Primary Requirement ID

    Secondary Requirement ID

    Unit Test ID

    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDs
    • Add UDSP rule that designates a pair of a router that doesn't exist and a local network remote peer NID as preferred
    • Add UDSP rule that gives the same remote peer NID highest priority
    • Start traffic
    • Stop traffic
    • Verify that the all peer NIDs and both routers were used



    pass - manually run

    UDSP rule

    lnetctl udsp add --dst tcp1 --rte 10.240.43.80@tcp

    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDs
    • Add UDSP rule that designates a pair of a router and a local network that doesn't exist peerB NID as preferred
    • Start trafficStop traffic
    • Verify that the preferred router was used
    • Bring down the preferred router
    • Verify that both routers were used
    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDs
    • Add UDSP rule that designates a pair of a router that doesn't exist and a remote peer NID as preferred
    • Add UDSP rule that gives the same remote peer NID highest priority
    • Start traffic
    • Stop traffic
    • Verify that the all peer NIDs and both routers were used
    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDs
    • Add UDSP rule that designates a pair of a router and a peerB NID as preferred
    • Start traffic
    • Verify that the preferred router was used
    • Bring down the preferred router
    • Verify that the remaining router was used
    • Stop traffic

    Interoperability with LNet Health feature

    Tests cases listed in this section are intended to verify that UDSP and LNet Health features work properly when both are enabled. The expected behaviour is that UDSP decisions come second to the health-based decisions, such that, for example, given healthier alternatives, NID with a lower health score won't be chosen even if it is preferred according to a UDSP rule. More detail on how to simulate events that affect the health score can be found here: LNet Health Test Plan [3].

    • the remaining router was used
    • Stop traffic

    Interoperability with LNet Health feature

    Tests cases listed in this section are intended to verify that UDSP and LNet Health features work properly when both are enabled. The expected behaviour is that UDSP decisions come second to the health-based decisions, such that, for example, given healthier alternatives, NID with a lower health score won't be chosen even if it is preferred according to a UDSP rule. More detail on how to simulate events that affect the health score can be found here: LNet Health Test Plan [3].


    Primary Requirement ID

    Secondary Requirement ID

    Unit Test ID

    Issue

    Unit Test Description

    Primary Requirement ID

    Secondary Requirement ID

    Unit Test ID

    Unit Test Description




    pass - manually run

    add UDSP rule to interface A

    simulate the error by "lctl net_drop_add -s A -d B -i 2"

    do lctl ping B from A

    check lnetctl net show -v 3 and found send_count is increased on A as well as other interfaces,  the same time, health stats dropped number increased too

    delete the rule by "lctl net_drop_del" and do lctl ping again

    send_count shows only A increased by 1

    *health value didn't been noticed decrease, but developer checked lnet debug log and it showed the changing

    • MR Node with Multiple interfaces
    • Add UDSP rule that gives one of the local interfaces highest priority
    • Send a PING
    • Verify that the NID with the highest priority was used
    • Send a PING
    • Simulate an <error>
    • PING msg should be queued on resend queue
    • PING msg will be resent on a different interface
    • Failed interface's health value will be decremented
    • Failed interface will be placed on the recovery queue
    • Eventually health score should recover so then only the preferred interface will be used (sensitivity > 0)will be used (sensitivity > 0)



    pass - manually run

    add UDSP on  tcp1

    then set that nid A in tcp1 in net_drop, the traffic will go either the good interface on tcp1 or failed. we got failed ping every other time. 

    • MR Node with Multiple interfaces: configure two networks, two NIDs on each network
    • Add UDSP rule that gives one of the networks highest priority
    • Start traffic
    • Stop traffic
    • Verify that NIDs on the network with the highest priority were used
    • Start traffic
    • Simulate an <error> (drop on one of the NIDs on the highest priority network)
    • Failed interface's health value will be decremented
    • Failed interface will be placed on the recovery queue
    • Verify that the remaining interface on the failed network is used
    • Eventually health score should recover so then only both interface on the preferred network will be used (sensitivity > 0)



    pass - manually run
    • Configure single local network with two NIDs on the network 
    • Configure two routers/gateways each providing access to a remote network
    • Configure remote peer to have access to the remote network with two NIDs
    • Add UDSP rule that designates a router as preferred for all remote peer NIDs 
    • Start traffic
    • Stop traffic
    • Verify that the preferred router was used
    • Add a PUT drop rule on the router to drop traffic on one of the interfaces.
    • Verify that traffic goes to the other interfaces of the preferred router. There shouldn't be any drop in traffic.
    • Add a PUT drop rule on the router to drop traffic on all interfaces.
    • Verify that traffic goes through the non-preferred router until one of the interfaces of the preferred router recovers.

    ...

    Primary Requirement ID

    Secondary Requirement ID

    Unit Test ID

    Issue

    Unit Test Description




    pass - manually run

    (disable by lnetctl set discovery 0)

    • Setup: configure single network, 2 NIDs on the network, peer with MR disabled 
    • Add UDSP rule that gives one of the interfaces highest priority
    • Start traffic
    • Stop traffic
    • Verify that the NID with the highest priority was used 



    pass - manually run

    (interop with a 2.10.8 node which doesn't have MR)

    • Setup: configure single network, 2 NIDs on the network, peer incapable of MR ("down-rev")
    • Add UDSP rule that gives one of the interfaces highest priority
    • Start traffic
    • Stop traffic
    • Verify that the NID with the highest priority was used 



    pass - manually run
    • Setup: configure single network, 2 NIDs on the network, 2 NIDs on the local peer with MR disabled
    • Add UDSP rule that gives the highest priority to one of the local peer NIDs
    • Start traffic
    • Stop traffic
    • Verify that local peer NID with the highest priority was used



    pass - manually run
    • Setup: configure single network, 2 NIDs on the network, 2 NIDs on the local peer incapable of MR ("down-rev")
    • Add UDSP rule that gives the highest priority to one of the local peer NIDs
    • Start traffic
    • Stop traffic
    • Verify that local peer NID with the highest priority was used

    ...