• Configure 3 NIDs on the same TCP network.
• Show the NIDs

• Configure 3 NIDs on the same IB network
• Show the NIDs

• Configure 3 NIDs on the same TCP/IB Network
• Show the NIDs
• Delete 1 NID from the TCP/IB Network
• Show the NIDs

• Configure 2 NIDs on tcp0/o2ib0
• Configure 2 NIDs on tcp1/o2ib1
• Show the NIDs
• Delete 1st NID from tcp0
• Delete 2nd NID from tcp0
• Show NIDs
  • No more tcp 0 should exist
  • o2ib0 should be unaffected

• Configure the system to have 4 CPTs
  • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
• Configure 2 NIDs on tcp0
  • NID 1 should be on CPTs 0, 3
  • NID 2 should be on CPTs 1, 2
• Show NIDs
  • proper CPT association should be displayed

• Configure the system to have 4 CPTs
  • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
• Configure 3 NIDs on tcp0
  • NID 1 should be on CPTs 0, 3
  • NID 2 should be on CPTs 1, 2
  • NID 3 should be on all CPTs
• Show NIDs
  • proper CPT association should be displayed
  • NID 3 should exist on all CPTs

• Configure 1st NID on tcp0 using the legacy ip2nets parameter from DLC
• Show NIDs

• Configure 1st NID on tcp*/o2ib* in the following ip2nets form:
  • tcp(<eth intf>)[<cpt>] <pattern>
• Show NIDs to ensure that the interface has been added to the correct CPTs

• Configure 1st NID on tcp*/o2ib* in the following ip2nets form:
  • tcp(<eth intf>, <eth intf>, ...)[<cpt>] <pattern>
• Show NIDs to ensure that the interface has been added to the correct CPTs

• Configure 1st NID on tcp*/o2ib* in the following ip2nets form:
  • tcp(<eth intf>[<ctp>], <eth intf>[<ctp>], ...) <pattern>
• Show NIDs to ensure that the interface has been added to the correct CPTs

Go through the following lnetctl commands and excercise their parameters:

• net
• set num_range

• Configure 32 NIDs on the same TCP/IB Network
• Show the NIDs

• Configure 32 NIDs on the same TCP/IB Network
• Show the NIDs.
• Delete 32 NIDs on the same TCP/IB Network

• Configure NID A, B and C on tcp0/o2ib0 Network
• Configure NID A and B on tcp1/o2ib1 Network
• Show the NIDs
  • Configuration should succeed. NIs can exist on different networks

Go through the following lnetctl commands and excercise their parameters, by providing out of range values:

• net
• set num_range

• Don't configure any LNet modprobe.
• Load LNet where there exists only one commissioned IB interface with IPoIB configured
• a TCP network should be created with that IB interface
• Configure an o2ib network with the same IB interface
• Now you should have two interfaces with exactly the same IB

• Configure 128 NIDs on the same TCP/IB Network
• Only the maximum NIDs will be configured

• Configure a non-existent NID on tcp0
• Configuration should fail with INVALID PARAMETER

• Configure the system to have 4 CPTs
  • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
• Configure 3 NIDs on tcp0
  • NID 1 should be on CPTs 0, 4
  • NID 2 should be on CPTs 1, 2
  • NID 3 should be on all CPTs
• Show NIDs
  • NID 1 should fail since no CPT 4

Go through the following lnetctl commands and excercise their parameters, by providing error values:

• net
• set num_range

• add a new peer with only 1 NID

• add a new peer with only 1 NID
• add more nids to that peer

• add a new peer with mulitple NIDs

• add a new peer with only 1 NID
• Delete that NID

• add a new peer with multiple NIDs
• Delete each NID one at a time until the peer is removed

• add a new peer with multiple NIDs
• Delete all NIDs but one.
• Re-add multiple NIDs one at a time.

• add a new peer with multiple NIDs
• Delete all NIDs but one.
• Re-add multiple NIDs in one shot.

• add a new peer with multiple NIDs on different networks

• add a new peer with 32 NIDs

• delete a peer and all its 32 NIDs

• load lnet
• lnetctl lnet configure
• add 2 or more peers on a non-local network
• delete peer 1
• delete peer 2

• load lnet
• lnetctl lnet configure
• add 2 or more peers on a non-local network
• lnetctl lnet unconfigure
• lustre_rmmod

• startup lnet
• startup traffic
• add a peer ni on a non-local network
• add a local network for that peer
• Send traffic over that peer_ni

• add more than 32 NIDs

• add a peer with multiple NIDs
• delete a non-existent peer NID from the peer identified by key-NID

• add peer 1 with NIDs A, B and C
• add peer 2 with NIDs D, C and E
  • Adding NID C should fail

• Set the NUMA range to 0
• The NI closest to the message memory NUMA will be picked.

• Increase the NUMA range step by step
• Note that more NIs are picked when sending

• Set the NUMA range to a large value
• start traffic
• NIs are picked in round robin

• Set the NUMA range to < 0
• This should be rejected

• Configure multiple NIs
• Configure multipe Peers with multiple NIDs
• set NUMA range value
• Dump the YAML configuration
• use the YAML configuration file to delete all configuration
• use the YAML configuration file to reconfigure the node.

• Configure 3 NIs with equadistant NUMA distance
• Send three or more messages
• Dump statistics on each NI to verify that each NI was used to send messages

• Configure 3 NIs closer to different NUMA nodes
• dump the NI statistics
• Verify that each NI has the correct device CPT

• Configure 3 NIs with different NUMA distances
• Send messages
• Confirm through statistics that messages are being sent over the nearest NI (NUMA wise)

• Configure 2 NIs with different NUMA distances
• Send messages
• Confirm through statistics that messages are being sent over the nearest NI (NUMA wise)
• add another NI which is close NUMA wise than the current nearest
• confirm through statistics that messages are not being sent over the newly added NI

• Configure 3 NIs, one EDR, one FDR and one QDR
• set the NUMA range to a large value so all NIs are considered through RR
• start traffic
• monitor statistics on each NI.
• Confirm that EDR is preferred until it becomes saturated, then FDR is selected then QDR

• Configure 3 NIs
• set the NUMA range to a large value so all NIs are considered through RR
• start traffic
• monitor statistics on each NIs to confirm all are being used.
• Remove one of the NIs
• Confirm that that NI is no longer used for new messages
• Confirm that the other 2 NIs are being used.
• No messages should be dropped.

• Configure 3 NIs
• Configure a peer with 3 NIDs
• Send messages to the peer
• Confirm through statistics that peer NIDs are being used based on their available credits.

• Configure 3 NIs which are not equadistant all on the same network
• configure a peer with 3 NIDs all on the same network
• start traffic
• Confirm closest NUMA NI is being used
• Confirm peer NIDs are being used
• set NUMA range to a large value
• Confirm all NIs are being used
• Confirm no change in traffic pattern to the peers

• Configure NIs A, B and C
• Configure the peer with the same NIDs
• Send 1 message which requires a response from NI A
• Confirm that responses are being sent to the same NI

• Configure NIs A, B and C
• Configure the peer with the same NIDs
• Send 1 message which requires a response from NI A
• bring down NI A
• confirm that response is sent to one of the other configured NIDs

• Configure an MR system
• Start traffic
• monitor traffic is being sent to all configured peers
• bring down one of the peer NIDs
• monitory traffic is no longer sent to that peer NID
• no messages should be dropped

• Configure an MR system
• Start traffic
• monitor traffic is being sent to all configured peers
• bring down one of the peer NIDs
• monitory traffic is no longer sent to that peer NID
• bring up the peer NID again
• monitor traffic is being sent to it again

• Configure an MR system
• Start traffic
• monitor traffic is being sent to all configured peers
• bring down one of the peer NIDs
• monitory traffic is no longer sent to that peer NID
• bring down all peer NIDs
• message should fail.

• Configure an MR system
• Start traffic
• monitor traffic is being sent to all configured peers over all NIs
• bring down the local NIs one by one
• note traffic is migrated to the NIs still up, until no NIs are left then messages are dropped

• Configure an MR system
• Start traffic
• monitor traffic is being sent to all configured peers over all NIs
• bring down the local NIs one by one
• note traffic is migrated to the NIs still up, until no NIs are left then messages are dropped
• bring up the NIs again and confirm that NIs are being reused.

• Configure two networks tcp and o2ib
• Configure nodes to have multiple interfaces on each of the networks
• start traffic over the o2ib network
• o2ib should be used
• bring down the o2ib network
• traffic should migrage to the tcp network.
• no traffic should be dropped.

• Configure an MR system
• bring down an NI
• confirm that the show info shows the NI as down

• TODO: how do we test device failure?

• Configure an MR system
• Configure peers via DLC
• Run traffic
• Delete one of the peer_nis we're sending to via DLC
• Traffic going over that peer_ni should continue but no more traffic should use that NI

• Configure an MR system
• Configure peers via DLC
• Run traffic
• Delete one of the peer_nis we're sending to via DLC
• Bring that peer_ni back
• Note traffic stops and starts on that peer with no traffic loss
• Repeat the deletion and reconfiguration of the peer_ni

• Configure an MR system
• Configure peers via DLC
• Run traffic
• Delete the entire peer
• The peer should be recreated on the next message, but it won't be MR capable.

• dump per NI statistics
  • transmitted
  • received
  • dropped
  • timeouts
  • state

• configure multiple NIs
• run traffic
• dump stats on all NIs

• configure multiple NIs
• run traffic
• dump stats on all NIs
• Filter on specific NID

• dump LNet level statistics

• configure multiple peers
• start traffic
• dump per peer statistics

• configure multiple NIs
• toggle their state from ACTIVE to DOWN
• confirm that state change is being printed to console.

• configure an MR system
• start traffic
• bring down an NI
• confirm that messages indicating that another NI/peer is being used is printed.

• Configure an MR system
• run traffic
• stop traffic
• dump NI statistics
• dump peer statistics
• dump LNet level statistics
• zero out stats
• dump all statistics above to confirm they've been zeroed out.

• Configure an MR client
• Configure an MR OSS
• Configure a non-MR MDS/MGS
• create an FS
• Run IO tests
• Make sure that MR clients/OSS integrates seamlessly in the system.