Versions Compared

Old Version 4

changes.mady.by.user Amir Shehata

Saved on

compared with

New Version 5

changes.mady.by.user Amir Shehata

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The Unit Test Plan (UTP) will follow the same section breakdown as the Requirements in the Scope & Requirement Document.

Each Section will include the The following types of tests shall be included where it makes sense.

  1. In-Range UT - these are the test cases which cover normal operations.
  2. Out-of-Range UT - these are the test cases which cover out of range scenarios: 
    1. border cases
    2. race conditions
    3. unexpected events
      1. EX: Tearing down an active Network Interface
  3. Error UT
    1. Error parameters
    2. Error Conditions
      1. network goes down unexpectedly
      2. Wire gets disconnected, etc

...

Primary Requirement ID

Secondary Requirement IDUnit Test IDUnit Test Description
cfg-020cfg-005, cfg-010, cfg-015,cfg-045, cfg-055, cfg-060, cfg-065UT-000005 
  • Configure 3 NIDs on the same TCP network.
  • Show the NIDs
  UT-000010 
  • Configure 3 NIDs on the same IB network
  • Show the NIDs
  UT-000015 
  • Configure 3 NIDs on the same TCP/IB Network
  • Show the NIDs
  • Delete 1 NID from the TCP/IB Network
  • Show the NIDs
   
  • Configure 2 NIDs on tcp0/o2ib0
  • Configure 2 NIDs on tcp1/o2ib1
  • Show the NIDs
  • Delete 1st NID from tcp0
  • Delete 2nd NID from tcp0
  • Show NIDs
    • No more tcp 0 should exist
    • o2ib0 should be unaffected
cfg-025cfg-005, cfg-010, cfg-015,cfg-045, cfg-055, cfg-060, cfg-065 
  • Configure the system to have 4 CPTs
    • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
  • Configure 2 NIDs on tcp0
    • NID 1 should be on CPTs 0, 3
    • NID 2 should be on CPTs 1, 2
  • Show NIDs
    • proper CPT association should be displayed
cfg-035cfg-040, cfg-045, cfg-055, cfg-060, cfg-065 
  • Configure the system to have 4 CPTs
    • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
  • Configure 3 NIDs on tcp0
    • NID 1 should be on CPTs 0, 3
    • NID 2 should be on CPTs 1, 2
    • NID 3 should be on all CPTs
  • Show NIDs
    • proper CPT association should be displayed
    • NID 3 should exist on all CPTs
   
  • Configure 1st NID on tcp0 using the legacy ip2nets parameter from DLC
  • Show NIDs
   
  • Configure 1st NID on tcp*/o2ib* in the following form:
    • tcp(<eth intf>)[<cpt>] <pattern>
  • Show NIDs to ensure that the interface has been added to the correct CPTs
   
  • Configure 1st NID on tcp*/o2ib* in the following form:
    • tcp(<eth intf>, <eth intf>, ...)[<cpt>] <pattern>
  • Show NIDs to ensure that the interface has been added to the correct CPTs
   
  • Configure 1st NID on tcp*/o2ib* in the following form:
    • tcp(<eth intf>[<ctp>], <eth intf>[<ctp>], ...) <pattern>
  • Show NIDs to ensure that the interface has been added to the correct CPTs
cfg-060cfg-065 

Go through the following lnetctl commands and excercise their parameters:

  • net
  • set num_range

...

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
cfg-020cfg-005, cfg-010, cfg-015 
  • Configure 32 NIDs on the same TCP/IB Network
  • Show the NIDs
   
  • Configure 32 NIDs on the same TCP/IB Network
  • Show the NIDs.
  • Delete 32 NIDs on the same TCP/IB Network
   
  • Configure NID A, B and C on tcp0/o2ib0 Network
  • Configure NID A and B on tcp1/o2ib1 Network
  • Show the NIDs
    • Configuration should succeed. NIs can exist on different networks
cfg-060cfg-065 

Go through the following lnetctl commands and excercise their parameters, by providing out of range values:

  • net
  • set num_range
   
  • Don't configure any LNet modprobe.
  • Load LNet where there exists only one commissioned IB interface with IPoIB configured
  • a TCP network should be created with that IB interface
  • Configure an o2ib network with the same IB interface
  • Now you should have two interfaces with exactly the same IB
    
  

 

...

 

Error UT

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
   
  • Configure 128 NIDs on the same TCP/IB Network
  • Only the maximum NIDs will be configured
   
  • Configure a non-existent NID on tcp0
  • Configuration should fail with INVALID PARAMETER
   
  • Configure the system to have 4 CPTs
    • options libcfs cpu_npartitions=4 cpu_pattern="0[0] 1[1] 2[2] 3[3]"
  • Configure 3 NIDs on tcp0
    • NID 1 should be on CPTs 0, 4
    • NID 2 should be on CPTs 1, 2
    • NID 3 should be on all CPTs
  • Show NIDs
    • NID 1 should fail since no CPT 4
   

Go through the following lnetctl commands and excercise their parameters, by providing error values:

  • net
  • set num_range
   Delete a non-existent network
   Delete a non existent NID on tcp/o2ib

...

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
cfg-070  
  • add a new peer with only 1 NID
cfg-070  
  • add a new peer with only 1 NID
  • add more nids to that peer
cfg-070  
  • add a new peer with mulitple NIDs
cfg-070  
  • add a new peer with only 1 NID
  • Delete that NID
cfg-070  
  • add a new peer with multiple NIDs
  • Delete each NID one at a time until the peer is removed
cfg-070  
  • add a new peer with multiple NIDs
  • Delete all NIDs but one.
  • Re-add multiple NIDs one at a time.
cfg-070  
  • add a new peer with multiple NIDs
  • Delete all NIDs but one.
  • Re-add multiple NIDs in one shot.
cfg-075  
  • add a new peer with multiple NIDs on different networks

Out-of-Range UT

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
  git  
  • add a new peer with 32 NIDs
   
  • delete a peer and all its 32 NIDs

Error UT

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
cfg-070  
  • add more than 32 NIDs
cfg-070  
  • add a peer with multiple NIDs
  • delete a non-existent peer NID from the peer identified by key-NID
cfg-080snd-065 
  • add peer 1 with NIDs A, B and C
  • add peer 2 with NIDs D, C and E
    • Adding NID C should fail

 

Policy Configuration

In-Range UT

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
cfg-090  
  • Set the NUMA range to 0
  • The NI closest to the message memory NUMA will be picked.
cfg-090  
  • Increase the NUMA range step by step
  • Note that more NIs are picked when sending
cfg-090snd-025 
  • Set the NUMA range to a large value
  • start traffic
  • NIs are picked in round robin

Error UT

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
 cfg-090  
  • Set the NUMA range to < 0
  • This should be rejected

General Configuration

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
 cfg-170  
  • Configure multiple NIs
  • Configure multipe Peers with multiple NIDs
  • set NUMA range value
  • Dump the YAML configuration
  • use the YAML configuration file to delete all configuration
  • use the YAML configuration file to reconfigure the node.

Functional Requirements

Interface Selection and Message Sending Requirements

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
snd-005  
  • Configure 3 NIs with equadistant NUMA distance
  • Send three or more messages
  • Dump statistics on each NI to verify that each NI was used to send messages
snd-010snd-015 
  • Configure 3 NIs closer to different NUMA nodes
  • dump the NI statistics
  • Verify that each NI has the correct device CPT
snd-020  
  • Configure 3 NIs with different NUMA distances
  • Send messages
  • Confirm through statistics that messages are being sent over the nearest NI (NUMA wise)
snd-020  
  • Configure 2 NIs with different NUMA distances
  • Send messages
  • Confirm through statistics that messages are being sent over the nearest NI (NUMA wise)
  • add another NI which is close NUMA wise than the current nearest
  • confirm through statistics that messages are not being sent over the newly added NI
snd-030  
  • Configure 3 NIs, one EDR, one FDR and one QDR
  • set the NUMA range to a large value so all NIs are considered through RR
  • start traffic
  • monitor statistics on each NI.
  • Confirm that EDR is preferred until it becomes saturated, then FDR is selected then QDR
snd-030  
  • Configure 3 NIs
  • set the NUMA range to a large value so all NIs are considered through RR
  • start traffic
  • monitor statistics on each NIs to confirm all are being used.
  • Remove one of the NIs
  • Confirm that that NI is no longer used for new messages
  • Confirm that the other 2 NIs are being used.
  • No messages should be dropped.
snd-035  
  • Configure 3 NIs
  • Configure a peer with 3 NIDs
  • Send messages to the peer
  • Confirm through statistics that peer NIDs are being used based on their available credits.
snd-040  
  • Configure 3 NIs which are not equadistant all on the same network
  • configure a peer with 3 NIDs all on the same network
  • start traffic
  • Confirm closest NUMA NI is being used
  • Confirm peer NIDs are being used
  • set NUMA range to a large value
  • Confirm all NIs are being used
  • Confirm no change in traffic pattern to the peers
snd-045snd-070 
  • Configure NIs A, B and C
  • Configure the peer with the same NIDs
  • Send 1 message which requires a response from NI A
  • Confirm that responses are being sent to the same NI
snd-050  
  • Configure NIs A, B and C
  • Configure the peer with the same NIDs
  • Send 1 message which requires a response from NI A
  • bring down NI A
  • confirm that response is sent to one of the other configured NIDs
snd-050  
  • Configure an MR system
  • Start traffic
  • monitor traffic is being sent to all configured peers
  • bring down one of the peer NIDs
  • monitory traffic is no longer sent to that peer NID
  • no messages should be dropped
snd-050snd-060, snd-075 
  • Configure an MR system
  • Start traffic
  • monitor traffic is being sent to all configured peers
  • bring down one of the peer NIDs
  • monitory traffic is no longer sent to that peer NID
  • bring up the peer NID again
  • monitor traffic is being sent to it again
snd-055snd-060, snd-075 
  • Configure an MR system
  • Start traffic
  • monitor traffic is being sent to all configured peers
  • bring down one of the peer NIDs
  • monitory traffic is no longer sent to that peer NID
  • bring down all peer NIDs
  • message should fail.
snd-055snd-060, snd-075, snd-085 
  • Configure an MR system
  • Start traffic
  • monitor traffic is being sent to all configured peers over all NIs
  • bring down the local NIs one by one
  • note traffic is migrated to the NIs still up, until no NIs are left then messages are dropped
snd-055snd-060, snd-075, snd-085 
  • Configure an MR system
  • Start traffic
  • monitor traffic is being sent to all configured peers over all NIs
  • bring down the local NIs one by one
  • note traffic is migrated to the NIs still up, until no NIs are left then messages are dropped
  • bring up the NIs again and confirm that NIs are being reused.
snd-055snd-060, snd-075 
  • Configure two networks tcp and o2ib
  • Configure nodes to have multiple interfaces on each of the networks
  • start traffic over the o2ib network
  • o2ib should be used
  • bring down the o2ib network
  • traffic should migrage to the tcp network.
  • no traffic should be dropped.
snd-080  
  • Configure an MR system
  • bring down an NI
  • confirm that the show info shows the NI as down
snd-080  
  • TODO: how do we test device failure?

Dynamic NID Discovery

Debugging Requirements

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
dbg-005 dbg-010, dbg-015, dbg-020, dbg-025, dbg-030, dbg-035, dbg-080 
  • dump per NI statistics
    • transmitted
    • received
    • dropped
    • timeouts
    • state
dbg-040dbg-080, dbg-095 
  • configure multiple NIs
  • run traffic
  • dump stats on all NIs
dbg-040dbg-080 
  • configure multiple NIs
  • run traffic
  • dump stats on all NIs
  • Filter on specific NID
dbg-045dbg-080 
  • dump LNet level statistics
dbg-050dbg-080, dbg-100 
  • configure multiple peers
  • start traffic
  • dump per peer statistics
dbg-110  
  • configure multiple NIs
  • toggle their state from ACTIVE to DOWN
  • confirm that state change is being printed to console.
dbg-115  
  • configure an MR system
  • start traffic
  • bring down an NI
  • confirm that messages indicating that another NI/peer is being used is printed.
dbg-120  
  • Configure an MR system
  • run traffic
  • stop traffic
  • dump NI statistics
  • dump peer statistics
  • dump LNet level statistics
  • zero out stats
  • dump all statistics above to confirm they've been zeroed out.

Network interface Health

Backwards Compatibility Requirements

Primary Requirement IDSecondary Requirement IDUnit Test IDUnit Test Description
bck-025  
  • Configure an MR client
  • Configure an MR OSS
  • Configure a non-MR MDS/MGS
  • create an FS
  • Run IO tests
  • Make sure that MR clients/OSS integrates seamlessly in the system.