...
For LUTF-Autotest integration, the first step in this process is to build LUTF along with lustre just like other test-suites are build. The previous step "Build along Lustre tree using GNU tools" discussed fulfills this purpose. Once, the LUTF is build along with lustre, we have all the needed binary files and swig generated wrapper files used to run python test-scripts. After this -
- Autotest reads the config file (similar to how Auster has) provided specifically for LUTF to identify nodes involved in test-suite and set up environment variables.
- AT runs the Master script which triggers the execution of the test-suites.
- AT collects the
Infrastructure
Automatic Deployment
With LUTF-Autotest integration, an infrastructure is created that makes AT to deploy LUTF on the test nodes, collect results of the tests run and then pass the test results to Maloo to be displayed there.
Deploy LUTF
- A config file (similar to what Auster has) is created which can define environment variables needed. This config file is then used by AT to set these environment variables. This file would also have information like IP addresses of about the nodes involved in test-suite and their IP addresses.
- A Master script is created which can read the IP addresses of the nodes involved in the test-suite from the config file and run LUTF on the identified Agent and Master nodes.
- This Master script also triggers to run a child script that can fetch the information about the Network Interfaces (NIDs) on all the nodes involved in test-suite.
- This information of NIDs can then further be provided to each batch test (scripts to run all the similar tests related to one feature bundled together) to execute.
- The Master script then triggers the batch test script to run on the Agent nodes through the Master node identified to be used for the test-suite.
Collect Results
- A YAML format is decided for the results of the entire test-run and a result YAML file is generated per that format.
- The YAML file also points to the path where the test result file for each test is stored.
- This YAML file is then passed to AT which further passes it to Maloo.
Maloo
- A separate section is to be created in Maloo to display LUTF test results.
- The results from output YAML file passed from AT are displayed in the LUTF results section.
- A Test-parameter specifically for LUTF tests to be defined that will allow to run only LUTF tests. This will help in avoiding unnecessary tests to run for only LNet related changes.
...
* NOTE - Currently the LUTF is designed to have the Python infrastructure establish a telnet Telnet connection to facilitate Master to scp the test - scripts to Agent and then execute those test scripts. The Telnet approach can be improved upon by using SSH instead.
Batch test
how should we execute a collection of tests. You can discuss how it's currently done, and if it can be imporved.