There may be times that you wish to influence the building and testing carried out on your change. You might for example be fixing an issue that affects a particular distribution or combination of distributions, for this reason you can change the tests carried out by indicating in the commit message the changes you require.
To do this place the following comments with space-separated name value pairs in your commit message:
Test-Parameters: trivial envdefinitions=ONLY=32 testlist=conf-sanity
To cater for multiple requirements we allow for lists of requirements, with each Test-Parameters: line being a separate test that can be run in parallel with other tests:
Test-Parameters: trivial ostfilesystemtype=ldiskfs testlist=sanity,sanity,sanity,sanity Test-Parameters: trivial ostfilesystemtype=zfs testlist=sanity,sanity,sanity,sanity
Long lists can be catered for by escaping the carriage return:
Test-Parameters: ostcount=2 clients=1 ostsizegb=2 mdssizegb=2 envdefinitions=SLOW=yes \ testlist=sanity,liblustre
testlist=
Multiple tests are separated by a comma, and can be specified multiple times. This is useful if a patch is fixing a problem that is only hit intermittently by a test script, and multiple tests need to be run in order to gain more confidence that the problem is actually fixed. For example, if replay-dual was failing 1/3 of test runs, it would be useful to run it at least twice the average number of times needed to hit a failure to have some reasonable expectation that the bug is fixed. With just a single test run, there would be a 66% chance a single test would pass even if the bug was not fixed, but only about a 9% chance (2/36 = 64/729) that 6 separate tests would pass without the bug being fixed. If the number of tests needed is more than about 3-4 (depending on how long each test runs), then multiple Test-Parameters: lines should be used, so that the tests are run in separate sessions (usually in parallel) rather than in series.
Test-Parameters: ostcount=2 clients=1 ostsizegb=2 mdssizegb=2 envdefinitions=SLOW=yes \
testlist=replay-dual,replay-dual,replay-dual,replay-dual,replay-dual,replay-dual
fortestonly
Test-Parameters: supplied with a normal commit are in addition to the normal tests that would be run against a patch. This allows a patch with specific or unusual testing requirements to ensure that sufficient additional testing is run to gain confidence in the change being made. For patches that are of an experimental nature (i.e. developer is not sure of functionality, or only wants a limited set of tests to be run just to try something), it is also possible to submit a patch with the fortestonly parameter.
The fortestonly parameter marks that the patch is not intended for landing. Any testlist= specified with Test-Parameters: fortestonly will replace the default tests that will be run, so it is possible to run only a subset of tests, possibly multiple times each. Patches marked with fortestonly will not receive the Verified label from Maloo and cannot be landed. In order to land such a patch, it should be rebased and submitted without the fortestonly keyword once the patch is known to be good, or the commit comment in Gerrit should be edited to remove the fortestonly label which will rebuild and retest the patch and will preserve any reviews that the patch has received).
trivial
The trivial keyword can be used to reduce the testing time (both wall-clock time as well as total test system hours) for patches that do not affect code functionality, such as changes to whitespace, comments, man pages, and test scripts (in conjunction with the testlist keyword to ensure the modified test script is run if it isn't already). Patches marked with trivial in the Test-Parameters: list will only run the review-ldiskfs test session, which currently runs only sanity and lnet-selftest to ensure basic functionality. If changes are only being made to a test script outside of those already run by default, an additional testlist=<test-script(s)> keyword should be added with a comma-separated list of modified tests to ensure they are run to validate the changes. The trivial keyword will reduce the test completion time from approximately 10h elapsed and 30h of total test system time to approximately 3h elapsed/system time. This saves test system resources that may be better spent on other patches.
Permissible Test-Parameters: options
Name | Description | Valid Values |
|---|---|---|
trivial | Patch contains only whitespace, comment, man page, or test script (with additional testlist=) updates and requires minimal testing. Currently, only review-ldiskfs is run for trivial patches (see Autotest Plan for details). The Verified flag will be set upon successful test completion(s). | |
| Test group to test with |
|
| Comma separated list of test names to run in place of a standard test group |
|
| Patch will be built by Jenkins. A single review test session will be run by default, unless another specific test list is specified using the |
|
forbuildonly | Patch will be built by Jenkins, but no testing will be done. The Verified flag will not be set by Maloo for landing. | |
| Number of clients to use | 2-4 |
| Number of MDSs to test against | 1-4 |
| Number of MDTs to test against | 1-4 |
| Number of OSTs to test against | 1-4 |
| Number of OSTs per OSS to test against | 1-4 |
| Cobbler profile to use for clients | test |
| Cobbler profile to use for servers | test |
| Distribution to use for clients |
|
| Distribution to use for servers |
|
| Architecture to use for clients |
|
| Architecture to use for servers |
|
| Jenkins 'job' to fetch from for the clients under test | Any valid Jenkins job, such as |
| Jenkins build number to fetch for the clients under test | Any valid Jenkins buildno for |
| Jenkins 'job' to fetch from for the servers under test | Any valid Jenkins job, such as |
| Jenkins build number to fetch for the servers under test | Any valid Jenkins build number for |
| Network types to use as a comma separated array |
|
| Size of OSTs in GB. Note this is a max size and will be limited by the disk available | >0 |
| Size of MDS in GB. Note this is a max size and will be limited by the disk available | >0 |
| File system to use for MDSs |
|
| File system to use for MDTs |
|
| File system to use for OSTs |
|
| IB stack to use on the client. |
|
| IB stack to use on the server. |
|
| Comma separated environment definitions passed to the environment. Be very careful setting environment variables directly (for example |
|
alwaysuploadlogs | By default autotest will upload logs on passing tests. This will force it do upload logs. This option is no longer necessary. | |
failover | Setup cluster in failover configuration | |
iscsi | Use iSCSI for failover testing | 0, 1 |
combinedmdsmgs | Use a combined server storage target for the MDS and MGS or use separate targets for the MDS and MGS. If false, the MGS and MDS will be on the same node, but the MDT and MGT will be separate storage targets. Default value is true; a combined storage target. | true, false |
You do not need to specify all the values only those values important for your requirements, the test system will use your request to alter a regular test. Also to ensure that you all reviews are fully tested to a known standard. The test system will run a regular, unmodified test set as well as the special request.
Here is an example:
LU-1234 recovery: handle swabbing during recover
Handle byte swabbing of requests properly during recovery. There were
problems with the handling of replayed creates that sent the requests
with client-endian order but a little-endian LOV EA.
Test-Parameters: clientdistro=el5 clientarch=ppc64 serverarch=x86_64 \
ostcount=10 nettype=tcp testlist=conf-sanity,replay-single
Test-Parameters: clientdistro=sles11 clientarch=ppc64 serverdistro=el6 \
nettype=o2ib testlist=conf-sanity,replay-single
Signed-off-by: Random J Developer <random@developer.example.org>
Change-Id: Ica9ed1612eab0c4673dee088f8b441d806c64932
Complex Examples
Test a review for failover
Test-Parameters: fortestonly envdefinitions="SLOW=yes" clientcount=4 osscount=2 mdscount=2 austeroptions=-R failover=true iscsi=1 testgroup=failover
Overview
Content Tools