diff options
Diffstat (limited to 'tools/regression/README')
-rw-r--r-- | tools/regression/README | 57 |
1 files changed, 52 insertions, 5 deletions
diff --git a/tools/regression/README b/tools/regression/README index 0c98abb..3bc4c8e 100644 --- a/tools/regression/README +++ b/tools/regression/README @@ -5,16 +5,63 @@ This directory is for regression test programs. A regression test program is one that will exercise a particular bit of the system to check that we have not reintroduced an old bug. +Tests should be implemented in files with a .t extension. Each .t file +can contain more than one test, and can be implemented in any scripting +language -- /bin/sh, Perl... + +The test protocol is quite simple. At its most basic, each .t file should, +when run, print a line in this format: + + 1..m + +where m is the number of tests that will be run. + +Each test should produce a single line of output. This line should start +with one of + + ok n + not ok n + +to indicate whether or not the test succeeded. 'n' is the test's number. +Anything after this on the line (up to the first '#' if present) is +considered to be the name of the test. Naming tests is optional, but +encouraged. + +A test may be written which is conditional, and may need to be skipped. +For example, the netatalk tests require 'options NETATALK' in the kernel. +A test may be skipped by printing '# skip Reason for skipping' after the +test name. For example, + + ok 1 - netatalk # skip 'options NETATALK' not compiled in + +A test may be flagged as 'todo'. This indicates that you expect the test +to fail (perhaps because the necessary functionality hasn't been written +yet). 'todo' tests are expected to fail, so when they start working the +test framework can alert you to this happy occurence. Flag these tests +with a '# TODO' comment after the test name + + not ok 1 - infiniteloop # TODO write test for an infinite loop + +This is modelled on the protocol followed by the Test::Harness Perl +module (and therefore much of the automated testing carried out by the +Perl community). More documentation can be found at: + + http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm + +To run the tests and parse their output install the devel/p5-Test-Harness +port. This includes the prove(1) command which is used to run the tests +and collate the output. + + prove geom_concat # run all the tests in geom_concat + prove -r lib # run all tests in lib/, and subdirectories + prove -r -v lib # as above, with verbose output + prove -r # run *all* the tests + Tests that are for parts of the base system should go into a directory here which is the same as their path relative to src/, for example the uuencode(1) utility resides in src/usr.bin/uuencode so its regression test resides in src/tools/regression/usr.bin/uuencode. -Base system regression tests should prefix their test results with 'PASS' or -'FAIL' accordingly, to make checking for failed tests easier. For now the -tests are very free-form, and it's up to you to do that. Eventually, it -should be a part of the regression test framework. - Please make a subdir per other regression test, and add a brief description to this file. |