summaryrefslogtreecommitdiffstats
path: root/tools/regression/README
diff options
context:
space:
mode:
Diffstat (limited to 'tools/regression/README')
-rw-r--r--tools/regression/README57
1 files changed, 52 insertions, 5 deletions
diff --git a/tools/regression/README b/tools/regression/README
index 0c98abb..3bc4c8e 100644
--- a/tools/regression/README
+++ b/tools/regression/README
@@ -5,16 +5,63 @@ This directory is for regression test programs.
A regression test program is one that will exercise a particular bit of the
system to check that we have not reintroduced an old bug.
+Tests should be implemented in files with a .t extension. Each .t file
+can contain more than one test, and can be implemented in any scripting
+language -- /bin/sh, Perl...
+
+The test protocol is quite simple. At its most basic, each .t file should,
+when run, print a line in this format:
+
+ 1..m
+
+where m is the number of tests that will be run.
+
+Each test should produce a single line of output. This line should start
+with one of
+
+ ok n
+ not ok n
+
+to indicate whether or not the test succeeded. 'n' is the test's number.
+Anything after this on the line (up to the first '#' if present) is
+considered to be the name of the test. Naming tests is optional, but
+encouraged.
+
+A test may be written which is conditional, and may need to be skipped.
+For example, the netatalk tests require 'options NETATALK' in the kernel.
+A test may be skipped by printing '# skip Reason for skipping' after the
+test name. For example,
+
+ ok 1 - netatalk # skip 'options NETATALK' not compiled in
+
+A test may be flagged as 'todo'. This indicates that you expect the test
+to fail (perhaps because the necessary functionality hasn't been written
+yet). 'todo' tests are expected to fail, so when they start working the
+test framework can alert you to this happy occurence. Flag these tests
+with a '# TODO' comment after the test name
+
+ not ok 1 - infiniteloop # TODO write test for an infinite loop
+
+This is modelled on the protocol followed by the Test::Harness Perl
+module (and therefore much of the automated testing carried out by the
+Perl community). More documentation can be found at:
+
+ http://search.cpan.org/~petdance/Test-Harness-2.42/lib/Test/Harness.pm
+
+To run the tests and parse their output install the devel/p5-Test-Harness
+port. This includes the prove(1) command which is used to run the tests
+and collate the output.
+
+ prove geom_concat # run all the tests in geom_concat
+ prove -r lib # run all tests in lib/, and subdirectories
+ prove -r -v lib # as above, with verbose output
+ prove -r # run *all* the tests
+
Tests that are for parts of the base system should go into a directory here
which is the same as their path relative to src/, for example the uuencode(1)
utility resides in src/usr.bin/uuencode so its regression test resides in
src/tools/regression/usr.bin/uuencode.
-Base system regression tests should prefix their test results with 'PASS' or
-'FAIL' accordingly, to make checking for failed tests easier. For now the
-tests are very free-form, and it's up to you to do that. Eventually, it
-should be a part of the regression test framework.
-
Please make a subdir per other regression test, and add a brief description to
this file.
OpenPOWER on IntegriCloud