diff options
| -rw-r--r-- | doc/fate.txt | 118 |
1 files changed, 105 insertions, 13 deletions
diff --git a/doc/fate.txt b/doc/fate.txt index ddcbdcd..f57000f 100644 --- a/doc/fate.txt +++ b/doc/fate.txt @@ -1,23 +1,115 @@ FATE Automated Testing Environment +================================== -FATE provides a regression testsuite that can be run locally or configured -to send reports to fate.ffmpeg.org. -In order to run, it needs a large amount of data (samples and references) -that is provided separately from the actual source distribution. + FATE is an extended regression suite on the client-side and a means +for results aggregation and presentation on the server-side. -Use the following command to get the fate test samples + The first part of this document explains how you can use FATE from +your FFmpeg source directory to test your ffmpeg binary. The second +part describes how you can run FATE to submit the results to FFmpeg's +FATE server. + + In any way you can have a look at the publicly viewable FATE results +by visiting this website: + + http://fate.ffmpeg.org/ + + This is especially recommended for all people contributing source +code to FFmpeg, as it can be seen if some test on some platform broke +with there recent contribution. This usually happens on the platforms +the developers could not test on. + + The second part of this document describes how you can run FATE to +submit your results to FFmpeg's FATE server. If you want to submit your +results be sure to check that your combination of CPU, OS and compiler +is not already listed on the above mentioned website. + + In the third part you can find a comprehensive listing of FATE makefile +targets and variables. + + +1. Using FATE from your FFmpeg source directory +----------------------------------------------- + + If you want to run FATE on your machine you need to have the samples +in place. You can get the samples via the build target fate-rsync. +Use this command from the top-level source directory: # make fate-rsync SAMPLES=fate-suite/ +# make fate SAMPLES=fate-suite/ + + The above commands set the samples location by passing a makefile +variable via command line. It is also possible to set the samples +location at source configuration time by invoking configure with +`--samples=<path to the samples directory>'. Afterwards you can +invoke the makefile targets without setting the SAMPLES makefile +variable. This is illustrated by the following commands: + +# ./configure --samples=fate-suite/ +# make fate-rsync +# make fate -To inform the build system about the testsuite location, pass -`--samples=<path to the samples>` to configure or set the SAMPLES Make -variable or the FATE_SAMPLES environment variable to a suitable value. -NOTE: Do not put a '~' character in the samples path to indicate a home + Yet another way to tell FATE about the location of the sample +directory is by making sure the environment variable FATE_SAMPLES +contains the path to your samples directory. This can be achieved +by e.g. putting that variable in your shell profile or by setting +it in your interactive session. + +# FATE_SAMPLES=fate-suite/ make fate + +NOTE: +Do not put a '~' character in the samples path to indicate a home directory. Because of shell nuances, this will cause FATE to fail. -For information on how to set up FATE to send results to the official FFmpeg -testing framework, please refer to the following wiki page: -http://wiki.multimedia.cx/index.php?title=FATE + +2. Submitting the results to the FFmpeg result aggregation server +----------------------------------------------------------------- + + To submit your results to the server you should run fate through the +shell script tests/fate.sh from the FFmpeg sources. This script needs +to be invoked with a configuration file as its first argument. + +# tests/fate.sh /path/to/fate_config + + A configuration file template with comments describing the individual +configuration variables can be found at tests/fate_config.sh.template . + + Create a configuration that suits your needs, based on the configuration +template. The `slot' configuration variable can be any string that is not +yet used, but it is suggested that you name it adhering to the following +pattern <arch>-<os>-<compiler>-<compiler version>. The configuration file +itself will be sourced in a shell script, therefore all shell features may +be used. This enables you to setup the environment as you need it for your +build. + + For your first test runs the `fate_recv' variable should be empty or +commented out. This will run everything as normal except that it will omit +the submission of the results to the server. The following files should be +present in $workdir as specified in the configuration file: + + - configure.log + - compile.log + - test.log + - report + - version + + When you have everything working properly you can create an SSH key and +send its public part to the FATE server administrator. + + Configure your SSH client to use public key authentication with that key +when connecting to the FATE server. Also do not forget to check the identity +of the server and to accept its host key. This can usually be achieved by +running your SSH client manually and killing it after you accepted the key. +The FATE server's fingerprint is: + + b1:31:c8:79:3f:04:1d:f8:f2:23:26:5a:fd:55:fa:92 + + The only thing left is to automate the execution of the fate.sh script and +the synchronisation of the samples directory. + + +3. FATE makefile targets and variables +-------------------------------------- FATE Makefile targets: @@ -27,7 +119,7 @@ fate-list fate Run the FATE test suite (requires the fate-suite dataset). -Fate Makefile variables: +FATE Makefile variables: V Verbosity level, can be set to 0, 1 or 2. |
