doc: rewrite FATE documentation to be self-contained

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

1 files changed, 105 insertions, 13 deletions

diff --git a/doc/fate.txt b/doc/fate.txt
index ddcbdcd..f57000f 100644
--- a/doc/fate.txt
+++ b/doc/fate.txt
@@ -1,23 +1,115 @@
 FATE Automated Testing Environment
+==================================
 
-FATE provides a regression testsuite that can be run locally or configured
-to send reports to fate.ffmpeg.org.
-In order to run, it needs a large amount of data (samples and references)
-that is provided separately from the actual source distribution.
+  FATE is an extended regression suite on the client-side and a means
+for results aggregation and presentation on the server-side.
 
-Use the following command to get the fate test samples
+  The first part of this document explains how you can use FATE from
+your FFmpeg source directory to test your ffmpeg binary. The second
+part describes how you can run FATE to submit the results to FFmpeg's
+FATE server.
+
+  In any way you can have a look at the publicly viewable FATE results
+by visiting this website:
+
+  http://fate.ffmpeg.org/
+
+  This is especially recommended for all people contributing source
+code to FFmpeg, as it can be seen if some test on some platform broke
+with there recent contribution. This usually happens on the platforms
+the developers could not test on.
+
+  The second part of this document describes how you can run FATE to
+submit your results to FFmpeg's FATE server. If you want to submit your
+results be sure to check that your combination of CPU, OS and compiler
+is not already listed on the above mentioned website.
+
+  In the third part you can find a comprehensive listing of FATE makefile
+targets and variables.
+
+
+1. Using FATE from your FFmpeg source directory
+-----------------------------------------------
+
+  If you want to run FATE on your machine you need to have the samples
+in place. You can get the samples via the build target fate-rsync.
+Use this command from the top-level source directory:
 
 # make fate-rsync SAMPLES=fate-suite/
+# make fate       SAMPLES=fate-suite/
+
+  The above commands set the samples location by passing a makefile
+variable via command line. It is also possible to set the samples
+location at source configuration time by invoking configure with
+`--samples=<path to the samples directory>'. Afterwards you can
+invoke the makefile targets without setting the SAMPLES makefile
+variable. This is illustrated by the following commands:
+
+# ./configure --samples=fate-suite/
+# make fate-rsync
+# make fate
 
-To inform the build system about the testsuite location, pass
-`--samples=<path to the samples>` to configure or set the SAMPLES Make
-variable or the FATE_SAMPLES environment variable to a suitable value.
-NOTE: Do not put a '~' character in the samples path to indicate a home
+  Yet another way to tell FATE about the location of the sample
+directory is by making sure the environment variable FATE_SAMPLES
+contains the path to your samples directory. This can be achieved
+by e.g. putting that variable in your shell profile or by setting
+it in your interactive session.
+
+# FATE_SAMPLES=fate-suite/ make fate
+
+NOTE:
+Do not put a '~' character in the samples path to indicate a home
 directory. Because of shell nuances, this will cause FATE to fail.
 
-For information on how to set up FATE to send results to the official FFmpeg
-testing framework, please refer to the following wiki page:
-http://wiki.multimedia.cx/index.php?title=FATE
+
+2. Submitting the results to the FFmpeg result aggregation server
+-----------------------------------------------------------------
+
+  To submit your results to the server you should run fate through the
+shell script tests/fate.sh from the FFmpeg sources. This script needs
+to be invoked with a configuration file as its first argument.
+
+# tests/fate.sh /path/to/fate_config
+
+  A configuration file template with comments describing the individual
+configuration variables can be found at tests/fate_config.sh.template .
+
+  Create a configuration that suits your needs, based on the configuration
+template. The `slot' configuration variable can be any string that is not
+yet used, but it is suggested that you name it adhering to the following
+pattern <arch>-<os>-<compiler>-<compiler version>. The configuration file
+itself will be sourced in a shell script, therefore all shell features may
+be used. This enables you to setup the environment as you need it for your
+build.
+
+  For your first test runs the `fate_recv' variable should be empty or
+commented out. This will run everything as normal except that it will omit
+the submission of the results to the server. The following files should be
+present in $workdir as specified in the configuration file:
+
+  - configure.log
+  - compile.log
+  - test.log
+  - report
+  - version
+
+  When you have everything working properly you can create an SSH key and
+send its public part to the FATE server administrator.
+
+  Configure your SSH client to use public key authentication with that key
+when connecting to the FATE server. Also do not forget to check the identity
+of the server and to accept its host key. This can usually be achieved by
+running your SSH client manually and killing it after you accepted the key.
+The FATE server's fingerprint is:
+
+  b1:31:c8:79:3f:04:1d:f8:f2:23:26:5a:fd:55:fa:92
+
+  The only thing left is to automate the execution of the fate.sh script and
+the synchronisation of the samples directory.
+
+
+3. FATE makefile targets and variables
+--------------------------------------
 
 FATE Makefile targets:
 
@@ -27,7 +119,7 @@ fate-list
 fate
     Run the FATE test suite (requires the fate-suite dataset).
 
-Fate Makefile variables:
+FATE Makefile variables:
 
 V
     Verbosity level, can be set to 0, 1 or 2.