bitbake: user-manual-fetching.xml: Re-write of the Fetching chapter.

Based on a Richard Purdie re-write.

(Bitbake rev: fad9a6258f8c04bbe0168e46898dd27b86c39ee0)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 436 insertions, 146 deletions

diff --git a/bitbake/doc/user-manual/user-manual-fetching.xml b/bitbake/doc/user-manual/user-manual-fetching.xml
index 8469684..87951fd 100644
--- a/bitbake/doc/user-manual/user-manual-fetching.xml
+++ b/bitbake/doc/user-manual/user-manual-fetching.xml
@@ -5,57 +5,112 @@
 <title>File Download Support</title>
 
     <para>
-        BitBake's <filename>fetch</filename> and
-        <filename>fetch2</filename> modules support downloading
-        files.
-        This chapter provides an overview of the fetching process
-        and also presents sections on each of the fetchers BitBake
-        supports.
-        <note>
-            The original <filename>fetch</filename> code, for all
-            practical purposes, has been replaced by
-            <filename>fetch2</filename> code.
-            Consequently, the information in this chapter does not
-            apply to <filename>fetch</filename>.
-        </note>
+        BitBake's fetch module is a standalone piece of library code
+        that deals with the intricacies of downloading source code
+        and files from remote systems.
+        Fetching source code is one of the corner stones of building software.
+        As such, this module forms an important part of BitBake.
     </para>
 
-    <section id='file-download-overview'>
-        <title>Overview</title>
+    <para>
+        The current fetch module is called "fetch2" and refers to the
+        fact that it is the second major version of the API.
+        The original version is obsolete and removed from the codebase.
+        Thus, in all cases, "fetch" refers to "fetch2" in this
+        manual.
+    </para>
+
+    <section id='the-download-fetch'>
+        <title>The Download (Fetch)</title>
+
+        <para>
+            BitBake takes several steps when fetching source code or files.
+            The fetcher codebase deals with two distinct processes in order:
+            obtaining the files from somewhere (cached or otherwise)
+            and then unpacking those files into a specific location and
+            perhaps in a specific way.
+            Getting and unpacking the files is often optionally followed
+            by patching.
+            Patching, however, is not covered by the fetch.
+        </para>
 
         <para>
-            When BitBake starts to execute, the very first thing
-            it does is to fetch the source files needed.
-            This section overviews the process.
+            The code to execute the first part of this process, a fetch,
+            looks something like the following:
+            <literallayout class='monospaced'>
+     src_uri = (d.getVar('SRC_URI', True) or "").split()
+     fetcher = bb.fetch2.Fetch(src_uri, d)
+     fetcher.download()
+            </literallayout>
+            This code sets up an instance of the fetch module.
+            The instance uses a space-separated list of URLs from the
+            <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
+            variable and then calls the <filename>download</filename>
+            method to download the files.
         </para>
 
         <para>
-            When BitBake goes looking for source files, it follows a search
-            order:
-            <orderedlist>
+            The instance of the fetch module is usually followed by:
+            <literallayout class='monospaced'>
+     rootdir = l.getVar('WORKDIR', True)
+     fetcher.unpack(rootdir)
+            </literallayout>
+            This code unpacks the downloaded files to the
+            specified by <filename>WORKDIR</filename>.
+            <note>
+                For convenience, the naming in these examples matches
+                the variables used by OpenEmbedded.
+            </note>
+            The <filename>SRC_URI</filename> and <filename>WORKDIR</filename>
+            variables are not coded into the fetcher.
+            They variables can (and are) called with different variable names.
+            In OpenEmbedded for example, the shared state (sstate) code uses
+            the fetch module to fetch the sstate files.
+        </para>
+
+        <para>
+            When the <filename>download()</filename> method is called,
+            BitBake tries to fulfill the URLs by looking for source files
+            in a specific search order:
+            <itemizedlist>
                 <listitem><para><emphasis>Pre-mirror Sites:</emphasis>
-                    BitBake first uses pre-mirrors to try and find source
-                    files.
+                    BitBake first uses pre-mirrors to try and find source files.
                     These locations are defined using the
                     <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
                     variable.
                     </para></listitem>
                 <listitem><para><emphasis>Source URI:</emphasis>
-                    If pre-mirrors fail, BitBake uses
-                    <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>.
+                    If pre-mirrors fail, BitBake uses the original URL (e.g from
+                    <filename>SRC_URI</filename>).
                     </para></listitem>
                 <listitem><para><emphasis>Mirror Sites:</emphasis>
-                    If fetch failures occur using <filename>SRC_URI</filename>,
-                    BitBake next uses mirror location as defined by the
+                    If fetch failures occur, BitBake next uses mirror location as
+                    defined by the
                     <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
                     variable.
                     </para></listitem>
-            </orderedlist>
+            </itemizedlist>
+        </para>
+
+        <para>
+            For each URL passed to the fetcher, the fetcher
+            calls the submodule that handles that particular URL type.
+            This behavior can be the source of some confusion when you
+            are providing URLs for the <filename>SRC_URI</filename>
+            variable.
+            Consider the following two URLs:
+            <literallayout class='monospaced'>
+     http://git.yoctoproject.org/git/poky;protocol=git
+     git://git.yoctoproject.org/git/poky;protocol=http
+            </literallayout>
+            In the former case, the URL is passed to the
+            <filename>wget</filename> fetcher, which does not
+            understand "git".
+            Therefore, the latter case is the correct form since the
+            Git fetcher does know how to use HTTP as a transport.
         </para>
 
         <para>
-            Because cross-URLs are supported, it is possible to mirror
-            a Git repository on an HTTP server as a tarball.
             Here are some examples that show commonly used mirror
             definitions:
             <literallayout class='monospaced'>
@@ -74,19 +129,29 @@
          http://.*/.*     http://somemirror.org/sources/ \n \
          https://.*/.*    http://somemirror.org/sources/ \n"
             </literallayout>
+            It is useful to note that BitBake supports
+            cross-URLs.
+            It is possible to mirror a Git repository on an HTTP
+            server as a tarball.
+            This is what the <filename>git://</filename> mapping in
+            the previous example does.
         </para>
 
         <para>
-            Any source files that are not local (i.e. downloaded from
-            the Internet) are placed into the download directory,
-            which is specified by
-            <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>.
+            Since network accesses are slow, Bitbake maintains a
+            cache of files downloaded from the network.
+            Any source files that are not local (i.e.
+            downloaded from the Internet) are placed into the download
+            directory, which is specified by the
+            <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
+            variable.
         </para>
 
         <para>
+            File integrity is of key importance for reproducing builds.
             For non-local archive downloads, the fetcher code can verify
-            sha256 and md5 checksums to ensure
-            the archives have been downloaded correctly.
+            sha256 and md5 checksums to ensure the archives have been
+            downloaded correctly.
             You can specify these checksums by using the
             <filename>SRC_URI</filename> variable with the appropriate
             varflags as follows:
@@ -97,66 +162,133 @@
             You can also specify the checksums as parameters on the
             <filename>SRC_URI</filename> as shown below:
             <literallayout class='monospaced'>
-     SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
+     SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07f
+db994d"
             </literallayout>
-            If
-            <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link>
-            is set, any download without a checksum triggers an error message.
-            In cases where multiple files are listed using
-            <filename>SRC_URI</filename>, the name parameter is used
-            assign names to the URLs and these are then specified
-            in the checksums using the following form:
+            If multiple URIs exist, you can specify the checksums either
+            directly as in the previous example, or you can name the URLs.
+            The following syntax shows how you name the URIs:
             <literallayout class='monospaced'>
-     SRC_URI[name.sha256sum]
+     SRC_URI = "http://example.com/foobar.tar.bz2;name=foo"
+     SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d
             </literallayout>
+            After a file has been downloaded and has had its checksum checked,
+            a ".done" stamp is placed in <filename>DL_DIR</filename>.
+            BitBake uses this stamp during subsequent builds to avoid
+            downloading or comparing a checksum for the file again.
+            <note>
+                It is assumed that local storage is safe from data corruption.
+                If this were not the case, there would be bigger issues to worry about.
+            </note>
+        </para>
+
+        <para>
+            If
+            <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link>
+            is set, any download without a checksum triggers an
+            error message.
+            The
+            <link linkend='var-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></link>
+            variable can be used to make any attempted network access a fatal
+            error, which is useful for checking that mirrors are complete
+            as well as other things.
         </para>
     </section>
 
-    <section id='bb-fetchers'>
-        <title>Fetchers</title>
+    <section id='bb-the-unpack'>
+        <title>The Unpack</title>
+
+        <para>
+            The unpack process usually immediately follows the download.
+            For all URLs except Git URLs, BitBake uses the common
+            <filename>unpack</filename> method.
+        </para>
 
         <para>
-            As mentioned in the previous section, the
-            <filename>SRC_URI</filename> is normally used to
-            tell BitBake which files to fetch.
-            And, the fetcher BitBake uses depends on the how
-            <filename>SRC_URI</filename> is set.
+            A number of parameters exist that you can specify within the
+            URL to govern the behavior of the unpack stage:
+            <itemizedlist>
+                <listitem><para><emphasis>unpack:</emphasis>
+                    Controls whether the URL components are unpacked.
+                    If set to "1", which is the default, the components
+                    are unpacked.
+                    If set to "0", the unpack stage leaves the file alone.
+                    This parameter is useful when you want an archive to be
+                    copied in and not be unpacked.
+                    </para></listitem>
+                <listitem><para><emphasis>dos:</emphasis>
+                    Applies to <filename>.zip</filename> and
+                    <filename>.jar</filename> files and specifies whether to
+                    use DOS line ending conversion on text files.
+                    </para></listitem>
+                <listitem><para><emphasis>basepath:</emphasis>
+                    Instructs the unpack stage to strip the specified
+                    directories from the source path when unpacking.
+                    </para></listitem>
+                <listitem><para><emphasis>subdir:</emphasis>
+                    Unpacks the specific URL to the specified subdirectory
+                    within the root directory.
+                    </para></listitem>
+            </itemizedlist>
+            The unpack call automatically decompresses and extracts files
+            with ".Z", ".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm".
+            ".srpm", ".deb" and ".bz2" extensions as well as various combinations
+            of tarball extensions.
         </para>
 
         <para>
-            These next few sections describe the available fetchers and
-            their options.
-            Each fetcher honors a set of variables URI parameters,
-            which are separated by semi-colon characters and consist
-            of a key and a value.
-            The semantics of the variables and parameters are
-            defined by the fetcher.
-            BitBake tries to have consistent semantics between the
-            different fetchers.
+            As mentioned, the Git fetcher has its own unpack method that
+            is optimized to work with Git trees.
+            Basically, this method works by cloning the tree into the final
+            directory.
+            The process is completed using references so that there is
+            only one central copy of the Git metadata needed.
         </para>
+    </section>
 
-        <section id='local-file-fetcher'>
-            <title>Local file fetcher</title>
+    <section id='bb-fetchers'>
+        <title>Fetchers</title>
 
-            <para>
-                The URN for the local file fetcher is file.
-            </para>
+        <para>
+            As mentioned earlier, the URL prefix determines which
+            fetcher submodule BitBake uses.
+            Each submodule can support different URL parameters,
+            which are described in the following sections.
+        </para>
+
+        <section id='local-file-fetcher'>
+            <title>Local file fetcher (<filename>file://</filename>)</title>
 
             <para>
-                The filename can be either absolute or relative.
-                If the filename is relative,
+                This submodule handles URLs that begin with
+                <filename>file://</filename>.
+                The filename you specify with in the URL can
+                either be an absolute or relative path to a file.
+                If the filename is relative, the contents of the
                 <link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>
-                is used.
+                variable is used in the same way
+                <filename>PATH</filename> is used to find executables.
                 Failing that,
                 <link linkend='var-FILESDIR'><filename>FILESDIR</filename></link>
                 is used to find the appropriate relative file.
+                <note>
+                    <filename>FILESDIR</filename> is deprecated and can
+                    be replaced with <filename>FILESPATH</filename>.
+                    Because <filename>FILESDIR</filename> is likely to be
+                    removed, you should not use this variable in any new code.
+                </note>
+                If the file cannot be found, it is assumed that it is available in
+                <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
+                by the time the <filename>download()</filename> method is called.
             </para>
 
             <para>
-                The metadata usually extend these variables to include
-                variations of the values in
-                <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>.
-                Single files and complete directories can be specified.
+                If you specify a directory, the entire directory is
+                unpacked.
+            </para>
+
+            <para>
+                Here are some example URLs:
                 <literallayout class='monospaced'>
      SRC_URI = "file://relativefile.patch"
      SRC_URI = "file://relativefile.patch;this=ignored"
@@ -166,36 +298,53 @@
         </section>
 
         <section id='cvs-fetcher'>
-            <title>CVS fetcher</title>
-
-            <para>
-                The URN for the CVS fetcher is cvs.
-            </para>
+            <title>CVS fetcher (<filename>(cvs://</filename>)</title>
 
             <para>
-                This fetcher honors the variables <filename>CVSDIR</filename>,
-                <filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>,
-                <filename>UPDATECOMMAND_cvs</filename>.
-                The
-                <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
-                variable specifies where a
-                temporary checkout is saved.
-                The
-                <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link>
-                variable specifies which date to
-                use when doing the fetching.
-                The special value of "now" causes the checkout to be
-                updated on every build.
-                The <filename>FETCHCOMMAND</filename> and
-                <filename>UPDATECOMMAND</filename> variables specify the executables
-                to use for the CVS checkout or update.
+                This submodule handles checking out files from the
+                CVS version control system.
+                You can configure it using a number of different variables:
+                <itemizedlist>
+                    <listitem><para><emphasis><filename>FETCHCMD_cvs</filename>:</emphasis>
+                        The name of the executable to use when running
+                        the <filename>cvs</filename> command.
+                        This name is usually "cvs".
+                        </para></listitem>
+                    <listitem><para><emphasis><filename>SRCDATE</filename>:</emphasis>
+                        The date to use when fetching the CVS source code.
+                        A special value of "now" causes the checkout to
+                        be updated on every build.
+                        </para></listitem>
+                    <listitem><para><emphasis><filename>CVSDIR</filename>:</emphasis>
+                        Specifies where a temporary checkout is saved.
+                        The location is often <filename>DL_DIR/cvs</filename>.
+                        </para></listitem>
+                    <listitem><para><emphasis><filename>CVS_PROXY_HOST</filename>:</emphasis>
+                        The name to use as a "proxy=" parameter to the
+                        <filename>cvs</filename> command.
+                        </para></listitem>
+                    <listitem><para><emphasis><filename>CVS_PROXY_PORT</filename>:</emphasis>
+                        The port number to use as a "proxyport=" parameter to
+                        the <filename>cvs</filename> command.
+                        </para></listitem>
+                </itemizedlist>
+                As well as the standard username and password URL syntax,
+                you can also configure the fetcher with various URL parameters:
             </para>
 
             <para>
                 The supported parameters are as follows:
                 <itemizedlist>
+                    <listitem><para><emphasis>"method":</emphasis>
+                        The protocol over which to communicate with the cvs server.
+                        By default, this protocol is "pserver".
+                        If "method" is set to "ext", BitBake examines the
+                        "rsh" parameter and sets <filename>CVS_RSH</filename>.
+                        You can use "dir" for local directories.
+                        </para></listitem>
                     <listitem><para><emphasis>"module":</emphasis>
                         Specifies the module to check out.
+                        You must supply this parameter.
                         </para></listitem>
                     <listitem><para><emphasis>"tag":</emphasis>
                         Describes which CVS TAG should be used for
@@ -210,23 +359,36 @@
                         The special value of "now" causes the checkout to be
                         updated on every build.
                         </para></listitem>
-                    <listitem><para><emphasis>"method":</emphasis>
-                        By default <filename>pserver</filename>.
-                        If "method" is set to "ext", BitBake examines the "rsh"
-                        parameter and sets <filename>CVS_RSH</filename>.
-                        </para></listitem>
                     <listitem><para><emphasis>"localdir":</emphasis>
-                        Used to checkout force into a special
+                        Used to rename the module.
+                        Effectively, you are renaming the output directory
+                        to which the module is unpacked.
+                        You are forcing the module into a special
                         directory relative to <filename>CVSDIR</filename>.
                         </para></listitem>
                     <listitem><para><emphasis>"rsh"</emphasis>
                         Used in conjunction with the "method" parameter.
                         </para></listitem>
                     <listitem><para><emphasis>"scmdata":</emphasis>
-                        I need a description for this.
+                        Causes the CVS metadata to be maintained in the tarball
+                        the fetcher creates when set to "keep".
+                        The tarball is expanded into the work directory.
+                        By default, the CVS metadata is removed.
+                        </para></listitem>
+                    <listitem><para><emphasis>"fullpath":</emphasis>
+                        Controls whether the resulting checkout is at the
+                        module level, which is the default, or is at deeper
+                        paths.
+                        </para></listitem>
+                    <listitem><para><emphasis>"norecurse":</emphasis>
+                        Causes the fetcher to only checkout the specified
+                        directory with no recurse into any subdirectories.
+                        </para></listitem>
+                    <listitem><para><emphasis>"port":</emphasis>
+                        The port to which the CVS server connects.
                         </para></listitem>
                 </itemizedlist>
-                Following are two examples using cvs:
+                Some example URLs are as follows:
                 <literallayout class='monospaced'>
      SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
      SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
@@ -235,19 +397,27 @@
         </section>
 
         <section id='http-ftp-fetcher'>
-            <title>HTTP/FTP fetcher</title>
+            <title>HTTP/FTP wget fetcher (<filename>http://</filename>, <filename>ftp://</filename>, <filename>https://</filename>)</title>
 
             <para>
-                The URNs for the HTTP/FTP fetcher are http, https, and ftp.
+                This fetcher obtains files from web and FTP servers.
+                Internally, the fetcher uses the wget utility.
             </para>
 
             <para>
-                This fetcher honors the variables
-                <filename>FETCHCOMMAND_wget</filename>.
-                The <filename>FETCHCOMMAND</filename> variable
-                contains the command used for fetching.
-                “${URI}” and “${FILES}” are replaced by the URI and
-                the base name of the file to be fetched.
+                The executable and parameters used are specified by the
+                <filename>FETCHCMD_wget</filename> variable, which defaults
+                to a sensible values.
+                The fetcher supports a parameter "downloadfilename" that
+                allows the name of the downloaded file to be specified.
+                Specifying the name of the downloaded file is useful
+                for avoiding collisions in
+                <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
+                when dealing with multiple files that have the same name.
+            </para>
+
+            <para>
+                Some example URLs are as follows:
                 <literallayout class='monospaced'>
      SRC_URI = "http://oe.handhelds.org/not_there.aac"
      SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac"
@@ -257,36 +427,46 @@
         </section>
 
         <section id='svn-fetcher'>
-            <title>SVN Fetcher</title>
+            <title>Subversion (SVN) Fetcher (<filename>svn://</filename>)</title>
 
             <para>
-                The URN for the SVN fetcher is svn.
-            </para>
-
-            <para>
-                This fetcher honors the variables
-                <filename>FETCHCOMMAND_svn</filename>,
-                <filename>SVNDIR</filename>,
-                and
-                <link linkend='var-SRCREV'><filename>SRCREV</filename></link>.
-                The <filename>FETCHCOMMAND</filename> variable contains the
-                <filename>subversion</filename> command.
-                The <filename>SRCREV</filename> variable specifies which revision
-                to use when doing the fetching.
+                This fetcher submodule fetches code from the
+                Subversion source control system.
+                The executable used is specified by
+                <filename>FETCHCMD_svn</filename>, which defaults
+                to "svn".
+                The fetcher's temporary working directory is set
+                by <filename>SVNDIR</filename>, which is usually
+                <filename>DL_DIR/svn</filename>.
             </para>
 
             <para>
                 The supported parameters are as follows:
                 <itemizedlist>
-                    <listitem><para><emphasis>"proto":</emphasis>
-                        The Subversion protocol.
+                    <listitem><para><emphasis>"module":</emphasis>
+                        The name of the svn module to checkout.
+                        You must provide this parameter.
+                        You can think of this parameter as the top-level
+                        directory of the repository data you want.
+                        </para></listitem>
+                    <listitem><para><emphasis>"protocol":</emphasis>
+                        The protocol to use, which defaults to "svn".
+                        Other options are "svn+ssh" and "rsh".
+                        For "rsh", the "rsh" parameter is also used.
                         </para></listitem>
                     <listitem><para><emphasis>"rev":</emphasis>
-                        The Subversion revision.
+                        The revision of the source code to checkout.
+                        </para></listitem>
+                    <listitem><para><emphasis>"date":</emphasis>
+                        The date of the source code to checkout.
+                        Specific revisions are generally much safer to checkout
+                        rather than by date as they do not involve timezones
+                        (e.g. they are much more deterministic).
                         </para></listitem>
                     <listitem><para><emphasis>"scmdata":</emphasis>
-                        Set to "keep" causes the “.svn” directories
-                        to be available during compile-time.
+                        Causes the “.svn” directories to be available during
+                        compile-time when set to "keep".
+                        By default, these directories are removed.
                         </para></listitem>
                 </itemizedlist>
                 Following are two examples using svn:
@@ -298,40 +478,150 @@
         </section>
 
         <section id='git-fetcher'>
-            <title>GIT Fetcher</title>
-
-            <para>
-                The URN for the Git Fetcher is git.
-            </para>
+            <title>GIT Fetcher (<filename>git://</filename>)</title>
 
             <para>
-                The variable <filename>GITDIR</filename> is used as the
-                base directory in which the Git tree is cloned.
+                This fetcher submodule fetches code from the Git
+                source control system.
+                The fetcher works by creating a bare clone of the
+                remote into <filename>GITDIR</filename>, which is
+                usually <filename>DL_DIR/git</filename>.
+                This bare clone is then cloned into the work directory during the
+                unpack stage when a specific tree is checked out.
+                This is done using alternates and by reference to
+                minimize the amount of duplicate data on the disk and
+                make the unpack process fast.
+                The executable used can be set with
+                <filename>FETCHCMD_git</filename>.
             </para>
 
             <para>
-                The supported parameters are as follows:
+                This fetcher supports the following parameters:
                 <itemizedlist>
-                    <listitem><para><emphasis>"tag":</emphasis>
-                        The Git tag.
-                        The default is "master".
-                        </para></listitem>
                     <listitem><para><emphasis>"protocol":</emphasis>
-                        The Git protocol.
+                        The protocol used to fetch the files.
                         The default is "git" when a hostname is set.
                         If a hostname is not set, the Git protocol is "file".
+                        You can also use "http", "https", "ssh" and "rsync".
                         </para></listitem>
-                    <listitem><para><emphasis>"scmdata":</emphasis>
-                        When set to “keep”, the “.git” directory is available
-                        during compile-time.
+                    <listitem><para><emphasis>"nocheckout":</emphasis>
+                        Tells the fetcher to not checkout source code when
+                        unpacking when set to "1".
+                        Set this option for the URL where there is a custom
+                        routine to checkout code.
+                        The default is "0".
+                        </para></listitem>
+                    <listitem><para><emphasis>"rebaseable":</emphasis>
+                        Indicates that the upstream Git repository can be rebased.
+                        You should set this parameter to "1" if
+                        revisions can become detached from branches.
+                        In this case, the source mirror tarball is done per
+                        revision, which has a loss of efficiency.
+                        Rebasing the upstream Git repository could cause the
+                        current revision to disappear from the upstream repository.
+                        This option reminds the fetcher to preserve the local cache
+                        carefully for future use.
+                        The default value for this parameter is "0".
+                        </para></listitem>
+                    <listitem><para><emphasis>"nobranch":</emphasis>
+                        Tells the fetcher to not check the SHA validation
+                        for the branch when set to "1".
+                        The default is "0".
+                        Set this option for the recipe that refers to
+                        the commit that is valid for a tag instead of
+                        the branch.
+                        </para></listitem>
+                    <listitem><para><emphasis>"bareclone":</emphasis>
+                        Tells the fetcher to clone a bare clone into the
+                        destination directory without checking out a working tree.
+                        Only the raw Git metadata is provided.
+                        This parameter implies the "nocheckout" parameter as well.
+                        </para></listitem>
+                    <listitem><para><emphasis>"branch":</emphasis>
+                        The branch(es) of the Git tree to clone.
+                        If unset, this is assumed to be "master".
+                        The number of branch parameters much match the number of
+                        name parameters.
+                        </para></listitem>
+                    <listitem><para><emphasis>"rev":</emphasis>
+                        The revision to use for the checkout.
+                        The default is "master".
+                        </para></listitem>
+                    <listitem><para><emphasis>"tag":</emphasis>
+                        Specifies a tag to use for the checkout.
+                        To correctly resolve tags, BitBake must access the
+                        network.
+                        For that reason, tags are often not used.
+                        As far as Git is concerned, the "tag" parameter behaves
+                        effectively the same as the "revision" parameter.
+                        </para></listitem>
+                    <listitem><para><emphasis>"subpath":</emphasis>
+                        Limits the checkout to a specific subpath of the tree.
+                        By default, the whole tree is checked out.
+                        </para></listitem>
+                    <listitem><para><emphasis>"destsuffix":</emphasis>
+                        The name of the path in which to place the checkout.
+                        By default, the path is <filename>git/</filename>.
                         </para></listitem>
                 </itemizedlist>
-                Following are two examples using git:
+                Here are some example URLs:
                 <literallayout class='monospaced'>
      SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1"
      SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"
                 </literallayout>
             </para>
         </section>
+
+        <section id='other-fetchers'>
+            <title>Other Fetchers</title>
+
+            <para>
+                Fetch submodules also exist for the following:
+                <itemizedlist>
+                    <listitem><para>
+                        Bazzar (<filename>bzr://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Perforce (<filename>p4://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        SVK
+                        </para></listitem>
+                    <listitem><para>
+                        Git Submodules (<filename>gitsm://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Trees using Git Annex (<filename>gitannex://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Secure FTP (<filename>sftp://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Secure Shell (<filename>ssh://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Repo (<filename>repo://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        OSC (<filename>osc://</filename>)
+                        </para></listitem>
+                    <listitem><para>
+                        Mercurial (<filename>hg://</filename>)
+                        </para></listitem>
+                </itemizedlist>
+                No documentation currently exists for these lesser used
+                fetcher submodules.
+                However, you might find the code helpful and readable.
+            </para>
+        </section>
+    </section>
+
+    <section id='auto-revisions'>
+        <title>Auto Revisions</title>
+
+        <para>
+            We need to document <filename>AUTOREV</filename> and
+            <filename>SRCREV_FORMAT</filename> here.
+        </para>
     </section>
 </chapter>