diff options
author | Scott Rifenbark <scott.m.rifenbark@intel.com> | 2011-12-08 10:34:44 -0800 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2011-12-16 16:58:38 +0000 |
commit | a191e6e04892a67487ace11f5a94ea1c62701c8c (patch) | |
tree | 2746a0d603ca254fa77879e1b264a16ef78fe797 /documentation | |
parent | 9a99147c9146463ec2988e6670ed21929dc474ee (diff) | |
download | ast2050-yocto-poky-a191e6e04892a67487ace11f5a94ea1c62701c8c.zip ast2050-yocto-poky-a191e6e04892a67487ace11f5a94ea1c62701c8c.tar.gz |
documentation/poky-ref-manual: New chapter introduced
Long-term strategy for the YP Reference Manual is that it contains
reference material and not "how-to-information". A step in this
direction is to isolate any discussions on components and other
areas of YP that need talked about. So to start with, I have created
a new chapter for now named "Technical Details" that so far has
a discussion of some components and shared state. This is a
step in the direction of making this manual a reference manual and
not a "how to" manual.
Changes included removing redundant material from the 'usingpoky.xml'
chapter and also adding the new chapter 'technical-details' into the
'poky-ref-manual.xml' file used for the make.
(From yocto-docs rev: 567823392d903016d314ae9fcc1b4d7abf59d9ae)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r-- | documentation/poky-ref-manual/poky-ref-manual.xml | 2 | ||||
-rw-r--r-- | documentation/poky-ref-manual/technical-details.xml | 229 | ||||
-rw-r--r-- | documentation/poky-ref-manual/usingpoky.xml | 220 |
3 files changed, 232 insertions, 219 deletions
diff --git a/documentation/poky-ref-manual/poky-ref-manual.xml b/documentation/poky-ref-manual/poky-ref-manual.xml index 8c81d2b..0190713 100644 --- a/documentation/poky-ref-manual/poky-ref-manual.xml +++ b/documentation/poky-ref-manual/poky-ref-manual.xml @@ -92,6 +92,8 @@ <xi:include href="extendpoky.xml"/> + <xi:include href="technical-details.xml"/> + <xi:include href="../bsp-guide/bsp.xml"/> <xi:include href="development.xml"/> diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml new file mode 100644 index 0000000..8f83ecb --- /dev/null +++ b/documentation/poky-ref-manual/technical-details.xml @@ -0,0 +1,229 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> +<chapter id='technical-details'> +<title>Technical Details</title> + + <para> + This chapter provides technical details for various parts of the Yocto Project. + Currently, topics include Yocto Project components and shared state (sstate) cache. + </para> + +<section id='usingpoky-components'> + <title>Yocto Project Components</title> + + <para> + The BitBake task executor together with various types of configuration files form the + Yocto Project core. + This section overviews the BitBake task executor and the + configuration files by describing what they are used for and how they interact. + </para> + + <para> + BitBake handles the parsing and execution of the data files. + The data itself is of various types: + <itemizedlist> + <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular + pieces of software</para></listitem> + <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build + information (e.g. how to build a Linux kernel).</para></listitem> + <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings, + policy decisions, etc. + Configuration data acts as the glue to bind everything together.</para></listitem> + </itemizedlist> + For more information on data, see the + <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#yocto-project-terms'> + Yocto Project Terms</ulink> section in + <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html'> + The Yocto Project Development Manual</ulink>. + </para> + + <para> + BitBake knows how to combine multiple data sources together and refers to each data source + as a <link linkend='usingpoky-changes-layers'>'layer'</link>. + </para> + + <para> + Following are some brief details on these core components. + For more detailed information on these components see the + <link linkend='ref-structure'>'Reference: Directory Structure'</link> + appendix. + </para> + + <section id='usingpoky-components-bitbake'> + <title>BitBake</title> + + <para> + BitBake is the tool at the heart of the Yocto Project and is responsible + for parsing the metadata, generating a list of tasks from it, + and then executing those tasks. + To see a list of the options BitBake supports, use the following help command: + <literallayout class='monospaced'> + $ bitbake --help + </literallayout> + </para> + + <para> + The most common usage for BitBake is <filename>bitbake <packagename></filename>, where + <filename>packagename</filename> is the name of the package you want to build + (referred to as the "target" in this manual). + The target often equates to the first part of a <filename>.bb</filename> filename. + So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you + might type the following: + <literallayout class='monospaced'> + $ bitbake matchbox-desktop + </literallayout> + Several different versions of <filename>matchbox-desktop</filename> might exist. + BitBake chooses the one selected by the distribution configuration. + You can get more details about how BitBake chooses between different + target versions and providers in the + <link linkend='ref-bitbake-providers'>Preferences and Providers</link> section. + </para> + + <para> + BitBake also tries to execute any dependent tasks first. + So for example, before building <filename>matchbox-desktop</filename>, BitBake + would build a cross compiler and <filename>eglibc</filename> if they had not already + been built. + <note>This release of the Yocto Project does not support the <filename>glibc</filename> + GNU version of the Unix standard C library. By default, the Yocto Project builds with + <filename>eglibc</filename>.</note> + </para> + + <para> + A useful BitBake option to consider is the <filename>-k</filename> or + <filename>--continue</filename> option. + This option instructs BitBake to try and continue processing the job as much + as possible even after encountering an error. + When an error occurs, the target that + failed and those that depend on it cannot be remade. + However, when you use this option other dependencies can still be processed. + </para> + </section> + + <section id='usingpoky-components-metadata'> + <title>Metadata (Recipes)</title> + + <para> + The <filename>.bb</filename> files are usually referred to as "recipes." + In general, a recipe contains information about a single piece of software. + The information includes the location from which to download the source patches + (if any are needed), which special configuration options to apply, + how to compile the source files, and how to package the compiled output. + </para> + + <para> + The term "package" can also be used to describe recipes. + However, since the same word is used for the packaged output from the Yocto + Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), + this document avoids using the term "package" to refer to recipes. + </para> + </section> + + <section id='usingpoky-components-classes'> + <title>Classes</title> + + <para> + Class files (<filename>.bbclass</filename>) contain information that is useful to share + between metadata files. + An example is the Autotools class, which contains + common settings for any application that Autotools uses. + The <link linkend='ref-classes'>Reference: Classes</link> appendix provides details + about common classes and how to use them. + </para> + </section> + + <section id='usingpoky-components-configuration'> + <title>Configuration</title> + + <para> + The configuration files (<filename>.conf</filename>) define various configuration variables + that govern the Yocto Project build process. + These files fall into several areas that define machine configuration options, + distribution configuration options, compiler tuning options, general common configuration + options and user configuration options (<filename>local.conf</filename>, which is found + in the Yocto Project files build directory). + </para> + </section> +</section> + +<section id="considering-shared-state-cache"> + <title>Considering Shared State Cache</title> + + <para> + By design, the Yocto Project builds everything from scratch unless it can determine that + a given task's inputs have not changed. + While building from scratch ensures that everything is current, it does also + mean that a lot of time could be spent rebuiding things that don't necessarily need built. + </para> + + <para> + The Yocto Project build process uses a shared state caching scheme to avoid having to + rebuild software when it is not necessary. + Because the build time for a Yocto image can be significant, it is helpful to try and + determine what really needs built and what can be skipped given a particular project's + build process. + </para> + + <para> + The scheme that the Yocto Project uses involves checksum generation and comparison for + a task's inputs. + The scheme also employs an area of memory called the shared state cache that is + pointed to by the <filename>SSTATE_DIR</filename> variable. + This area contains task output generated from a previous build. + If a given task's checksum matches the checksum of a previous build for the same + task, the build process uses the state of the cache rather than rerunning that + task. + </para> + + <para> + The previous paragraph is a simplistic explanation of how the build process + uses checksums and shared state memory cache to avoide building tasks that + don't need built. + If you want a bit more explanation on the topic, + see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared + State - What does it mean and why should I care?</ulink>" from the Yocto + Project discussion archives. + </para> + + <para> + As with all schemes, this one has some drawbacks. + It is possible that you could make implicit changes that are not factored into the checksum + calculation, but do affect a task's output. + A good example is perhaps when a tool changes its output. + Let's say that the output of <filename>rpmdeps</filename> needed to change. + The result of the change should be that all the "package", "package_write_rpm", + and "package_deploy-rpm" sstate-cache items would become invalid. + But, because this is a change that is external to the code and therefore implicit, + the associated sstate-cache items do not become invalidated. + In this case, the build process would use the cache items rather than running the + task again. + Obviously, these types of implicit changes can cause problems. + </para> + + <para> + To avoid these problems during the build, you need to understand the effects of any + change you make. + Note that any changes you make directly to a function automatically are factored into + the checksum calculation and thus, will invalidate the associated area of sstate cache. + You need to be aware of any implicit changes that are not obvious changes to the + code and could affect the output of a given task. + Once you are aware of such a change, you can take steps to invalidate the cache + and force the task to run. + The step to take is as simple as changing a function's comments in the source code. + For example, to invalidate package sstate files, change the comment statments + of <filename>do_package</filename> or one of the functions it calls. + The change is purely cosmetic, but it causes the checksum to be recalculated and + forces the task to be run again. + </para> + + <note> + For an example of a commit that makes a cosmetic change to invalidate an sstate, + see this + <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. + </note> +</section> + +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> diff --git a/documentation/poky-ref-manual/usingpoky.xml b/documentation/poky-ref-manual/usingpoky.xml index 9a338b3..ea31659 100644 --- a/documentation/poky-ref-manual/usingpoky.xml +++ b/documentation/poky-ref-manual/usingpoky.xml @@ -4,150 +4,9 @@ <title>Using the Yocto Project</title> <para> - This section gives an overview of the components that make up the Yocto Project - followed by information about Yocto Project builds and dealing with any - problems that might arise. + This section describes the common usage for the Yocto Project. </para> -<section id='usingpoky-components'> - <title>Yocto Project Components</title> - - <para> - The BitBake task executor together with various types of configuration files form the - Yocto Project core. - This section overviews the BitBake task executor and the - configuration files by describing what they are used for and how they interact. - </para> - - <para> - BitBake handles the parsing and execution of the data files. - The data itself is of various types: - <itemizedlist> - <listitem><para><emphasis>Recipes:</emphasis> Provides details about particular - pieces of software</para></listitem> - <listitem><para><emphasis>Class Data:</emphasis> An abstraction of common build - information (e.g. how to build a Linux kernel).</para></listitem> - <listitem><para><emphasis>Configuration Data:</emphasis> Defines machine-specific settings, - policy decisions, etc. - Configuration data acts as the glue to bind everything together.</para></listitem> - </itemizedlist> - For more information on data, see the - <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#yocto-project-terms'> - Yocto Project Terms</ulink> section in - <ulink url='http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html'> - The Yocto Project Development Manual</ulink>. - </para> - - <para> - BitBake knows how to combine multiple data sources together and refers to each data source - as a <link linkend='usingpoky-changes-layers'>'layer'</link>. - </para> - - <para> - Following are some brief details on these core components. - For more detailed information on these components see the - <link linkend='ref-structure'>'Reference: Directory Structure'</link> - appendix. - </para> - - <section id='usingpoky-components-bitbake'> - <title>BitBake</title> - - <para> - BitBake is the tool at the heart of the Yocto Project and is responsible - for parsing the metadata, generating a list of tasks from it, - and then executing those tasks. - To see a list of the options BitBake supports, use the following help command: - <literallayout class='monospaced'> - $ bitbake --help - </literallayout> - </para> - - <para> - The most common usage for BitBake is <filename>bitbake <packagename></filename>, where - <filename>packagename</filename> is the name of the package you want to build - (referred to as the "target" in this manual). - The target often equates to the first part of a <filename>.bb</filename> filename. - So, to run the <filename>matchbox-desktop_1.2.3.bb</filename> file, you - might type the following: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop - </literallayout> - Several different versions of <filename>matchbox-desktop</filename> might exist. - BitBake chooses the one selected by the distribution configuration. - You can get more details about how BitBake chooses between different - target versions and providers in the - <link linkend='ref-bitbake-providers'>Preferences and Providers</link> section. - </para> - - <para> - BitBake also tries to execute any dependent tasks first. - So for example, before building <filename>matchbox-desktop</filename>, BitBake - would build a cross compiler and <filename>eglibc</filename> if they had not already - been built. - <note>This release of the Yocto Project does not support the <filename>glibc</filename> - GNU version of the Unix standard C library. By default, the Yocto Project builds with - <filename>eglibc</filename>.</note> - </para> - - <para> - A useful BitBake option to consider is the <filename>-k</filename> or - <filename>--continue</filename> option. - This option instructs BitBake to try and continue processing the job as much - as possible even after encountering an error. - When an error occurs, the target that - failed and those that depend on it cannot be remade. - However, when you use this option other dependencies can still be processed. - </para> - </section> - - <section id='usingpoky-components-metadata'> - <title>Metadata (Recipes)</title> - - <para> - The <filename>.bb</filename> files are usually referred to as "recipes." - In general, a recipe contains information about a single piece of software. - The information includes the location from which to download the source patches - (if any are needed), which special configuration options to apply, - how to compile the source files, and how to package the compiled output. - </para> - - <para> - The term "package" can also be used to describe recipes. - However, since the same word is used for the packaged output from the Yocto - Project (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files), - this document avoids using the term "package" to refer to recipes. - </para> - </section> - - <section id='usingpoky-components-classes'> - <title>Classes</title> - - <para> - Class files (<filename>.bbclass</filename>) contain information that is useful to share - between metadata files. - An example is the Autotools class, which contains - common settings for any application that Autotools uses. - The <link linkend='ref-classes'>Reference: Classes</link> appendix provides details - about common classes and how to use them. - </para> - </section> - - <section id='usingpoky-components-configuration'> - <title>Configuration</title> - - <para> - The configuration files (<filename>.conf</filename>) define various configuration variables - that govern the Yocto Project build process. - These files fall into several areas that define machine configuration options, - distribution configuration options, compiler tuning options, general common configuration - options and user configuration options (<filename>local.conf</filename>, which is found - in the Yocto Project files build directory). - </para> - </section> -</section> - - <section id='usingpoky-build'> <title>Running a Build</title> @@ -221,83 +80,6 @@ due to some components not being rebuilt. </para> </section> - - <section id="considering-shared-state-cache"> - <title>Considering Shared State Cache</title> - - <para> - By design, the Yocto Project builds everything from scratch unless it can determine that - a given task's inputs have not changed. - While building from scratch ensures that everything is current, it does also - mean that a lot of time could be spent rebuiding things that don't necessarily need built. - </para> - - <para> - The Yocto Project build process uses a shared state caching scheme to avoid having to - rebuild software when it is not necessary. - Because the build time for a Yocto image can be significant, it is helpful to try and - determine what really needs built and what can be skipped given a particular project's - build process. - </para> - - <para> - The scheme that the Yocto Project uses involves checksum generation and comparison for - a task's inputs. - The scheme also employs an area of memory called the shared state cache that is - pointed to by the <filename>SSTATE_DIR</filename> variable. - This area contains task output generated from a previous build. - If a given task's checksum matches the checksum of a previous build for the same - task, the build process uses the state of the cache rather than rerunning that - task. - </para> - - <para> - The previous paragraph is a simplistic explanation of how the build process - uses checksums and shared state memory cache to avoide building tasks that - don't need built. - If you want a bit more explanation on the topic, - see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared - State - What does it mean and why should I care?</ulink>" from the Yocto - Project discussion archives. - </para> - - <para> - As with all schemes, this one has some drawbacks. - It is possible that you could make implicit changes that are not factored into the checksum - calculation, but do affect a task's output. - A good example is perhaps when a tool changes its output. - Let's say that the output of <filename>rpmdeps</filename> needed to change. - The result of the change should be that all the "package", "package_write_rpm", - and "package_deploy-rpm" sstate-cache items would become invalid. - But, because this is a change that is external to the code and therefore implicit, - the associated sstate-cache items do not become invalidated. - In this case, the build process would use the cache items rather than running the - task again. - Obviously, these types of implicit changes can cause problems. - </para> - - <para> - To avoid these problems during the build, you need to understand the effects of any - change you make. - Note that any changes you make directly to a function automatically are factored into - the checksum calculation and thus, will invalidate the associated area of sstate cache. - You need to be aware of any implicit changes that are not obvious changes to the - code and could affect the output of a given task. - Once you are aware of such a change, you can take steps to invalidate the cache - and force the task to run. - The step to take is as simple as changing a function's comments in the source code. - For example, to invalidate package sstate files, change the comment statments - of <filename>do_package</filename> or one of the functions it calls. - The change is purely cosmetic, but it causes the checksum to be recalculated and - forces the task to be run again. - </para> - - <note> - For an example of a commit that makes a cosmetic change to invalidate an sstate, - see this - <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. - </note> - </section> </section> <section id='usingpoky-install'> |