summaryrefslogtreecommitdiff
path: root/documentation/poky-ref-manual/ref-bitbake.xml
diff options
context:
space:
mode:
authorRichard Purdie <richard.purdie@linuxfoundation.org>2011-04-20 14:20:19 +0100
committerRichard Purdie <richard.purdie@linuxfoundation.org>2011-04-20 15:49:17 +0100
commit50021cba20a09b1ed685db5466f940b17d4880ac (patch)
tree3bdafb797e6466ad58727b002f1235933010ab11 /documentation/poky-ref-manual/ref-bitbake.xml
parent690e87a2ffe8caa16379be26eb356c5bded17c1f (diff)
downloadopenembedded-core-50021cba20a09b1ed685db5466f940b17d4880ac.tar.gz
openembedded-core-50021cba20a09b1ed685db5466f940b17d4880ac.tar.bz2
openembedded-core-50021cba20a09b1ed685db5466f940b17d4880ac.zip
Drop documentation directory, this is replaced by the new yocto-docs repository
Diffstat (limited to 'documentation/poky-ref-manual/ref-bitbake.xml')
-rw-r--r--documentation/poky-ref-manual/ref-bitbake.xml347
1 files changed, 0 insertions, 347 deletions
diff --git a/documentation/poky-ref-manual/ref-bitbake.xml b/documentation/poky-ref-manual/ref-bitbake.xml
deleted file mode 100644
index 75b3bf5e54..0000000000
--- a/documentation/poky-ref-manual/ref-bitbake.xml
+++ /dev/null
@@ -1,347 +0,0 @@
-<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
-"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
-
-<appendix id='ref-bitbake'>
-
- <title>Reference: BitBake</title>
-
- <para>
- BitBake is a program written in Python that interprets the metadata that makes up Poky.
- At some point, people wonder what actually happens when you enter:
- <literallayout class='monospaced'>
- $ bitbake poky-image-sato
- </literallayout>
- </para>
-
- <para>
- This appendix provides an overview of what happens behind the scenes from BitBake's perspective.
- </para>
-
- <note><para>
- BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships.
- As such, it has no real knowledge of what the tasks being executed actually do.
- BitBake just considers a list of tasks with dependencies and handles metadata
- that consists of variables in a certain format that get passed to the tasks.
- </para></note>
-
- <section id='ref-bitbake-parsing'>
- <title>Parsing</title>
-
- <para>
- BitBake parses configuration files, classes, and <filename>.bb</filename> files.
- </para>
-
- <para>
- The first thing BitBake does is look for the <filename>bitbake.conf</filename> file.
- Poky keeps this file in <filename>meta/conf/</filename>.
- BitBake finds it by examining the <filename>BBPATH</filename> environment
- variable and looking for the <filename>meta/conf/</filename>
- directory.
- </para>
-
- <para>
- In Poky, <filename>bitbake.conf</filename> lists other configuration
- files to include from a <filename>conf/</filename>
- directory below the directories listed in <filename>BBPATH</filename>.
- In general the most important configuration file from a user's perspective
- is <filename>local.conf</filename>, which contains a user's customized
- settings for Poky.
- Other notable configuration files are the distribution
- configuration file (set by the <glossterm><link linkend='var-DISTRO'>
- DISTRO</link></glossterm> variable) and the machine configuration file
- (set by the <glossterm><link linkend='var-MACHINE'>MACHINE</link>
- </glossterm> variable).
- The DISTRO and MACHINE environment variables are both usually set in
- the <filename>local.conf</filename> file.
- Valid distribution
- configuration files are available in the <filename>
- meta/conf/distro/</filename> directory and valid machine configuration
- files in the <filename>meta/conf/machine/</filename>
- directory.
- Within the <filename>meta/conf/machine/include/</filename>
- directory are various <filename>tune-*.inc</filename> configuration files that provide common
- "tuning" settings specific to and shared between particular architectures and machines.
- </para>
-
- <para>
- After the parsing of the configuration files some standard classes are included.
- The <filename>base.bbclass</filename> file is always included.
- Other classes that are specified in the configuration using the
- <glossterm><link linkend='var-INHERIT'>INHERIT</link></glossterm>
- variable are also inculded.
- Class files are searched for in a classes subdirectory
- under the paths in <filename>BBPATH</filename> in the same way as
- configuration files.
- </para>
-
- <para>
- After classes are included, the
- variable <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm>
- is set, usually in
- <filename>local.conf</filename>, and defines the list of places to search for
- <filename>.bb</filename> files.
- By default, the BBFILES variable specifies the <filename>meta/recipes-*/
- </filename> directory within Poky.
- Adding extra content to BBFILES is best achieved through the use of BitBake
- <link linkend='usingpoky-changes-layers'>"layers"</link>.
- </para>
-
- <para>
- BitBake parses each <filename>.bb</filename> file in BBFILES and
- stores the values of various variables.
- In summary, for each <filename>.bb</filename>
- file the configuration plus the base class of variables are set, followed
- by the data in the <filename>.bb</filename> file
- itself, followed by any inherit commands that
- <filename>.bb</filename> file might contain.
- </para>
-
- <para>
- Because parsing <filename>.bb</filename> files is a time
- consuming process, a cache is kept to speed up subsequent parsing.
- This cache is invalid if the timestamp of the <filename>.bb</filename>
- file itself changes, or if the timestamps of any of the include,
- configuration or class files the <filename>.bb</filename>
- file depends on changes.
- </para>
- </section>
-
- <section id='ref-bitbake-providers'>
- <title>Preferences and Providers</title>
-
- <para>
- Once all the <filename>.bb</filename> files have been
- parsed, BitBake starts to build the target (poky-image-sato in the previous section's
- example) and looks for providers of that target.
- Once a provider is selected, BitBake resolves all the dependencies for
- the target.
- In the case of "poky-image-sato", it would lead to <filename>task-base.bb</filename>,
- which in turn leads to packages like <application>Contacts</application>,
- <application>Dates</application> and <application>BusyBox</application>.
- These packages in turn depend on glibc and the toolchain.
- </para>
-
- <para>
- Sometimes a target might have multiple providers.
- A common example is "virtual/kernel", which is provided by each kernel package.
- Each machine often elects the best kernel provider by using a line similar to the
- following in the machine configuration file:
- </para>
-
- <programlisting>
-PREFERRED_PROVIDER_virtual/kernel = "linux-rp"
- </programlisting>
-
- <para>
- The default <glossterm><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></glossterm>
- is the provider with the same name as the target.
- </para>
-
- <para>
- Understanding how providers are chosen is made complicated by the fact
- that multiple versions might exist.
- BitBake defaults to the highest version of a provider.
- Version comparisons are made using the same method as Debian.
- You can use the <glossterm><link linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm>
- variable to specify a particular version (usually in the distro configuration).
- You can influence the order by using the
- <glossterm><link linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm>
- variable.
- By default, files have a preference of "0".
- Setting the DEFAULT_PREFERENCE to "-1" makes the package unlikely to be used unless it is
- explicitly referenced.
- Setting the DEFAULT_PREFERENCE to "1" makes it likely the package is used.
- PREFERRED_VERSION overrides any DEFAULT_PREFERENCE setting.
- DEFAULT_PREFERENCE is often used to mark newer and more experimental package
- versions until they have undergone sufficient testing to be considered stable.
- </para>
-
- <para>
- In summary, BitBake has created a list of providers, which is prioritized, for each target.
- </para>
- </section>
-
- <section id='ref-bitbake-dependencies'>
- <title>Dependencies</title>
-
- <para>
- Each target BitBake builds consists of multiple tasks such as fetch, unpack, patch, configure,
- and compile.
- For best performance on multi-core systems, BitBake considers each task as an independent
- entity with its own set of dependencies.
- </para>
-
- <para>
- Dependencies are defined through several variables.
- You can find information about variables BitBake uses in the
- <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
- At a basic level it is sufficient to know that BitBake uses the
- <glossterm><link linkend='var-DEPENDS'>DEPENDS</link></glossterm> and
- <glossterm><link linkend='var-RDEPENDS'>RDEPENDS</link></glossterm> variables when
- calculating dependencies.
- </para>
-
- </section>
-
- <section id='ref-bitbake-tasklist'>
- <title>The Task List</title>
-
- <para>
- Based on the generated list of providers and the dependency information,
- BitBake can now calculate exactly what tasks it needs to run and in what
- order it needs to run them.
- The build now starts with BitBake forking off threads up to the limit set in the
- <glossterm><link linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></glossterm> variable.
- BitBake continues to fork threads as long as there are tasks ready to run,
- those tasks have all their dependencies met, and the thread threshold has not been
- exceeded.
- </para>
-
- <para>
- As each task completes, a timestamp is written to the directory specified by the
- <glossterm><link linkend='var-STAMPS'>STAMPS</link></glossterm> variable (usually
- <filename>build/tmp/stamps/*/</filename>).
- On subsequent runs, BitBake looks at the STAMPS directory and does not rerun
- tasks that are already completed unless a timestamp is found to be invalid.
- Currently, invalid timestamps are only considered on a per
- <filename>.bb</filename> file basis.
- So, for example, if the configure stamp has a timestamp greater than the
- compile timestamp for a given target then the compile task would rerun.
- Running the compile task again, however, has no effect on other providers
- that depend on that target.
- This behavior could change or become configurable in future versions of BitBake.
- </para>
-
- <note><para>
- Some tasks are marked as "nostamp" tasks.
- No timestamp file is created when these tasks are run.
- Consequently, "nostamp" tasks are always rerun.
- </para></note>
- </section>
-
- <section id='ref-bitbake-runtask'>
- <title>Running a Task</title>
-
- <para>
- Tasks can either be a shell task or a python task.
- For shell tasks, BitBake writes a shell script to
- <filename>${WORKDIR}/temp/run.do_taskname.pid</filename> and then executes the script.
- The generated shell script contains all the exported variables, and the shell functions
- with all variables expanded.
- Output from the shell script goes to the file <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
- Looking at the expanded shell functions in the run file and the output in the log files
- is a useful debugging technique.
- </para>
-
- <para>
- For Python tasks, BitBake executes the task internally and logs information to the
- controlling terminal.
- Future versions of BitBake will write the functions to files similar to the way
- shell tasks are handled.
- Logging will be handled in way similar to shell tasks as well.
- </para>
-
- <para>
- Once all the tasks have been completed BitBake exits.
- </para>
- </section>
-
-
- <section id='ref-bitbake-commandline'>
- <title>BitBake Command Line</title>
-
- <para>
- Following is the bitbake manpage:
- </para>
-
- <screen>
-$ bitbake --help
-Usage: bitbake [options] [package ...]
-
-Executes the specified task (default is 'build') for a given set of BitBake files.
-It expects that BBFILES is defined, which is a space separated list of files to
-be executed. BBFILES does support wildcards.
-Default BBFILES are the .bb files in the current directory.
-
-Options:
- --version show program's version number and exit
- -h, --help show this help message and exit
- -b BUILDFILE, --buildfile=BUILDFILE
- execute the task against this .bb file, rather than a
- package from BBFILES.
- -k, --continue continue as much as possible after an error. While the
- target that failed, and those that depend on it,
- cannot be remade, the other dependencies of these
- targets can be processed all the same.
- -a, --tryaltconfigs continue with builds by trying to use alternative
- providers where possible.
- -f, --force force run of specified cmd, regardless of stamp status
- -i, --interactive drop into the interactive mode also called the BitBake
- shell.
- -c CMD, --cmd=CMD Specify task to execute. Note that this only executes
- the specified task for the providee and the packages
- it depends on, i.e. 'compile' does not implicitly call
- stage for the dependencies (IOW: use only if you know
- what you are doing). Depending on the base.bbclass a
- listtasks tasks is defined and will show available
- tasks
- -r FILE, --read=FILE read the specified file before bitbake.conf
- -v, --verbose output more chit-chat to the terminal
- -D, --debug Increase the debug level. You can specify this more
- than once.
- -n, --dry-run don't execute, just go through the motions
- -p, --parse-only quit after parsing the BB files (developers only)
- -d, --disable-psyco disable using the psyco just-in-time compiler (not
- recommended)
- -s, --show-versions show current and preferred versions of all packages
- -e, --environment show the global or per-package environment (this is
- what used to be bbread)
- -g, --graphviz emit the dependency trees of the specified packages in
- the dot syntax
- -I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS
- Stop processing at the given list of dependencies when
- generating dependency graphs. This can help to make
- the graph more appealing
- -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
- Show debug logging for the specified logging domains
- -P, --profile profile the command and print a report
- </screen>
- </section>
-
- <section id='ref-bitbake-fetchers'>
- <title>Fetchers</title>
-
- <para>
- BitBake also contains a set of "fetcher" modules that allow
- retrieval of source code from various types of sources.
- For example, BitBake can get source code from a disk with the metadata, from websites,
- from remote shell accounts or from Source Code Management (SCM) systems
- like <filename>cvs/subversion/git</filename>.
- </para>
-
- <para>
- Fetchers are usually triggered by entries in
- <glossterm><link linkend='var-SRC_URI'>SRC_URI</link></glossterm>.
- You can find information about the options and formats of entries for specific
- fetchers in the <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
- </para>
-
- <para>
- One useful feature for certain SCM fetchers is the ability to
- "auto-update" when the upstream SCM changes version.
- Since this ability requires certain functionality from the SCM, not all
- systems support it.
- Currently Subversion, Bazaar and to a limited extent, Git support the ability to "auto-update".
- This feature works using the <glossterm><link linkend='var-SRCREV'>SRCREV</link></glossterm>
- variable.
- See the
- <link linkend='platdev-appdev-srcrev'>Developing within Poky with an External SCM-based Package</link>
- section for more information.
- </para>
-
- </section>
-
-</appendix>
-<!--
-vim: expandtab tw=80 ts=4 spell spelllang=en_gb
--->