summaryrefslogtreecommitdiff
path: root/meta/lib/oe
AgeCommit message (Collapse)AuthorFiles
2015-08-30image.py: write bitbake variables to .env fileEd Bartosh1
Write set of bitbake variables used by wic into build/tmp/sysroots/<machine>/imagedata/<image>.env List of variables is defined in WICVARS variable in meta/classes/image_types.bbclass. This is needed for wic to be able to get bitbake variables without running 'bitbake -e'. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30image.py: add script output to the rootfs logEd Bartosh1
Let's add output of image creation script to the bitbake log as it can contain useful information. One good example of such an information is wic report about artifacts and .wks file used for image creation. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-30image.py: set bitbake variable ROOTFS_SIZEEd Bartosh1
This variable is going to be used by wic to set partition size. Setting it in image.py makes it possible for wic to use it without calculating it again. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-30lib/oe/patch: handle patches that delete files with older gitPaul Eggleton1
When running "git add" on a directory with older git versions, deleted files aren't added to what is to be committed unless you explicitly specify the -A option. The result of this is that when applying a patch from a recipe which doesn't apply with "git am" (and we fall back to applying the patch through other means then "git add" following by a "git commit") these deletes weren't committed with the patch, leaving them sitting deleted but not committed at the end. This should fix test_devtool_modify_native (which unwittingly exercises this scenario by attempting to run "devtool modify" on apt-native) on machines with older git versions. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-29Empty image: package list splitting and iterationAlex Franco2
A few short fixes to splitting/iteration done over package lists in license.bbclass, package_manager.py and rootfs.py. [YOCTO #7664] Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19lib/oe/recipeutils: avoid parsing in get_var_files()Paul Eggleton1
Let's have the caller do this and then the function is a bit more flexible (e.g. we can choose to parse with bbappends or not); fix up calls to this function appropriately (of which there are only two, both within devtool). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16IMAGES_FSTYPES: default to EXT4Juro Bystricky1
The following IMAGES_FSTYPES defaulted to ext3: "vmdk", "vdi", "qcow2", "live", "iso", "hddimg" This patch changes the default for those IMAGES_FSTYPES to ext4 in order to bring the images more in line with other BSPs. Besides improvements in performance and reliability ext4 provides additional functionality as well (option to turn off the journaling, dynamic resizing of VDI volumes etc.). Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-14lib/oe/package_manager: fix opkg feed generationJoshua Lock1
The insert_feed_uris() method of OpkgPM was creating an initial entry in the feeds list which pointed to the root of the ipk directory, however the on-device package manager can't consume this feed resulting in runtime errors - therefore we remove the code to generate that initial feed uri. Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09recipeutils.py: allow all characters in regexes used to parse version stringsAlexander Kanavin1
Previously only numeric characters were matches and anything else was discarded, so 4.0-rc3, 2005e, 1.0.2a and similar versions got truncated. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01rootfs.py: fix PRE/POSTPROCESS_COMMANDS for rpm and debRobert Yang1
The rpm didn't run RPM_PREPROCESS_COMMANDS or RPM_POSTPROCESS_COMMANDS, the similar to deb, this patch fix the problem. And fix a typo: DEB_POSTPROCESS_COMMAND -> DEB_POSTPROCESS_COMMANDS Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-29lib/oe/classextend: Fix nativesdk double name mappingRichard Purdie1
Handle the case a name has already been extended in the nativesdk case (avoids double name extensions which can happen with nativesdk-gcc). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22recipeutils.py: get_recipe_pv_without_srcpv remove prefixes from pvAníbal Limón1
Some recipes uses v or r prefixes in versions that makes wrong comparisions over recipes like lz4 r123 > 128. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oe/recipeutils.py: get_recipe_upstream_info update to get revisionAníbal Limón1
Bitbake fetcher latest_versionstring now returns a tuple with (version, revision) that helps SCM's like git to build current upstream version. [YOCTO #7605] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oe/recipeutils.py: get_recipe_upstream_info only use sfx and pfx when exitsAníbal Limón1
Don't use pfx and sfx when not exist because cause formatting errors like 2.9HASH instead of 2.9+gitAUTOINC+HASH. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-20lib/oe/package_manager: Use log-level warning instead of quietRichard Purdie1
quiet means that no warnings are shown on the console and really is too quiet for our needs. This is more verbose but that is better than too little and does aid debugging things when they go wrong. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-16lib/oe/recipeutils: fix replace_dir_vars to return the correct variablesPaul Eggleton1
If we sort by length of name here we get the variables we are interested in first. I've tested this with all of the variables we care about (the ones at the top of bitbake.conf) and it returns the right results. This fixes the failures we've been seeing in the oe-selftest test_recipetool_appendfile_* tests. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-12oe/recipeutils.py: get_recipe_upstream_version returns 1.0 when not SRC_URIAníbal Limón1
Recipes that don't have SRC_URI means that don't use upstream sources so returns 1.0. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12lib/oe/recipeutils: Ensure order of data store doesn't influence the resultRichard Purdie1
The order of the keys from the data store is not prescribed. If target_datadir comes before datadir the selftests fail since the 'wrong' variable is used for substitutions. This highlights an issue with the replace_dir_vars() function. Fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02rootfs.py: Add check for kernel modules before running depmodSaul Wold1
Add a check for kernel modules so we don't un-necessarily run the depmods, this will also handle the case with linux-dummy does not place the kernel-abiversion since it also does not have kernel modules. [YOCTO #7884] Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-27oe.recipeutils: fix line.split error in bbappend_recipeChristopher Larson1
Cc: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26oe/rootfs.py: DpkgRootfs/OpkgRootfs add support for dependency handling in ↵Aníbal Limón1
postinsts scripts. The old code don't take into account package dependencies causing undefined execution order in postinsts scripts, in order to fix: Add DpkgOpkgRootfs class for store common operations in DpkgRootfs and OpkgRootfs. Add _get_delayed_postinsts_common method that process Depends from status file in dpkg/opkg and resolve dependency order causing an execption if found circular dependencies. [YOCTO #5318] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26terminal.py: Allow devshell/menuconfig on recent gnome-terminalLeonardo Sandoval1
Recent versions of gnome-terminal does not support non-UTF8 charset: https://bugzilla.gnome.org/show_bug.cgi?id=732127 as a result, devshell and menuconfig tasks silently hang (error found on trace log of 'strace -f -v -s 8192 -e write=2 bitbake -c devshell quilt-native': "Non UTF-8 locale (ANSI_X3.4-1968) is not supported!"). As a workaround, clearing the LC_ALL environment variable so it uses the locale. Once fixed on the gnome-terminal project, this should be removed. Tested on gnome-terminal versions: GNOME Terminal 3.4.1.1 GNOME Terminal 3.14.2 (Default on Ubuntu 15.04) [YOCTO #7791] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-26lib/oe/patch: Fix git patch application for source in subdirectoryRichard Purdie1
Similarly to: http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/lib/oe/patch.py?id=f205ccaf48ac36f4b26efc4aeb2e9d2939b28646 we need to fix patch application for source which is in a subdirectory. Passing "." as the git directory or work-dir appears to work (or is ignored) in some versions of git but does not work in others, probably quite correctly. Since we have reporoot from the above patch, pass this in directly. This bug caused this sanity test failure on some machines: FAIL: test_devtool_modify_git (oeqa.selftest.devtool.DevtoolTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/devtool.py", line 390, in test_devtool_modify_git self.assertEqual(result.output.strip(), "", 'Created git repo is not clean') AssertionError: '?? util/mkelfImage/patches/' != '' : Created git repo is not clean since git apply would fail, it would then fall back to quilt and the git tree would be left unclean. [YOCTO #7911] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26oe/utils.py: Add support for init/end helper functions in ThreadWorker.Aníbal Limón1
Add init/end helper functions for ThreadWorker also pass ThreadWorker as first argument to init/end/func functions this enables per-thread storage handling. classes/sstate.bbclass: Add thread_worker argument to checkstatus function. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-26oe/utils.py: Fix thread leakage in ThreadPoolAníbal Limón1
In order to fix Thread leakage caused by not call join() in Threads, Pass num_tasks in ThreadPool for add all the tasks into a Queue this enable catch of Queue.Empty exception and exit the threads. classes/sstate.bbclass: Change checkstatus function to match new ThreadPool operation. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23meta: Add explict getVar param for (non) expansionRichard Purdie1
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11image: Support for VDIJuro Bystricky1
Added support for VirtualBox VDI format. The support was implemented by merging with the already existing VMDK support for VM player by creating a new class image-vm.bbclass. This class replaces the previous VMDK only image-vmdk.class. Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-11lib/oe/rootfs: tidy up log warning reportingPaul Eggleton1
* bb.warn() should only be called once per warning - UIs such as Toaster assume that this is the case, so adjust the output accordingly. (It's tricky here because we have to include "log_check" on every line or we'll end up looping forever as the log checking code's own messages retrigger the log check, sigh...) * Iterating over a file already splits by line, there's no need to do it manually. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-06-08recipeutils: Improve get_recipe_pv_without_srcpv functionAníbal Limón1
Use pv instead of rd this make the function more generic and avoid copy recipe data. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2015-06-03recipeutils: Add get_recipe_upstream_version and get_recipe_pv_without_srcpv ↵Aníbal Limón1
functions The get_recipe_upstream_version functions tries to get the current version of recipe in upstream it uses bb.fetch2 latest_versionstring method also latest_revision when is SCM. The get_recipe_pv_without_srcpv discards the SRCPV in SCM's recipe like git it returns a tuple with the version, prefix and suffix of a PV. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-01oe/utils: Add simple threaded pool implementationRichard Purdie1
Python 2.7 doesn't have a threaded pool implementation, just a multiprocessing one. We have need of a threaded implementation so add some simple class code to support this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-23image.bbclass: Add a method for creating a companion debug filesystemMark Hatle1
The companion debug filesystem contains only the package database and the complementary *-dbg packages for the main filesystem component. This is useful in a production environment to produce a companion filesystem capable of remote system debugging, without requiring corresponding debug symbols or source code on the device. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com> If dbg pkgs have already been installed to the rootfs image, the installation to companion debug filesystem will fail, because both of image creation make use of the same pm database. In this situation, try to copy installed dbg files from rootfs image to companion debug filesystem. Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Acked-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-23utils: add helper to perform the intersection of two string listsRoss Burton1
Useful for e.g. generating a COMBINED_FEATURES list from DISTRO_FEATURES and MACHINE_FEATURES. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-22rootfs.py: Make set -x not break rootfs creationPeter Seebach1
The rootfs logfile check errors out if it finds an "exit 1" in a log. But consider the shell idiom: if ! $command; then exit 1; fi and consider a postinstall script with a "set -x" for debugging purposes (to get log output in /var/log/postinstall.log.) Solution: Ignore lines prefixed with a +, because those show shell code even if a specific fragment won't be executed. Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-21rootfs.py: Improve rpm log_check_regexEd Bartosh1
Current regex can cause false negatives if paths in bitbake log files contain "ERR" or "Fail". do_rootfs fails with return code 1 in this case. Improved regexp is based on error messages produced by rpm. Those are found by analyzing rpm source code. [YOCTO #7789] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-19recipetool: add appendfile subcommandPaul Eggleton2
Locating which recipe provides a file in an image that you want to modify and then figuring out how to bbappend the recipe in order to replace it can be a tedious process. Thus, add a new appendfile subcommand to recipetool, providing the ability to create a bbappend file to add/replace any file in the target system. Without the -r option, it will search for the recipe packaging the specified file (using pkgdata from previously built recipes). The bbappend will be created at the appropriate path within the specified layer directory (which may or may not be in your bblayers.conf) or if one already exists it will be updated appropriately. Fairly extensive oe-selftest tests are also provided. Implements [YOCTO #6447]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19lib/oe/patch: use with open() for all file operationsPaul Eggleton1
with open(...)... is preferred for reading/writing files as it is neater and takes care of closing the file for you. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15lib/oe/recipeutils: add a parse_recipe_simple() functionPaul Eggleton1
Add a function that simply parses a recipe by name and optionally the bbappends that apply to it. (Note that if you're using tinfoil you need to have initialised it with config_only=False so that it can map the recipe name to a recipe file.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-05-09license: Add support for handle INCOMPATIBLE_LICENSE in manifest creationAníbal Limón1
When INCOMPATIBLE_LICENSE's is specified it need to be removed from license.manifest and also avoid copy to target image. Add ManifestVisitor that walk the license string searching for INCOMPATIBLE_LICENSE's if found remove it. [YOCTO #6765] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09license: Split visit_string in LicenseVisitorAníbal Limón1
Create get_elements and visit_elements in LicenseVisitor based on visit_string this allow to do modifications on elements before parsing with AST. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09license_class: Generalize license_ok functionAníbal Limón1
Add dont_want_licenses as parameter to license_ok function and move it to oe.license module in order to use in other modules. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29split_and_strip_files: regroup hardlinks to make build deterministicEd Bartosh1
Reverted 7c0fd561bad0250a00cef63e3d787573112a59cf Created separate group of hardlinks for the files inside the same package. This should prevent stripped files to be populated outside of package directories. This turns out not to be straightforward and has overlap with the other hardlink handling code in this area. The code is condensed into a more concise and documented form. [Original patch from Ed with tweaks from RP] [YOCTO #7586] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29lib/oe/package: Improve strip subprocess handlingRichard Purdie1
Currently if the strip process fails, we get a message but don't know why. This adds code to show the return value and any error output. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: handle . in recipe namePaul Eggleton1
Names such as glib-2.0 are valid (and used) recipe names, so we need to support them. Fixes [YOCTO #7643]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: include bbappends in recipe parsingMarkus Lehtonen1
In order to get correct metadata, SRCREV for example. Fixes [YOCTO #7648]. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-19lib/oe/package_manager: avoid traceback with no packagesPaul Eggleton1
If you were using deb packaging, had buildhistory enabled and produced an SDK that contained no packages in one of the sysroots (such as with uninative-tarball) then the do_populate_sdk was failing with a python traceback because there were no fields to split in the output line. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-13package_manager: RpmPM: Fix scriptlet for rpm 4Jean-Benoit MARTIN1
Path of rpm post install script is different between rpm 4 and 5 Signed-off-by: Jean-Benoit MARTIN <jean-benoit.martin@open.eurogiciel.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-06oe.sstatesig: align swspec handling with sstate.bbclassChristopher Larson1
The logic tries to remove the -native suffix from pn to handle this (though it doesn't succeed, as it doesn't assign the new pn to the variable), but we need to do more for the swspec tasks than just not set the extrapath, we also need to change from SSTATE_PKGSPEC to SSTATE_SWSPEC. Alter to correct the spec for these cases, and also add preconfigure to align with the current logic in sstate.bbclass, which includes that task as well in the list of tasks to adjust to use swspec. [YOCTO #7563] Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-06meta/lib/oe/utils.py: Corrected the return value of both_contain()Jun Zhu1
oe.utils.both_contain() should return the result as "checkvalue" or "", but the latest implement returns as "set(['checkvalue'])" or ""; It causes that bitbake.conf generates the wrong result of COMBINED_FEATURES, which contains the common components in both DISTRO_FEATURE and MACHINE_FEATURES. For example, build in Dizzy branch, COMBINED_FEATURES is "alsa usbhost ...", but recently, COMBINED_FEATURES is like "set(['alsa']) set(['usbhost']) ...". Signed-off-by: Jun Zhu <R01007@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-02package_manager: call createrepo with --dbpath pointing inside WORKDIREd Bartosh1
Rpm database in staging area is used only by createrepo. createrepo fails with the error "rpmdb: BDB0060 PANIC: fatal region error detected" if rpm database is broken during previous run of createrepo. Made createrepo to create rpm db in $WORKDIR/rpmdb/ from scratch for every build and architecture. This should potentially fix the failure as every run of createrepo will be using separate db. [YOCTO #6571] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>