summaryrefslogtreecommitdiff
path: root/meta/lib
AgeCommit message (Collapse)AuthorFiles
2015-06-01oe/utils: Add simple threaded pool implementationRichard Purdie1
Python 2.7 doesn't have a threaded pool implementation, just a multiprocessing one. We have need of a threaded implementation so add some simple class code to support this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-23image.bbclass: Add a method for creating a companion debug filesystemMark Hatle1
The companion debug filesystem contains only the package database and the complementary *-dbg packages for the main filesystem component. This is useful in a production environment to produce a companion filesystem capable of remote system debugging, without requiring corresponding debug symbols or source code on the device. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com> If dbg pkgs have already been installed to the rootfs image, the installation to companion debug filesystem will fail, because both of image creation make use of the same pm database. In this situation, try to copy installed dbg files from rootfs image to companion debug filesystem. Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Acked-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-23oe-selftest: Build wic runtime requirements and images before testingEd Bartosh1
Some native tools (syslinux, parted, mtools, etc) are required by wic to produce images. Unit tests fail if the tools are not available. Baked tools and image-core-minimal used by wic before running tests. [YOCTO #7730] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-23utils: add helper to perform the intersection of two string listsRoss Burton1
Useful for e.g. generating a COMBINED_FEATURES list from DISTRO_FEATURES and MACHINE_FEATURES. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-22rootfs.py: Make set -x not break rootfs creationPeter Seebach1
The rootfs logfile check errors out if it finds an "exit 1" in a log. But consider the shell idiom: if ! $command; then exit 1; fi and consider a postinstall script with a "set -x" for debugging purposes (to get log output in /var/log/postinstall.log.) Solution: Ignore lines prefixed with a +, because those show shell code even if a specific fragment won't be executed. Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-21rootfs.py: Improve rpm log_check_regexEd Bartosh1
Current regex can cause false negatives if paths in bitbake log files contain "ERR" or "Fail". do_rootfs fails with return code 1 in this case. Improved regexp is based on error messages produced by rpm. Those are found by analyzing rpm source code. [YOCTO #7789] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-19devtool: update-recipe: add option to write changes to bbappendPaul Eggleton1
Quite often what you want to do having made customisations to a piece of software is to apply those customisations in your own layer rather than in the original recipe. Thus, add a -a/--append option to the update-recipe subcommand which allows you to specify the layer to write a bbappend into. The bbappend will be created at the appropriate path within the specified layer directory (which may or may not be in your bblayers.conf) or if one already exists it will be updated appropriately. (This re-uses code written for recipetool appendfile.) Implements [YOCTO #7587]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19recipetool: add appendfile subcommandPaul Eggleton5
Locating which recipe provides a file in an image that you want to modify and then figuring out how to bbappend the recipe in order to replace it can be a tedious process. Thus, add a new appendfile subcommand to recipetool, providing the ability to create a bbappend file to add/replace any file in the target system. Without the -r option, it will search for the recipe packaging the specified file (using pkgdata from previously built recipes). The bbappend will be created at the appropriate path within the specified layer directory (which may or may not be in your bblayers.conf) or if one already exists it will be updated appropriately. Fairly extensive oe-selftest tests are also provided. Implements [YOCTO #6447]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19oe-selftest: move recipetool tests to their own modulePaul Eggleton2
These tests really belong in their own module; if we refactor out a base class from DevtoolTests with shared functions then we can move them out easily. Also create temp directory in setupLocal() so we don't have to do that in individual tests anymore. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-19lib/oe/patch: use with open() for all file operationsPaul Eggleton1
with open(...)... is preferred for reading/writing files as it is neater and takes care of closing the file for you. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15lib/oe/recipeutils: add a parse_recipe_simple() functionPaul Eggleton1
Add a function that simply parses a recipe by name and optionally the bbappends that apply to it. (Note that if you're using tinfoil you need to have initialised it with config_only=False so that it can map the recipe name to a recipe file.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-05-14oeqa/utils: Fixed a problem with get_bb_var not returning right variable.Lucian Musat1
It searches using regex now and should be more accurate. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-09license: Add support for handle INCOMPATIBLE_LICENSE in manifest creationAníbal Limón1
When INCOMPATIBLE_LICENSE's is specified it need to be removed from license.manifest and also avoid copy to target image. Add ManifestVisitor that walk the license string searching for INCOMPATIBLE_LICENSE's if found remove it. [YOCTO #6765] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09license: Split visit_string in LicenseVisitorAníbal Limón1
Create get_elements and visit_elements in LicenseVisitor based on visit_string this allow to do modifications on elements before parsing with AST. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-09license_class: Generalize license_ok functionAníbal Limón1
Add dont_want_licenses as parameter to license_ok function and move it to oe.license module in order to use in other modules. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07oe-selftest: devtool: add a proper test to see if tap devices existPaul Eggleton1
Check up front in test_devtool_deploy_target whether the tap devices exist and skip if not. If we don't do this we get a significantly less comprehensible error via pexpect. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07devtool: fix for rename of gcc-sourcePaul Eggleton1
After OE-Core commit 67db7182faf6742b0d971d61d8c5ba34f69d2e12, PV is appended to the end of the gcc-source PN, thus we need to handle that in devtool and the corresponding test. Part of the fix for [YOCTO #7729]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-07oe-selftest: devtool: fix broken URL in test_devtool_add_fetchPaul Eggleton1
I already had the file fetched from some previous work and thus it didn't attempt to download the invalid URL when I tested it earlier. Part of the fix for [YOCTO #7729]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-03wic tests: Move wic tests to oe-selftest infraEd Bartosh1
Modified existing wic test modules to work with oe-selftest infrastructure. $ oe-selftest --run-tests wic ... test01_help (oeqa.selftest.wic.Wic) Test wic --help ... ok test02_createhelp (oeqa.selftest.wic.Wic) Test wic create --help ... ok test03_listhelp (oeqa.selftest.wic.Wic) Test wic list --help ... ok test04_build_image_name (oeqa.selftest.wic.Wic) Test wic create directdisk --image-name core-image-minimal ... ok test05_build_artifacts (oeqa.selftest.wic.Wic) Test wic create directdisk providing all artifacts. ... ok ---------------------------------------------------------------------- Ran 5 tests in 6.434s OK Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-03oeqa/utils: Allow ~ in bblayersEd Bartosh1
Bitbake can parse ~ in bblayer's paths. Added this functionality to oeqa code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01oe-selftest: devtool: add some tests for devtool add -fPaul Eggleton1
Tests the following aspects of "devtool add" with the -f option: * Works for a git and https URI * Extracts to the appropriate directory * Uses the correct recipe file name with and without -V * Sets SRC_URI and S appropriately with and without -V * Sets SRCREV if rev= is specified in the URI A lot of this functionality relies on "recipetool create" which "devtool add" wraps around, so the associated behaviour of that is also being tested here. Refactor out the code to check a recipe for variable values / inherits at the same time so we can use it to check the generated recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-04-29split_and_strip_files: regroup hardlinks to make build deterministicEd Bartosh1
Reverted 7c0fd561bad0250a00cef63e3d787573112a59cf Created separate group of hardlinks for the files inside the same package. This should prevent stripped files to be populated outside of package directories. This turns out not to be straightforward and has overlap with the other hardlink handling code in this area. The code is condensed into a more concise and documented form. [Original patch from Ed with tweaks from RP] [YOCTO #7586] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29lib/oe/package: Improve strip subprocess handlingRichard Purdie1
Currently if the strip process fails, we get a message but don't know why. This adds code to show the return value and any error output. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29lib/oeqa: Add sdk tests for gcc/perl/pythonRichard Purdie4
Enhance testing of the generated SDK tarballs by adding tests for gcc/perl/python based on the existing runtime tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-29oeqa/targetbuild: Use DL_DIR for downloads if presentRichard Purdie1
This isn't a test of wget so if the files we need are present in DL_DIR, use them from there and save a bit of speed/bandwidth and skip the wget. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: add: use correct bbappend file name with -V optionPaul Eggleton1
We weren't adding the version into the bbappend file name when -V was specified which meant that building or resetting failed. Also adjust one of the tests so that we're testing devtool add both with and without this option. Fixes [YOCTO #7647]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: handle . in recipe namePaul Eggleton1
Names such as glib-2.0 are valid (and used) recipe names, so we need to support them. Fixes [YOCTO #7643]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: include bbappends in recipe parsingMarkus Lehtonen1
In order to get correct metadata, SRCREV for example. Fixes [YOCTO #7648]. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27oe-selftest: devtool: fix test_devtool_update_recipe_gitPaul Eggleton1
Make this test work after recent changes to the mtd-utils recipe, and hopefully make it robust against any future changes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27oeqa/selftest/toaster: fix bad indentCristian Iorga1
oe-selftest --list-classes crashes because of a bad indent (see below). Fixed. systemd-build$ oe-selftest --list-classes 2015-04-22 12:12:39,141 - selftest - INFO - Listing all available test modules: _sstatetests_noauto (hidden) -- SStateBase -- RebuildFromSState -- -- test_sstate_rebuild -- -- test_sstate_relocation _toaster (hidden) Traceback (most recent call last): File "/mnt/Yo/yoctoproject/poky/scripts/oe-selftest", line 201, in <module> ret = main() File "/mnt/Yo/yoctoproject/poky/scripts/oe-selftest", line 157, in main modlib = importlib.import_module(test) File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/mnt/Yo/yoctoproject/poky/meta/lib/oeqa/selftest/_toaster.py", line 438 time.sleep(3) ^ IndentationError: unexpected indent Signed-off-by: Cristian Iorga <cristian.iorga@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-21oeqa/utils/decorators: Fix up frame selectionNathan Rossi1
Use threading to get the current threads ident to select the current executing frame to ensure that the scanned stack is the expected stack. This allows for the decorators to be used within a process that has multiple threads executing. Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-19lib/oe/package_manager: avoid traceback with no packagesPaul Eggleton1
If you were using deb packaging, had buildhistory enabled and produced an SDK that contained no packages in one of the sysroots (such as with uninative-tarball) then the do_populate_sdk was failing with a python traceback because there were no fields to split in the output line. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-15oeqa: Add test for layer append and FILESPATH trackingRichard Purdie1
This test actually tests a spectrum of bitbake functionality. Three layers are created, one containing a recipe, one with a bbappend adding a file to the recipe and another which overwrites the file in another bbappend. The correct outcomes in building the recipe are tested, with the file in the final layer added, removed and then re-added. This tests bitbake's cache handling as well as restoration from sstate which happens in the final test phase. Based on a test case from: [YOCTO #7019] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-13package_manager: RpmPM: Fix scriptlet for rpm 4Jean-Benoit MARTIN1
Path of rpm post install script is different between rpm 4 and 5 Signed-off-by: Jean-Benoit MARTIN <jean-benoit.martin@open.eurogiciel.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime: Boot test for poky-tiny.Lucian Musat1
Bug 6705. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/utils: Add runner for poky-tiny qemu.Lucian Musat1
The connection and commands are done via serial. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/targetcontrol: Add support for poky-tiny in QemuTarget.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/smart: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/rpm: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ping: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/parselogs: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ldd: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09testimage: sort modules based on dependenciesPatrick Ohly2
TEST_SUITES="auto" is useful to run all suitable tests without having to hard-code the list. However, it did not take test dependencies into account, which can be an issue for tests which really depend on some other test to run first. To fix this, modules get loaded in the order determined by TESTS_SUITES, but then get re-ordered based on dependencies derived from @skipUnlessPassed before running them. The original order is used to break ties when there are no dependencies, so reordering only occurs when really necessary. @skipUnlessPassed gets extended such that it makes the test name a method depends on available for inspection by the test loader in oetest.py. Unfortunately Python's unittest offers no API to inspect tests in a TestSuite, so the code has to rely on implementation details to find all tests. The worst that can happen when the implementation changes is that tests are not found and reordering does not happen. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/systemd.py: skip instead of failing without avahiPatrick Ohly1
The SystemdServiceTests assume that avahi-daemon is installed, which is not necessarily the case depending on the image being tested. Better check this dependency before starting the tests and skip them if the service is not installed. This has to be done for each test instead of for the entire module, because other tests in the module can run without avavi. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/oetest.py: use bb loggingPatrick Ohly1
Plain print has two drawbacks: - the output is handled by different processes, causing the message about found tests to appear randomly after the result of the initial tests - the output is not part of the bitbake console output Affects image testing with testimage.bbclass. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09lib/oeqa/selftest/bblayers: improve test for add-layer and remove-layerYi Zhao1
Use absolute path instead of relative path for meta-skeleton. Otherwise the case would fail if the builddir outside the poky source tree. Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-06oe.sstatesig: align swspec handling with sstate.bbclassChristopher Larson1
The logic tries to remove the -native suffix from pn to handle this (though it doesn't succeed, as it doesn't assign the new pn to the variable), but we need to do more for the swspec tasks than just not set the extrapath, we also need to change from SSTATE_PKGSPEC to SSTATE_SWSPEC. Alter to correct the spec for these cases, and also add preconfigure to align with the current logic in sstate.bbclass, which includes that task as well in the list of tasks to adjust to use swspec. [YOCTO #7563] Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-06meta/lib/oe/utils.py: Corrected the return value of both_contain()Jun Zhu1
oe.utils.both_contain() should return the result as "checkvalue" or "", but the latest implement returns as "set(['checkvalue'])" or ""; It causes that bitbake.conf generates the wrong result of COMBINED_FEATURES, which contains the common components in both DISTRO_FEATURE and MACHINE_FEATURES. For example, build in Dizzy branch, COMBINED_FEATURES is "alsa usbhost ...", but recently, COMBINED_FEATURES is like "set(['alsa']) set(['usbhost']) ...". Signed-off-by: Jun Zhu <R01007@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-02package_manager: call createrepo with --dbpath pointing inside WORKDIREd Bartosh1
Rpm database in staging area is used only by createrepo. createrepo fails with the error "rpmdb: BDB0060 PANIC: fatal region error detected" if rpm database is broken during previous run of createrepo. Made createrepo to create rpm db in $WORKDIR/rpmdb/ from scratch for every build and architecture. This should potentially fix the failure as every run of createrepo will be using separate db. [YOCTO #6571] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-24package_deb: DpkgIndexer use apt-ftparchive that needs a valid configAndreas Oberritter1
Since we have support of log checking in deb/ipk [1] rootfs generation in non Debian-based hosts fails because apt-ftparchive generates a Warning when not find /etc/apt/apt.conf.d/ (available in Debian-based hosts). package_manager.py: DpkgPMIndexer add support for generate and export apt configuration. [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=86aec93902af2e2d7d73ca9a643707fcca45055c Signed-off-by: Andreas Oberritter <obi@opendreambox.org> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>