summaryrefslogtreecommitdiff
path: root/meta/lib/oe
AgeCommit message (Collapse)AuthorFiles
2019-01-28oeqa: Fix for QEMU_USE_KVMRobert Yang1
Fixed: MACHINE = "qemux86" QEMU_USE_KVM = "qemux86" IMAGE_CLASSES += "testimage" $ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs [snip] File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean raise ValueError("Invalid boolean value '%s'" % value) ValueError: Invalid boolean value 'qemux86' Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm will be enabled if target_arch == build_arch or both of them are x86 archs. (From OE-Core rev: 7c1a8a624cad8d967635c6cb5f99cf655bde3d44) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-12-06lib/oe/utils: Improve multiprocess_lauch exception handlingRichard Purdie1
We've seen a cryptic: "ERROR: Fatal errors occurred in subprocesses, tracebacks printed above" message from oe-selftest with no other traceback information. Improve the traceback logging to try and give a better indication of any errors that is ocurring. (From OE-Core rev: 521dd3d00979a27b6932e58d5497de68abac26e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-28package_manager.py: correct the deploydir when packagefeed-stability inheritedMingli Yu1
After create_packages_dir added in below commit: 85e72e1 package_manager: Filter to only rpms we depend upon When add below line into conf/local.conf INHERIT += "packagefeed-stability" There comes below error when do_rootfs Exception: FileExistsError: [Errno 17] File exists: '/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm' -> '/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm' def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencies): [snip] bb.utils.remove(subrepo_dir, recurse=True) [snip] In create_packages_dir function, there is a logic as bb.utils.remove(subrepo_dir, recurse=True) to clean subrepo_dir which is actually as example is /$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm. But currently when inherit packagefeed-stability class, the deploydir should be /$Prj/tmp/deploy/rpm-prediff, not the default /$Prj/tmp/deploy/rpm. If use /$Prj/tmp/deploy/rpm, then result in the logic as below: os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm") Update to the actual deploydir to guarantee the logic as below: os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm/i586/initscripts-1.0-r155.i586.rpm") Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18package_manager: handle remove() with no packagesRoss Burton1
If remove() is called with an empty package list, ensure we do nothing instead of calling the underlying package manager with an invalid command line. [ YOCTO #12900 ] Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-10-18package_manager: implement PACKAGE_EXCLUDE for opkgRoss Burton1
opkg has supported the --add-exclude option to install since 0.3.0, so use it to implement support for PACKAGE_EXCLUDE. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18rootfs: possibility to add packages to debug rootfsAndrej Valek1
Static libraries are not included in rootfs, it means, that sources are not going into debug rootfs. This option enables to install additional packages even if the standard package is not installed. Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11lib/oe/utils: add eol to format_pkg_list()grygorii tertychnyi via Openembedded-core1
Append '\n' to the non-empty formatted string before return. If you write it to the (manifest) file, it will ensure file ends with a newline. Many GNU utilities have problems processing the last line of a file if it is not '\n' terminated. E.g. if the last line is not terminated by a newline character, then "read" will read it but return false, leaving the broken partial line in the read variable(s). It can also break or adversely affect some text processing tools, that operate on the file. Signed-off-by: grygorii tertychnyi <gtertych@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25rootfs.py: Install -src packages when PACKAGE_DEBUG_SPLIT_STYLE is ↵Joshua Watt1
"debug-with-srcpkg" When the source packages are created, they need to be included in the debug image created when IMAGE_GEN_DEBUGFS = "1" for it to be usable for debugging. [YOCTO #12930] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25rootfs: always update the opkg indexIoan-Adrian Ratiu1
The previous logic assumed that if $BUILD_IMAGES_FROM_FEEDS=1 then a complete set of ipk feeds from which to build the image is already present under $IPK_FEED_URIS at do_rootfs runtime. $IPK_FEED_URIS usually contains "file://${DEPLOY_DIR_IPK}" which renders the above assumption bad because some recipes in the current build can contain code like do_install[nostamp] = "1" which will cause rebuilds bumping $PR and invalidating the index. Even when the index is manually re-created before an image build ("bitbake package-index"), the nostamp will cause failures because the dependency gets rebuilt before do_rootfs in the "bitbake <image>" call. So make the opkg rootfs index logic the same as for rpm/deb, to always update the index in $DEPLOY_DIR_IPK to fix the above nostamp failure. Feeds outside $DEPLOY_DIR_IPK added to $IPK_FEED_URIS continue to work as usual, for eg. by using a http:// URI. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21lib/oe/elf.py: Add LatticeMico32 architecture definitionNathan Rossi1
Add the ELF definition for the LaticeMico32 architecture. This architecture is 'elf' OS only as it does not target Linux. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-09-21package_manager: use normalized path when doing the filtered copyMax Krummenacher1
The linking/copying of the package files failes if the deploy dir is set in a non normalized way e.g. like this DEPLOY_DIR = "${TOPDIR}/../deploy" Then the simple string replacement which is used to calculated the link destination from the link source fails, as the link source is normalized but the deploydir prefix is not. Normalizing deploydir fixes this. Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-09-13insane: Recognise BPF as a valid EM_MACHINE typeKhem Raj1
BPF Linux ELF objects are generated with kernel-selftests with >= 4.18 kernel and when clang is enabled which packages BPF objects into packages, therefore recongnise this as a valid ELF target Add a selftest for BPF Do not flag BPF objects in target, since they pretty much will be ok for most of kernels architectures we care do support BPF Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-12oe/utils.py: Add vartrue()Robert Yang1
It can be used to simplify code like: "${@['iffalse', 'iftrue'][var]}" Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05populate_base_sdk: Stop running gcc --version all the timeRichard Purdie1
Running 'gcc --version' for every image recipe is slow and increases parsing time/resource usage for no good reason. Only compute the value in when we're really running the task/function. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05package_manager.py: add noarch to buildarch_compatKai Kang1
It fails to run rpmbuild to build a noarch package on target when it contains 'BuildArch: noarch' in the spec file: | error: No compatible architectures found for build Add 'noarch' to buildarch_compat in configure file rpmrc to fix it. Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30lib/oe/package_manager: turn postinst failure warnings into bitbake failuresAlexander Kanavin1
Sumo release provides a transition period so that deferrals to first boot via 'exit 1' can be converted to pkg_postinst_ontarget(). For the next release however, postinst script failures should be treated as such. [YOCTO #12607] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/patch.py: Clean up getstatusoutput usageRobert Yang1
We can't use subprocess.check_output() or subprocess.call() here since the one who invokes runcmd() needs handle CmdError() exception (error out or ignore it). Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/gpg_sign.py: Clean up getstatusoutput usageRobert Yang1
Replace usage of oe.utils.getstatusoutput() with direct subprocess calls. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/utils: Fix get_multilib_datastore to work for original tuneRichard Purdie1
Currently the original datastore returned by this function doesn't always work as the tune isn't set back to the original. Fix it to work like all_multilib_tune_list() in utils.bbclass and correct the data returned. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/package.py: use bb.utils.break_hardlinks helperRasmus Villemoes1
This does the same thing, but is more efficient in case st_nlinks is (already) 1. Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c . Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16oe.package_manager: support loading intercepts from multiple pathsChristopher Larson1
- if POSTINST_INTERCEPTS is set, use the listed intercept files, or - if POSTINST_INTERCEPTS_PATH is set, load from the listed paths, or - if POSTINST_INTERCEPTS_DIR is set, load from it (for compatibility), or - load from ${COREBASE}/meta/postinst-intercepts Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager: Add definitions for riscv machinesRichard Purdie1
Add definitions to the locale table for riscv architecture. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15sstate/lib.oe.path: Ensure file sparseness is preservedRichard Purdie1
Files when restored from sstate were missing their sparseness. Fix up various functions to preserve this and make things more deterministic. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager/sdk: Use filtered copies of the deploy ipk/deb directoriesRichard Purdie3
Similar to rpm, use copies of the ipk/deb directories for rootfs construction. This means the image creation code can no longer "see" recipes wich aren't in its dependency chain which is good for a variety of reasons including determinism, incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking performance. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager: Remove rpm specific pieces of create_packages_dir()Richard Purdie1
This function is generic, tweak the variable names and move out the rpm specific directory name to make it truly generic and reusable for deb/ipk. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15image: Add locale archive optimisationRichard Purdie2
Refactor the locale archive function from the SDK to also make it work during general image creation. This reduces the size of the locales from 900MB to 220MB in core-image-lsb-sdk. The exception handling around subprocess was dropped as the standard subprocess exception printing is better handled than the catchall exception. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-10lib/oe: Fix collections ABCs DeprecationWarning in Python 3.7+Khem Raj1
- Prefer collections.abc (new in Python 3.3) over collections for abstract base classes - In Python 3.8, the abstract base classes in collections.abc will no longer be exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes." - https://docs.python.org/3.7/whatsnew/3.7.html#deprecated - see https://github.com/python/cpython/commit/c66f9f8d3909f588c251957d499599a1680e2320 Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-09package: skip strip on signed kernel modulesfoocampo1
Executing strip action on kernel modules removes the signature. Is not possible to strip and keep the signature, therefore avoid strip signed kernel modules. Signed-off-by: Omar Ocampo <omar.ocampo.coronado@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-01multiconfig: Enable multiconfig dependencies on oe-coreAlejandro Enedino Hernandez Samaniego1
This patch enables multiconfig dependencies (mcdepends) to be used on recipes using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to bitbake.conf to enable mcdepends, and to sstatesig and staging.bbclass to avoid conflicts between packages from different multiconfigs. [YOCTO #10681] Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26lib/oe: split out machine to ELF data dictionaryRoss Burton1
2018-07-26logging: use warning instead warnChen Qi1
The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-24lib/oe/utils: Drop now unused multiprocess_execRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package/package_manager: multiprocess_exec -> multiprocess_launchRichard Purdie2
After this replacement, the parent exception handling works so we don't need subprocess wrapping with bb.error in the underlying functions. The underlying contexts also have better module handling so the imports can be cleaned up. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Call file to determine elf status in parallelRichard Purdie1
This allows the calls to is_elf (which calls file) to happen in parallel allowing a speedup of do_package and do_populate_sysroot for native recipes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Refactor to remove isElf/is_elf function duplicationRichard Purdie1
There are probably further cleanups needed here but this at least removes the major code duplication between these two similar funcitons, keeping the kernel module ".ko" extension check for efficiency to avoid opening and reading file contents in the general case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Don't use subshell to execute fileRichard Purdie1
We don't need any functionality from the shell here, its just extra fork overhead. Therefore remove it and use subprocess directly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24utils: Add multiprocess_launch API and testcaseRichard Purdie1
The current methods of spawning processes for parallel execution have issues around collection of results or exceptions. Take the code from package_ipk/deb, make it generic, add a results collection mechanism, fix the exception handling and for it into a standard library function. Also add a test case which tests both the success and failure modes of operation to stop this functionality regressiing again. In particular, compared to multiprocess_exec, this fork off the parent approach means we can pass in the datastore and functions work in the same scope as the parent. This removes some of the complexities found trying to scale multiprocess_exec to wider use. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18classes/buildhistory: handle packaged files with names containing spacesPaul Eggleton1
The FILELIST field of the package info file in the buildhistory repository is a space-separated list of all of the files in the package. If a name of a file packaged by a recipe contains a space character then of course the result was that we didn't handle its name properly. To fix that, use quotes around any filename containing spaces and at the other end use these quotes to extract the proper entries. Fixes [YOCTO #12742]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18lib/oe/buildhistory_analysis: drop related field featurePaul Eggleton1
The original idea here was that changes to certain fields might be able to be explained if there was a change to another field, for example if RDEPENDS changed it might be because DEPENDS changed. Thus we were printing this kind of thing out with each change. Unfortunately in practice this turned out to be noisy and not particularly useful, so we might as well remove it. Fixes [YOCTO #7336]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-07oe/copy_buildsystem.py: add SDK_LAYERS_EXCLUDE_PATTERNRobert Yang1
It is helpful when exclude a lot of layers. It uses python re, and supports multiple patterns (separated by space). Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06sstatesig: Remove trailing whitespaceJoshua Watt1
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05oe.types.boolean: treat None as FalseBinghua Guan1
It is better to return False for None. E.g. checking an undefined variable returned d.getVar(). Signed-off-by: Binghua Guan <freebendy@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-02oe.path: add which_wild functionChristopher Larson1
This is a function much like shutil.which or bb.utils.which, retaining shutil.which-like function semantics, bb.utils.which's support for returning available candidates for signatures, and most importantly, supports wildcards, returning only the first occurrance of each found pathname in the search path. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-02lib/oe/sstatesig: Fix task mappings from multilib<->non-multilib contextsRichard Purdie1
If we're in a multilib context already and want a non-multilib context this function returned incorrect values. Try and retain optimisations for the common case not needing to request a datastore but allow the different multilib/non-multilib combinations to work too. This fixes bugs where rootfs generation of a multilib image would write into incorrect locations, or be unable to find sstate manifest files due to incorrect data stores being used to expand data. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02lib/oe/utils: Improve get_multilib_datastoreRichard Purdie1
Currently this function assumes that no multilib is applied and that we're applying a multilib. This means if we're in multilib context and want the non-multilib context we can't obtain it (and no other function exists for this either). Improve the function to allow this to be requested. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02postinst-intercepts: do not execute any variant of delay_to_first_bootJoe Slater1
As of commit 2c5c6e3ff we create multilib variants of intercept hooks but we did not account for delay_to_first_boot variants. This was covered up until commit a335e7867, but will now cause an error. Signed-off-by: Joe Slater <joe.slater@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21oe/types: Allow boolean to accept an existing booleanRichard Purdie1
Exception: TypeError: boolean accepts a string, not '<class 'bool'> is a bit annoying if you pass in True/False. Tweak the function to make it forgive that situation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18package_manager.py: rework postinst_intercept failuresAlexander Kanavin2
Previously a warning was printed regardless of context and nature of the failure, and because it was only a warning, it was mostly ignored. Now, the following is considered when a failure happens: 1) whether we are installing packages into a target image, or populating a SDK with host or target packages. 2) whether the failure was due to qemu not supporting the target machine. Accordingly, warnings, notes, and failures are printed, and postponing to first boot happens if possible. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18rootfs.py: for dpkg/opkg, don't install postinsts if package management is ↵Stefan Agner1
present If package management is present opkg/dpkg will bring the original copy of the postinsts scripts with the metadata and will be able to handle postinsts just fine. In fact, it is preferred to let package management handle the postinsts scripts in this case since it will keep the package managers database up-to-date too. The run-postinsts scripts will make sure the package manager gets invoked instead of the scripts directly. Note: Before commit 5aae19959a44 ("rootfs.py: Change logic to unistall packages") rootfs.py did not install /etc/$pm-postinsts too. It is not clear whether that change was intentionally or just a bug. This commit fixes/reverts that aspect of the commit. Signed-off-by: Stefan Agner <stefan.agner@toradex.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18populate_sdk_ext.bbclass: fix corebase identificationDamien Riegel1
When generating the extended SDK, there is a copy step where this class goes through the layers and other stuff that have been copied to generate the SDK. The corebase; ie. the folder that contains the core layer 'meta' is treated in a special way. Unfortunately in our tree, we have: sources/meta/meta | `- core layer `------- corebase In populate_sdk_ext's copy_buildsystem, the heuristic to determine which element of the list returned by copy_bitbake_and_layers is corebase is fooled by such layout. In copy_bitbake_and_layers, corebase is already handled specifically and reliably, so we should let that function tell us which folder is corebase instead of trying to determine it. To do so, change the return type of copy_bitbake_and_layers to a tuple that contains (corebase, copied_layers). It also simplifies the code on the caller side. Signed-off-by: Damien Riegel <damien.riegel@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-20 16:33:01 +0000