summaryrefslogtreecommitdiff
path: root/meta/lib/oe
AgeCommit message (Collapse)AuthorFiles
2017-03-23package_manager.py: respect OPKGLIBDIRMartin Jansa1
* respect it for incremental rootfs generation * add lists_dir option to opkg.conf * also fix setting info_dir and status_file when they use default value, the problem is that self.opkg_dir is already prefixed with rootfs directory, comparing it with /var/lib/opkg always returned false and the options were appended to config file unnecessary * with opkg 0.3.4 we can use VARDIR prefix added in: commit d2a8e23dc669adc398f4bb8bcfcabfcf925708f7 Author: Florin Gherendi <floring2502@gmail.com> Date: Mon Dec 19 12:25:38 2016 +0200 libopkg: make the /var and /etc directories configurable at compile time. Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23rootfs.py: Respect OPKGLIBDIR variableMartin Jansa1
* when OPKGLIBDIR doesn't have the default /var/lib value it will silently fail to copy package database from normal rootfs to debugfs rootfs and then when trying to install *-dbg complimentary packages it won't install anything, because installed_pkgs.txt file generated from debugfs is empty Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22sign_rpm.bbclass: do not set/use RPM_GPG_PUBKEYAlexander Kanavin1
This is entirely unnecessary (we can ask the signer backend to export the key to a file when needed), and was causing confusing selftest failures due to the variable being set from two different places. [YOCTO #11191] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21meta: replace uses of bb.data.expand(VARNAME, d) with d.expand(VARNAME)Joshua Lock1
bb.data.expand(x, d) is deprecated API. [YOCTO #10678] Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17oe/path.py: fix for "Argument list too long"Robert Yang1
Issue: LIN9-1648 Fixed when len(TMPDIR) = 410: $ bitbake core-image-sato-sdk [snip] Subprocess output: /bin/sh: /bin/cp: Argument list too long ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Function failed: do_rootfs [snip] This is because "copyhardlinktree(src, dst)" does "cp -afl src/* dst", while src/* is expanded to "src/file1 src/file2, src/file3..." which causes the "Argument list too long", use ./* as src and change cwd in subprocess.check_output() to fix the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-17package_manager: fix "exlcude" vs. "exclude" typoPaul Gortmaker1
Which results in: -------------------------------- 0557: package_exclude = self.d.getVar('PACKAGE_EXCLUDE') *** 0558: exclude_pkgs = (bad_recommendations.split() if bad_recommendations else []) + (package_exlcude.split() if package_exclude else []) 0559: 0560: output = self._invoke_dnf((["--skip-broken"] if attempt_only else []) + 0561: (["-x", ",".join(exclude_pkgs)] if len(exclude_pkgs) > 0 else []) + 0562: (["--setopt=install_weak_deps=False"] if self.d.getVar('NO_RECOMMENDATIONS') == 1 else []) + Exception: NameError: name 'package_exlcude' is not defined ERROR: cube-builder-initramfs-1.0-r0 do_rootfs: Function failed: do_rootfs --------------------------------- Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17dnf: add /usr/bin/dnf symlink that points to /usr/bin/dnf-2Alexander Kanavin1
All documentation refers to dnf binary as 'dnf' yet make install does not create one - it's done by Fedora's spec file when building the rpm. Let's replicate this behavior. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17lib/oe/package_manager: import rpm signing key to rpmdbMarkus Lehtonen1
Import the gpg key used in rpm signing into rpmdb. This makes it possible again to create images when rpm signing is enabled. Also, instruct dnf to enforce signature check if rpm signing is enabled. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17rpm: add support for remote package feeds via PACKAGE_FEED_URIS variableAlexander Kanavin1
I've used a previous patch (which was never merged) by Humberto Ibarra <humberto.ibarra.lopez@intel.com> as a model for how to do runtime testing of this feature (e.g. we need to boot an image, run dnf on it, and check that it is indeed able to access the remote repo over http). Here's his original commit message: ===== Testing that feeds specified with PACKAGE_FEED_URIS var are set correctly has two parts. First a build with this var set is required, and then smart update needs to be issued in the running taget. The previous is not a common selftest practice because this is a simple test, but requires building and running a specific image, which takes a lot of time. testimage is not a good fit either, since the images tested there do not have the PACKAGE_FEED_URIS var set. For this test, the runtime-test module is being used, which is a selftest module but runs a testimage command. The var and test environment were set in runtime-perf.py and the actual test is done in a new testcase added to meta-selftest layer. ===== [YOCTO #10872] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14package: Fix various rpmdeps relocation issuesRichard Purdie1
There are several issues with rpmdeps after the rpm v5 -> v4 transition: * _rpmfc_magic_path is an invalid option for rpm4 * --rpmpopt is an invalid option for rpm4 * we need to use the path to rpmrc since otherwise it poitns at the original build path * we need to set MAGIC in the environment so libmagic can find its files. This patch addresses those and ensures rpmdeps works in relocated builds from sstate (or with rm_work). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13run-postinsts: simplify the logic of whether to install it to imagesAlexander Kanavin1
The logic is scattered all over the place, but amounts to "install, unless the rootfs is read only". Let's express that directly. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13gpg_sign.py: fix signing of rpm files using gpgAlexander Kanavin1
This means a) calling rpmkeys and rpmsign instead of rpm b) instructing gpg to run non-interactively; otherwise on my machine it pops up windows requesting a key passphrase Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13rootfs_rpm.bbclass: migrate image creation to dnfAlexander Kanavin3
To properly look at this patch, you probably need a side-by-side diff viewing tool. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13package_manager.py: improve the API for insert_feed_uris()Alexander Kanavin2
No need to store the configuration as class members, just pass it directly into the method. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-04lib/oe/package.py: remove @ from package nameAnders Darander1
@ isn't allowed in package names. Angular2 packages often have @ in their names. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01lib/oe/gpg_sign: fix rpm signing with gpg > 2.1Markus Lehtonen1
We need to check the gpg version and alter its command line options accordingly. [YOCTO #11054] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01lib/oe/gpg_sign: make gpg version a property of the signerMarkus Lehtonen1
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01lib/oe/gpg_sign: sign rpm packages in chunks of 100Markus Lehtonen1
Split the file list into chunks in order to avoid "OSError: [Errno 7] Argument list too long" This would happend when a package has huge amount of subpackages, e.g. glibc-locale. [YOCTO #11069] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-28kernel, license, sstate, rootfs.py: Remove deploy directory READMEMike Crowe1
It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt file in the deploy directory warrants the complexity it brings elsewhere. Let's just remove it entirely. In particular, if two do_image_complete tasks run in parallel they risk both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time. Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt file. In theory this should be safe because "cp -alf" will just cause one to overwrite the other. Unfortunately, coreutils cp also has a race[1] which means that if one copy creates the file at just the wrong point the other will fail with: cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to +‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680 Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23lib/oe/patch: Support xz-compressed patchesDominic Sacré1
Add .xz to the list of extensions recognized by patch_path(), so that compressed patches ending in .patch.xz or .diff.xz are automatically applied. (From OE-Core rev: f1a2c45765d14d3ca09657ad1f6b526554af2bb6) Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15classes: Drop now unneeded update_data callsRichard Purdie2
Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-09lib/oe/path: Add replace_absolute_symlinks functionRichard Purdie1
We need a function to walk a directory and replace absolute symlinks with relative ones. Add such a function to the path module of lib/oe. It does this relative to the directory being walked for files. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31meta: remove remaining True option to getVarFlag callsMing Liu1
This is a complementary fix to commit 2dea9e49: [ meta: remove True option to getVarFlag calls ] it intended to remove all True option to getVarFlag calls, but there are still some remaining. Search made with the following regex: getVarFlag ?\((.*), True\) Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23classes/oeqa: Replace subprocess.check_call() with check_output()Richard Purdie1
If you use subprocess.check_output() the traceback will contain the output when the command fails which is very useful for debugging. There is no good reason not to use this everywhere. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23package_manager: default to have scriptlet output captured in logChen Qi1
We need to have scriptlet output captured in log. If we don't do so, some useful information from scriptlets (especially postinstall script) would be missing. In case a script has a warning message but it does not necessarily have to fail, the message should be captured. Opkg has already done that. Change for rpm and dpkg so that scriptlet output is captured and no warning message is missing. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23oe/data: Add export2json functionAníbal Limón1
The export2json function export the variables contained in the data store to JSON format, the main usage for now will be to provide test data to QA framework. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-19lib/oe/utils: Add build_depends_string functionRichard Purdie1
This is useful when manipulating depends strings for task [depends] flags and is slightly easier to parse than some inline python. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19lib/oe/package_manager.py: Fix extract for ipk and debMariano Lopez1
With the move to use lists instead of strings in subprocess calls, package extraction was broken for ipk and deb. This fixes this issue. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16lib/oe/rootfs: reliably handle alternative symlinksAndré Draszik1
When removing unneeded packages from a (read-only) rootfs during rootfs creation, alternative symlinks from those packages may or may not be removed. The reason is as follows: update-alternatives(-native) is used during package installation as part of the image creation. It uses a database which contains entries for all the alternative symlinks possible, and the -native version uses the target's database by means of $OPKG_OFFLINE_ROOT, i.e. the rootfs we're in the process of creating. Once the rootfs has been created, OE removes certain packages because we have a read-only rootfs - in particular ROOTFS_RO_UNNEEDED which includes VIRTUAL-RUNTIME_update-alternatives, i.e. the update-alternatives. Recently, a change was made in OE, where uninstallation of update-alternatives from the rootfs causes removal of its database, too, to save space (700KiB (uncompressed) in a busybox system) b24a63d71b517af701dfedbc7f7b541d25af708f http://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb?id=b24a63d71b517af701dfedbc7f7b541d25af708f Following from that, if update-alternatives is removed from the target file system, update-alternatives-native has no database anymore, meaning it can't manage any of the alternative symlinks anymore. Because the order of packages to uninstall is non-deterministic, and update-alternatives could well be removed before any packages that use the mechanism provided, sometimes the extra symlinks are removed, sometimes not. By sorting the list of packages to be removed such that update-alternatives is removed last, we can ensure that that tings work reliably. (Certainly opkg seems to uninstall packages in the order given on the command line.) [YOCTO #10916] Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16rootfs: don't put /usr/lib/ssl and /etc into debugfsRoss Burton1
The /etc and /usr/lib/ssl directories were only put into the opkg-generated debugfs because of a bug in opkg which means that a conffile has to exist if we're running 'opkg status'. This is now fixed, so the workaround can be reverted. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09lib/oe.sstatesig: make locked sig file consistentJianxun Zhang1
Sort keys of dict 'types' prior to dumping, in order to have identical output every time. This could make it a little easier to diff these human-readable dumps. Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09lib/oe/path: add warning comment about oe.path.remove() with wildcarded ↵Paul Eggleton1
filenames Add a warning in the doc comment for oe.path.remove() about using that function on paths that may contain wildcards in the actual file/directory names. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie2
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22lib/oe/utils: Drop python2 compatibility codeRichard Purdie1
We've moved to python3, we don't need this compatibility code which just makes the code less readable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17lib/oe/package_manager: bail if createrepo can't be foundRoss Burton1
If createrepo isn't found then the errors later are mysterious, so explicitly check and error out early if it isn't there. (From OE-Core rev: e09636bbb3ea8ec58984197fd9c691bb908efe00) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16utils: Always use datastore's PATH for host_gcc_versionRoss Burton1
BUILD_CC may reference something like ccache and expect this to come from ccache-native, we at least have some selftests which assume this. Modify the code to use PATH when runnig BUILD_CC to ensure the tests continue to work as expected. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16meta: remove True option to getVarFlag callsJoshua Lock3
getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16meta: remove True option to getVar callsJoshua Lock20
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16buildhistory-diff: report directory renamesEd Bartosh1
The script detects directory renaming if two different directories with the same set of files are added and removed. [YOCTO #10691] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16uninative: rebuild uninative for gcc 4.8 and 4.9Ed Bartosh1
Some c++ libraries fail to build if uninative is built with gcc 5.x and host gcc version is either 4.8 or 4.9. The issue should be solved by making separate uninative sstate directory structure sstate-cache/universal-<gcc version> for host gcc versions 4.8 and 4.9. This causes rebuilds of uninative if host gcc is either 4.8 or 4.9 and it doesn't match gcc version used to build uninative. [YOCTO #10441] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16oeqa: move lib/oe tests to oe-selftestRoss Burton6
These tests don't get ran often (as demonstrated by the fact that some were not ported to Python 3), so move them to oeqa/selftest so they get executed frequently and can be extended easily. [ YOCTO #7376 ] Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14lib/oe/recipeutils: drop parse_recipe_simple()Paul Eggleton1
This was intended to be used with tinfoil, but tinfoil now has its own parse_recipe() method to do this which works properly in the memres case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14classes/patch: move several functions to oe.patchPaul Eggleton2
Move patch_path(), src_patches() and should_apply() to oe.patch, making them easier to call from elsewhere (particularly across the UI/server boundary). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14lib/oe/recipeutils: use cooker function instead of bb.providersPaul Eggleton1
We now have a function in cooker itself that can do this lookup; additionally, the rewritten tinfoil's cooker adapter has its own implementation that can work remotely, so if we use it then this function can work in that scenario as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14classes/image: suppress log_check mechanism for warnings/errors logged ↵Paul Eggleton1
through BitBake If you printed a warning through bb.warn() / bbwarn or an error through bb.error() / bberror, this was also being picked up by our log_check mechanism that was designed to pick up warnings and errors printed by other programs used during do_rootfs. This meant you saw not only the warning or error itself, you saw it a second time through log_check, which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the logger and add a handler that we can use to find out if any warning or error we find in the logs is one we should ignore as it has already been printed. Fixes [YOCTO #8223]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14lib/oe/rootfs: fix log_check warnings being printed twice with RPM packagingPaul Eggleton1
We were calling _log_check() in the RPM-specific rootfs class as well as in the base class; this is unnecessary and resulted in any errors/warnings generated during the actual package installation time triggering two warnings instead of one. Drop the call from RpmRootfs._create() to fix this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-08package_manager: remove strings and migrate to direct arraysStephano Cetola2
When using subprocess call and check_output, it is better to use arrays rather than strings when possible to avoid whitespace and quoting problems. [ YOCTO #9342 ] Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-30terminal.py: Pass string instead of bytes to ExecutionError to avoid exceptionMartin Vuille1
Based on run() in bitbake/lib/bb/process.py, ExecutionError() expects strings not bytes. Passing bytes results in a "TypeError: Can't convert 'bytes' object to str implicitly" exception. Fixes Bug 10729 Signed-off-by: Martin Vuille <jpmv27@yahoo.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23lib/oe/lsb: better handle missing fieldsJoshua Lock1
Some rolling release distros, such as Arch Linux, don't include a VERSION_ID field in their os-release file. Change release_dict_osr() to better handle this optional field being absent. Further improve the resilience of the release_dict_*() methods by always returning a dict and using dict.get() in distro_identifier() to supply a default, empty string, value when then key is missing. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23devshell: list commands when throwing NoSupportedTerminalsStephano Cetola1
When attempting to run devshell, if no terminal is available, the error being thrown was not very specific. This adds a list of commands that failed, informing the user of what they can install to fix the error. [ YOCTO #10472] Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-01 18:54:36 +0000