| Age | Commit message (Collapse) | Author | Files |
|---|
|
When calculating PKGSIZE we sum the size of files after doing lstat() so we
don't count directory metadata overhead, but were not correctly handling
hardlinks. This results in packages such as e2fsprogs-mke2fs having PKGSIZE of
1.5M when it's actually a single 300K binary with five hardlinks.
[ YOCTO #10423 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.
Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This sets a good example and avoids unnecessarily contributing to
perceived complexity and cargo culting.
Motivating quote below:
< kergoth> the *original* intent was for the function/task to error via
whatever appropriate means, bb.fatal, whatever, and
funcfailed was what you'd catch if you were calling
exec_func/exec_task. that is, it's what those functions
raise, not what metadata functions should be raising
< kergoth> it didn't end up being used that way
< kergoth> but there's really never a reason to raise it yourself
FuncFailed.__init__ takes a 'name' argument rather than a 'msg'
argument, which also shows that the original purpose got lost.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This is a non-functional change, which intends to correct element
names of a tuple returned by Popen.communicate().
Both in python2 and python3 subprocess.Popen.communicate() method
returns a tuple (stdoutdata, stderrdata), thus old assignments and
collateral comments are incorrect from human's point of view, however
formally there is no error in the code.
The change is desired to have to avoid copy-paste errors in future.
Signed-off-by: Vladimir Zapolskiy <vz@mleia.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
[YOCTO #9827]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
For example in a directory structure like this
.
├── symlink -> foo/bar
└── foo
└── bar
└── file
'file' could be referenced by specifying e.g. 'foo/bar/file' or
'symlink/file'. In cases like this populate_packages() might crash if
the file was referenced (in FILES) via the symlinked directory. The
outcome depends on how the user defined FILES_pn. This patch should
make the function behave more consistently. It looks for files which are
referenced via symlinked directories and handles them separately,
failing if their parent directory is a non-existent path. For example,
defining FILES_{PN} = "symlink/file" causes a build failure because
symlinks target 'foo/bar' is not included at all.
[YOCTO #9827]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
* The mode and owner info are saved in inode, hardlink won't change them,
so remove unneeded chmod() and chown().
* This can avoid the problem that when do_package re-run, the file's mode
maybe different if it is 0444 (changed to 0644 when re-run), this is
caused by pseudo adds 'w' on real file, and doesn't track linked source
when hard link, Peter and Mark may fix pseudo, but the removed code is not
needed, which can avoid the problem.
* To reproduce the problem, for example, version.c from gzip's ${B}:
1) bitbake gzip
2) Edit rpm-native or package.bbclass to make do_package re-run.
3) bitbake gzip
After the first build, build/version.c in gzip-dbg is 0444, but after
the second build, it will be 0644, this because do_package does:
$ ln ${B}/version.c gzip-dbg/version.c,
$ chmod 0444 gzip-dbg/version.c (it runs chmod 0644 on the real filesystem)
And in the second build, the gzip-dbg/version.c will be removed and
created again, so that stat() can't get 0444 but 0644 since
${B}/version.c is not tracked by pseudo.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This function uses chdir() heavily, so save and restore the cwd so that it
doesn't affect the system state.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order for strip and split to work together, we need to populate the
data structors if either split OR strip are not inhibited.
Original behaviour:
INHIBIT_PACKAGE_STRIP: no strip, no debug split
INHIBIT_PACKAGE_DEBUG_SPLIT: strip, no split
Behaviour after this patch:
INHIBIT_PACKAGE_STRIP: no strip, debug split
INHIBIT_PACKAGE_DEBUG_SPLIT: strip, no split
BOTH: no strip, no split, DNP data structures
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_split_package() constructs a list of packages that were created as it
iterates through the files, so if multiple files go into the same package then
the package will be repeated in the output.
Solve this by using a set() to store the created packages so that duplicates are
ignored.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Change fs_link_table to be keyed by path, just like fs_perms_table.
When a new entry is coming in for either table, remove any previous
entry for that path. This way later permission file entries override
earlier ones.
[YOCTO #9430]
Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When processing links, the directories are processed in unsorted order
which can result in cases like /var/lock -> /run/lock handled before
/var/run -> /run throwing an error for /var/run because /run already exists.
Change the link processing to ensure links are processed in sorted order of
the destination.
[YOCTO #9430]
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a recipe passes for example extra_depends="${PN}-dev" to do_split_packages()
then it isn't expanded, so the check for the multilib prefix doesn't work.
Solve this centrally by expanding extra_depends inside do_split_packages().
[ YOCTO #9381 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
-fdebug-prefix-map used
Tweak DEBUG_FLAGS to use "/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
as source target path in DWARF. While use gdb to debug binary, it could
work with sources in dbg package.
While -fdebug-prefix-map is used for compiling, we do not need invoking
debugedit to edit DWARF at do_package time, but list where sources files
are.
The copydebugsources uses the list to copy sources to dbg package. It
works whether -fdebug-prefix-map used or not.
[YOCTO #9305]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Typically in a node/npm compiled modules the module is named .node. This is a
binary module without a wrapper so it can actually be relatively large if
unstripped.
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This sets the scene for removing the default False for expansion
from getVarFlag. This would later allow True to become the default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g' -i `grep -ril getVar *`
In this case, the default was False, but True was used since in most
cases here expansion would be expected.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_split_packages is often called with parameters which need expansion.
This happens to work at the moment since python functions are expanded
before execution but likely will not happen in future and isn't good
code practise.
Expand the common parameters do_split_packages() to avoid regressions.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.
The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.
The solution is to check QA_SANE in places we use the handle_error
function.
[YOCTO #8907]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Creating FILES_${PN}-dbg is tedious and also pretty pointless. We might as well
assume ".debug" is a special directory name and split into -dbg automatically.
This change does so without changing the rest of the splitting logic too much.
It can be disabled for the cases where we really do want manual control of
the -dbg packages (e.g. qt4) with NOAUTOPACKAGEDEBUG = "1".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix some spelling mistakes in bbclass files
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
* there is PN at the beginning, then possibly long list of files and
at the end we don't see which recipe has this issue, add another
line which says which PN and how many files
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
PACKAGEVARS appended on this commit:
LICENSE SECTION
pkg_preinst
pkg_prerm
RREPLACES
GROUPMEMS_PARAM
SYSTEMD_AUTO_ENABLE
[Yocto #7754]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Currently the license exclusion code removes packages from PACKAGES
pre population of the package directories. This means that the FILES
entries for some packages are not seen and invariably results in
packaging errors.
Instead, remove the packages from PACKAGES post population of the packages
so the usual FILES entries work as expected but the file are not placed
into any packages and no packages containing embargoed licenses are generated.
This avoids errors from gcc-runtime with GPLv3 exclusion like:
ERROR: QA Issue: gcc-runtime: Files/directories were installed but not shipped in any package:
/usr/share
/usr/src
/usr/share/gcc-4.9.2
/usr/share/gcc-4.9.2/python
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. [installed-vs-shipped]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reverted 7c0fd561bad0250a00cef63e3d787573112a59cf
Created separate group of hardlinks for the files inside
the same package. This should prevent stripped files to be
populated outside of package directories.
This turns out not to be straightforward and has overlap with the
other hardlink handling code in this area. The code is condensed
into a more concise and documented form.
[Original patch from Ed with tweaks from RP]
[YOCTO #7586]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Based on the number of times I've had to explain it over the years it
seems that a lot of new users don't immediately realise what is meant
by "shipped" here (nor should we expect them to) so let's at least
mention packaging and briefly tell the user what they need to do. (I was
going to go into more detail in the message, but there is really more
detail than can be covered succinctly here.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Changes to SYSTEMD_SERVICE should change the sstate checksum. To make that
happen, it needs to be listed in the list of package specific variables,
therefore add it.
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow globs in CONFFILES.
This patch changes the way of CONFFILES handling. After this change,
the CONFFILES can take the same form as FILES. That means, we don't
have to list a bunch of files for CONFFILES. It will just be expanded
like the FILES variable.
We don't assume default value for CONFFILES in OE. But distro vendors could
provide a default value for CONFFILES in their distro configuration file
like below.
CONFFILES = "${sysconfdir}"
In this way, files under /etc are treated as configuration files by
default. Of course, setting CONFFILES in recipes take precedence over
the CONFFILES. For example, if the recipe author decides that package A
should only treat files under ${sysconfdir}/default/ as config files,
he/she can write like this.
CONFFILES_A = "${sysconfdir}/default"
[YOCTO #5200]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
PV is the package version as we need it to be during the build. PKGV is the
final version as it ends up in the package, and defaults to PV.
The packager handled builds without PR-server by replacing the AUTOINC string
in PKGV, but when the PR-server is being used, the script replaces the contents
of PKGV with the PV if the PV contains "AUTOINC". Thus the packager overrides
any change to PKGV the recipe might have made.
This breaks classes like gitpkgv that provide a correctly numbered PKGV, the
number as calculated by that class will simply be replaced with a 0-based index
from the PR-server.
This patch makes the packager look at the PKGV version instead of the PV, and
update the PKGV only based on the PKGV contents as set by the recipe.
See also the discussion here:
http://lists.openembedded.org/pipermail/openembedded-core/2015-January/100329.html
From investigating the history of the code and changes in the past year, the
use of "pv" instead of "pkgv" appears to be just an oversight, introduced in:
commit b27b438221e16ac3df6ac66d761b77e3bd43db67 "prs: use the PRServer to replace the BB_URI_LOCALCOUNT functionality"
A later commit 865d001de168915a5796e5c760f96bdd04cebd61 "package/prserv: Merge two similar functions into one"
silently fixed this only for the case without PR-server by using pkgv there.
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
* n is a tuple since this commit:
commit d3aa7668a9f001044d0a0f1ba2de425a36056102
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Mon Jul 7 18:41:23 2014 +0100
Subject package.bbclass: Improve shlibs needed data structure
since then 'n in private_libs' was always false and private libs
were always processed
* this is bad when we have libfoo in private libs, but also some package
providing libfoo, that way we ship own libfoo.so, but together with
runtime dependency on package providing libfoo
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This allows us to use this function elsewhere in the code.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A number of settings (DESCRIPTION, SUMMARY, postinst, postrm,
and appends to RDEPENDS) were made only if FILES_foo was not
set for a given package. If you had a modified glibc packaging
setup that was defining FILES_glibc-gconv-somelocale, this would
prevent the automatic append of glibc-gconv as a dependency,
because extra_depends was ignored.
I think the assumption may have been that if FILES_foo was set,
DESCRIPTION_foo and SUMMARY_foo would also be set, but it seems
to me that the right answer is probably to set them if they aren't
already set, and leave them alone if they are.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When we use ASSUME_SHLIBS,e.g.
ASSUME_SHLIBS = "libEGL.so.1:libegl-implementation"
then we end up with errors like below when using shlibs2 (dizzy+)
File: 'package_do_shlibs', lineno: 216, function: package_do_shlibs
0212: dep_pkg = dep_pkg.rsplit("_", 1)
0213: if len(dep_pkg) == 2:
0214: lib_ver = dep_pkg[1]
0215: dep_pkg = dep_pkg[0]
*** 0216: shlib_provider[l][libdir] = (dep_pkg, lib_ver)
0217:
0218: libsearchpath = [d.getVar('libdir', True),
d.getVar('base_libdir', True)]
0219:
0220: for pkg in packages.split():
Exception: KeyError: 'libEGL.so.1'
This is because the entry which is being populated does not exist
so lets create it if its not already there.
Change-Id: I9e292c5439e5d1e01ea48341334507aacc3784ae
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Having these two separate functions handling PR values seems pointless,
and worse, there are impossible code branches mixed within them.
Merge them into one function and tweak comments so at least you
don't have to read both functions to figure out what is going on.
This does restructure the conditionals to try and aid readability.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
dictionary
shlib2 code puts the information about path where a provider
will be found. e.g.
{'/usr/lib/llvm3.3': ('libllvm3.3-llvm-3.3', '3.3')}
This is obtained from new shlib2 pkgdata from
llvm3.3/3.3-r0/pkgdata/shlibs2/libllvm3.3-llvm-3.3.list
However when we search for NEEDED libraries we ignore the
key above which is the path where the provider library is installed
and instead just seach in libdir and base_libdir and hence
libraries which are not in above standard search paths gets
ignored even if they appear in DT_NEEDED sections
and a note is emitted
NOTE: Couldn't find shared library provider for libLLVM-3.3.so, used by
files: ....
IMO this note should actually become an error since if we do
not have all DT_NEEDED libraries in image the system is dysfunctional.
This patch extracts this libpath from key and add it to seach paths
when looing for a provider of a shared library
[YOCTO #6798]
Change-Id: Ie5f08632e37ba8d3439c8aaae33bc68b8996792f
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In case a package is not generated (is empty and does not has allow
empty flag set) the package data regarding reverse runtime dependency
shouldn't be done.
This were causing a false-positive in the meta-fsl-arm layer, when
building mesa, as:
,----[ Error during build of MX53 in meta-fsl-arm ]
| ERROR: The recipe mesa is trying to install files into a shared area
| when those files already exist. Those files and their manifest
| location are:
| /.../build/build/tmp/sysroots/imx53qsb/pkgdata/runtime-reverse/libopenvg-dev
| Matched in manifest-imx53qsb-amd-gpu-x11-bin-mx51.packagedata
| Please verify which recipe should provide the above files.
`----
Fixes [YOCTO: #6795]
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This quite coincidentally invalidates the sstate for do_package which
is needed due to the correction of oe.utils.multiprocess_exec().
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It doesn't really make sense to run the remapping code on these
variables. If specific renaming is needed, it should be applied
manually. This means that the debian RPROVIDES of the original package
name can be preserved. There was also a bug report about this on the
OE-Core mailing list recently where someone else ran into this problem
too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For package groups, use the original package names and don't use the
remapped (e.g. debian) naming.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We may as well use the common function for this rather than
duplicating the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We need to scan binaries as well as libraries for dependencies.
Also ensure if its not an object file (as found by otool), we handle
this case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, the darwin shlibs detection is done by parsing the .la file
dependency fields. This is very old code and is incomplete in some
cases so convert to using otool -l and otool -L to correctly load
the rpath and dependency information.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the code was rewritten we forgot to strip the pkg that is present
in the pkgdest path. This was fixed in the linux version of the code
but not the darwin one, this matches the fix.
Without this, the provider paths are broken.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_package() calls package_name_hook so that e.g. debian-style renaming through
debian.bbclass can happen. If there is no class providing a package_name_hook
then this causes "WARNING: Function package_name_hook doesn't exist" every time
do_package() is executed.
Silence this warning by providing an empty package_name_hook in package.bbclass.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
