summaryrefslogtreecommitdiff
path: root/meta/classes/package.bbclass
AgeCommit message (Collapse)AuthorFiles
2014-04-08package.bbclass: Add CONFFILES to list of package specific variablesRichard Purdie1
Changes to CONFFILES should change the sstate checksum. To make that happen, it needs to be listed in the list of package specific variables, therefore add it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-24package: Drop do_package_write taskRichard Purdie1
The reasons this task was introduced are lost in the mists of time. It allowed for the a single "package_write" task instead of spelling out the explicit package backends, however in all but one case we do that anyway. As such as might as well give in and delete the task, converting that single reference into explicit dependencies. This gives bitbake a bit less work to to when processing the runqueue since there are less tasks (but more dependencies in some cases). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17package.bbclass: do_split_packages should always return something.Josua Mayer1
So far, when do_split_packages is passed a non-existing folder, it will return nothing. While building Mesa with PACKAGECONFIG="egl gles" for qemux86, do_package threw a nice exception on a line reading "pkgs += do_split_packages": "Exception: TypeError: 'NoneType' object is not iterable" I think a function should always return something, and in this specific case, returning an empty list should be right. This patch simply fixes the return statement to do just that. Signed-off-by: Josua Mayer <josua.mayer97@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-02-12package: Remove the before do_build constraint of the do_package taskRichard Purdie1
Currently, if you bitbake xxxx, it will restore the package_write_, packagedata, populate_sysroot *and* the package data from sstate. We generally care about the packagedata and the actual packages, we no longer need/care about the do_package data itself unless something is going to repackage. By relaxing this constraint (and relying on the packagedata target), we can save things being installed from sstate and hence speedup build times. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11package.bbclass: fix strip and split logicKoen Kooi1
Marks original commit message and variable documentation state that stripping and splitting are independent of eachother, but package.bbclass ANDs the two INHIBIT flags to see which files can be stripped and/or split. Original behaviour: INHIBIT_PACKAGE_STRIP: no strip, no debug split INHIBIT_PACAKGE_DEBUG_SPLIT: no strip, no debug split Behaviour after this patch: INHIBIT_PACKAGE_STRIP: no strip, no debug split INHIBIT_PACKAGE_DEBUG_SPLIT: strip, no split Signed-off-by: Koen Kooi <koen.kooi@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29package.bbclass: Show which files require given dependency in debug outputMartin Jansa1
* when log.do_package shows some unexpected dependency, people usually need to grep package directory to find which binary was creating that dependency, show it directly in the debug output Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29package.bbclass: Don't search for providers of PRIVATE_LIBSMartin Jansa1
* split PRIVATE_LIBS and don't use find(), so that libfoo cannot be found in PRIVATE_LIBS = "libfoobar" Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28package.bbclass: add getter of additional metadataLeonid Borisenko1
Two variables are searched for value of additional package metadata: * PACKAGE_ADD_METADATA_<PKGTYPE> * PACKAGE_ADD_METADATA First found variable with defined value wins. <PKGTYPE> is a parameter of getter and expected to be a distinct name of specific package type. For example: 'DEB' or 'RPM'. Variable can contain multiple [one-line] metadata fields, separated by literal sequence '\n'. Separator can be redefined through variable flag 'separator'. Getter returns found value with separator replaced with newline character. As side-effect, searched variables acquired flags 'type' (equals to 'list') and 'separator'. Signed-off-by: Leonid Borisenko <ive.found@gmail.com> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-01-19package.bbclass: show warning when package is providing already provided shlibMartin Jansa1
* move read_shlib_providers before registering package as provider and show warning when different package tries to provide something already provided. [YOCTO #4628] Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-19package.bbclass: move reading shlibs providers to separate functionMartin Jansa1
* prepare for reading shlibs providers only from dependency tree of current recipe [YOCTO #4628] Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10package: Keep global pkgfiles[] up to date when snapping library symlinksPhil Blundell1
Since ca86603607a69a17cc5540d69de0e242b33382d3 we are now calling stat() on all the pkgfiles[] during emit_pkgdata(). If symlink snapping has removed some of the files then we will blow up trying to stat a path that no longer exists. Fix that by ensuring that pkgfiles[] is updated when we process the list of library renames. Signed-off-by: Phil Blundell <pb@pbcl.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02classes/package: set SUMMARY in do_split_packages()Paul Eggleton1
do_split_packages() is commonly used to split out plugin/module packages dynamically within recipes. If it doesn't set SUMMARY for each of these packages then they get a generic SUMMARY from the recipe, which isn't particularly useful; so add a parameter to set this and default it from the current description parameter (it ought to have been the other way around, but the description parameter is what we currently have in use by all recipes that use this function.) Fixes [YOCTO #5406]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-12-18sstate: Get rid of crazy name mappingRichard Purdie1
When originally developed, it was thought a task may have more than one associated sstate archive. The way the code has grown that idea is now not possible or needed. We can therefore assume one sstate archive per task and drop the crazy name mapping code. Simpler is better in this case. The downside is that various sstate archives will change name so this forces a cache rebuild. Given the other sstate changes going in at this time, this isn't really a bad thing as things would rebuild anyway. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03classes/package: write PE and PKGE out to pkgdataPaul Eggleton1
These are important parts of the version for every package, so we should include them in PKGDATA just as we include PV/PR/PKGV/PKGR. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03classes/package: record PKGSIZE as total file size in pkgdataPaul Eggleton1
We were using "du -sk" to collect the total size of all files in each package for writing out to PKGSIZE in each pkgdata file; however this reports the total space used on disk not the total size of all files, which means it is dependent on the block size and filesystem being used for TMPDIR on the build host. Instead, take the total of the size reported by lstat() for each packaged file, which we are already collecting for FILES_INFO in any case. Note: this changes PKGSIZE to be reported in bytes instead of kilobytes since this is what lstat reports, but this is really what we should be storing anyway so that we have the precision if we need it. Fixes [YOCTO #5334] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03classes/package: fix FILES_INFO serialisation in pkgdataPaul Eggleton1
The FILES_INFO entry in each pkgdata file stores the list of files for each package. Make the following improvements to how this is stored: * Store paths as they would be seen on the target rather than erroneously including the full path to PKGDEST (which is specific to the build host the package was built on) * For simplicity when loading the data, store complete paths for each entry instead of trying to break off the first part and use it as the dict key * Record sizes for each file (as needed by Toaster) * Serialise the value explicitly using json rather than just passing it through str(). Fixes [YOCTO #5443]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-04package.bbclass: Fix split_and_strip_files when file has single quote (')Felipe F. Tonello1
Fix false error report when a file that has a single quote by escaping the single quote. Some packages might install files with quotes, such as music files and other types, that will cause the problem. Signed-off-by: Felipe F. Tonello <eu@felipetonello.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-10-14classes/package: handle filenames containing wildcardsPaul Eggleton1
It is uncommon, but it is possible for upstream sources to contain files that have wildcard characters in their names (Webmin is an example). Because we were running glob.glob() on every entry in the list of entries in FILES and then adding the result to the files list to be processed, the process would loop infinitely if files whose names contained wildcard characters were present. Fix this by avoiding re-processing the output of glob.glob() with itself, and also "escape" wildcard characters in FILES entries added automatically from do_split_packages(). Fixes [YOCTO #1676]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-11classes: tar 1.27 fixesRichard Purdie1
tar version 1.27 returns: tar: --same-order option cannot be used with -c with the commandlines we have been using. We can remove the -s option (which is --same-order) to remove the error. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-26package.bbclass: Clear umask when using os.mkdirRichard Purdie1
We switched to using os.mkdir with the file creation mode specified as the second parameter. Python masks this with umask behind the scenes which isn't what we want, we really want the permissions we specify. To avoid this we zero the umask beforehand and restore afterwards. Other solutions are possible but would not perform as well which is why we're using os.mkdir in the first place. Martin Jansa deserves the credit for debugging where the problem was. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-13bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific ↵Richard Purdie1
directory Currently we have a hierarchy of pkgdata directories and the code has to put together a search path and look through each in turn until it finds the data it needs. This has lead to a number of hardcoded paths and file globing which is unpredictable and undesirable. Worse, certain tricks that should be easy like a GL specific package architecture become problematic with the curretn search paths. With the modern sstate code, we can do better and construct a single pkgdata directory for each machine in just the same way as we do for the sysroot. This is already tried and well tested. With such a single directory, all the code that iterated through multiple pkgdata directories and simply be removed and give a significant simplification of the code. Even existing build directories adapt to the change well since the package contents doesn't change, just the location they're installed to and the stamp for them. The only complication is the we need a different shlibs directory for each multilib. These are only used by package.bbclass and the simple fix is to add MLPREFIX to the shlib directory name. This means the multilib packages will repackage and the sstate checksum will change but an existing build directory will adapt to the changes safely. It is close to release however I believe the benefits this patch give us are worth consideration for inclusion and give us more options for dealing with problems like the GL one. It also sets the ground work well for shlibs improvements in 1.6. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06package.bbclass: skip already-stripped QA test if asked forKhem Raj1
Some packages like grub have already stripped binaries e.g. ERROR: QA Issue: File '/boot/grub/kernel.img' from grub was already stripped, this will prevent future debugging! ERROR: QA run found fatal errors. Please consider fixing them. We would like to have a possibility to skip it using something like INSANE_SKIP_${PN} = "already-stripped" This adds the logic to do so it acts at PN level and not at package level. so something like INSANE_SKIP_${PN}-misc = "already-stripped" wont work. Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-04package.bbclass: Fix darwin shlib handlingRichard Purdie1
shlibs dependency calculations on darwin we not functioning correctly, we need to process the filename without the complete path. If we don't, "." characters in the path cause problems. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-22package.bbclass: Fix handling of symlinks in debug packagesRichard Purdie1
When copying the sources for the debug source package we use cpio -Ll which means to copy files as hardlinks and to dereference symlinks. It appears there is a bug in cpio since -Ll will copy symlinks and not dereference them. We therefore do a second pass over copied symlinks resolving them into files. Ideally we would copy these as hardlinks as well however it doesn't seem worth the extra code and effort for what amounts to a corner case for a minor space improvement. This means that the -dbg packages no longer contain broken symlinks. [YOCTO #5020] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-25package.bbclass: avoid packing debug sources from other packagesAndreas Müller1
The error case was gcc-dbg packing files from libgcc-dbg reported in [1] [1] http://lists.openembedded.org/pipermail/openembedded-core/2013-July/080728.html Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-07-02classes/package: print command output when commands failPaul Eggleton1
When external commands such as debugedit fail, it can be useful to see their output, so use oe.utils.getstatusoutput() instead of subprocess.call() to capture this and print it on failure. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-06-28package: Ensure we iterate all the pool objectsRichard Purdie1
There is the possibility that if we don't iterate through the multiprocessing pool objects we might not catch return codes and this could lead to hung/zombie processes either temproarily or on a wider scale. Adding this certainly doesn't hurt anything and is better practise so we might as well do it. Its not 100% clear if this fixes some issues or not. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07insane/package: refactor packaging sanity testsRobert Yang1
Refactor packaging sanity tests from package.bbclass to insane.bbclass so that the message can respect WARN_QA (print the warning message and go on the task) and ERROR_QA (print the error message and fail the task). - For the bb.warn(), give it a message name and add it to WARN_QA, then use package_qa_handle_error() to handle it. - For the bb.error(), give it a message name and add it to ERROR_QA, then use package_qa_handle_error() to handle it. - All the bb.warn() and bb.error() have been replaced in package.bbclass. - A few bb.warn() and bb.error() in insane.bbclass have been kept since they can not be replaced or doesn't have to, for example the bb.error() in package_qa_check_license(), it will print the error message and then invoke bb.fatal() to fail the task, I think that we don't have to replace it with package_qa_handle_error(). - Put all the WARN_QA and ERROR_QA in one line, so that they can be redefined by the user easily. [YOCTO #3190] [YOCTO #4396] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-06-07insane/package: let package.bbclass inherit insane.bbclassRobert Yang1
RP's comment: "What we're trying to do is move everything to use a standard mechanism for reporting issues of this type (do_package). With insane.bbclass, you can elect whether a given type of error is a warning or error and fails the task." * The package.bbclass had used package_qa_handle_error() which is from insane.bbclass, and we will use it for handling other warnings and errors, so let package.bbclass inherit insane.bbclass, this change will make the insane as a requirement (always included). * Change the "PACKAGEFUNCS ?=" to "+=", otherwise there would be an error like: Exception: variable SUMMARY references itself! This is because we let package.bbclass inherit insane.bbclass, and PACKAGEFUNCS has been set in insane.bbclass, so the "PACKAGEFUNCS ?=" will set nothing, then the "emit_pkgdata" doesn't run which will cause this error. * Add a QA_SANE variable in insane.bbclass, once the error type is ERROR_QA, it will fail the task and stop the build. [YOCTO #3190] [YOCTO #4396] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-05-16bbclass: bb.fatal() clean upRobert Yang1
The bb.fatal() is defined as: def fatal(*args): logger.critical(''.join(args)) sys.exit(1) So anything after bb.fatal() in the same code block doesn't have any effect, e.g.: bb.fatal("%s_%s: %s" % (var, pkg, e)) raise e The "raise e" should be removed. I searched all the files which use bb.fatal(), only the following 3 classes have this issues: insane.bbclass package.bbclass package_rpm.bbclass [YOCTO #4461] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-05-14package.bbclass: Fix sources contentsRichard Purdie1
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c376f1f49cea182a2887945840ab97a20970a373 fixed a valid issue where the sources file was accumulating information and subsequent task runs of do_packacge were not cleaning it. The fix is wrong however since we're removing the file within a loop. This fix removes the file outside the loop ensuring it is not truncated and contains the correct information. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09class/lib: Fix up various file access methodsRichard Purdie1
There are various bits of cruft that have built up around our file accesses. This patch cleans some of them up, specifically: * Remove pointless "from __builtin__ import file" * Use open(), not file() * Wrap file usage in a with container to ensure files are closed * Add missing .close() calls in some cases Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09classes/lib: Fix getcmdstatus breakageRichard Purdie1
I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets add a wrapper and have this work in both worlds. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09sanity/patch.py: Remove commands module usageRichard Purdie1
The commands module is removed in python3. Use the subprocess module instead and the pipes module to replace the mkargs usage. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09clases/lib: Use modern exception syntaxRichard Purdie1
Update older code to use modern exception handling syntax which is the form accepted by python 3. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-22package.bbclass: ensure license excluded files aren't packagedChristopher Larson1
An excluded package left its files behind, which meant they could end up in another package instead, meaning we could ship GPLv3 binaries even with GPLv3 in INCOMPATIBLE_LICENSE. Skip the files belonging to the excluded packages to prevent this from occurring. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-04-22package.bbclass: add LICENSE_EXCLUSION to vardepsChristopher Larson1
Ensure that changes to INCOMPATIBLE_LICENSE re-run do_package for affected recipes. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-04-18package.bbclass: Add useradd variables to PACKAGEVARSRichard Purdie1
THe USERADD_PARAMS and similar group variables are package specific variables which should get added to the vardeps of the packaging process. This change also ensures they get remapped correctly by the multilib code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-17package.bbclass: Fix populate_packages for glob expansion issuesRichard Purdie1
If we put a valid glob like "*/foo/*" into FILES, populate_packages breaks with a "file exists" message. This is because the glob expansion does not have "./" prefix however there may already be an entry in the seen list which does have such a prefix. The easiest/simplest fix right now is to add the prefix if it doesn't exist which only happens for certain globs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-13package/image.bbclass: Fix multilib rprovidesRichard Purdie1
allarch multilib recipes are meant to provide a list of different multilib variants. Unfortunately since the pkgdata also has mappings for these, they get mapped back to the original package name which means the effect is undone at package creation time when the remapping code is called. This patch adds in a conditional to break that chain meaning the packages get the correct RPROVIDES and image builds work correctly with opkg. [YOCTO #3453] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-04Add file information to package information windowAndrei Dinu1
Removed the package files parsing routine from the packageinfo.bbclass file and added it to the package.bbclass file. Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-26package.bbclass: Correctly handle /usr/src/debug file ownershipRichard Purdie1
After the recent cached path operation improvements, we need to tell the cache about directory changes. /usr/src/debug was was cached as not created and this was leading to bugs in file ownership. This change updates the cache and avoids this issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-25package.bbclass: Ensure debug source file is generated correctlyRichard Purdie1
The command for generating debug sources appends to the file, therefore we need to ensure it does not exist before we run the command else we end up doing more work that we should have to. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-25package.bbclass: Handle subprocess errors correctlyRichard Purdie1
If an error occurs in subprocess.call() we currently don't catch it. In particular we have issues where debugedit is segfaulting unnoticed. This fixes up various code paths to catch the errors. [YOCTO #4089] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-18package: Add cachedpath optimisationRichard Purdie1
Currently, various standard library operations like os.walk(), os.path.isdir() and os.path.islink() each call stat or lstat which involves a syscall into the kernel. There is no caching since they could conceivably have changed on disk. The result is that for something like the do_package task of the kernel we're spending over two minutes making 868,000 individual stat calls for 23,000 files. This is suboptimal. This patch adds lib/oe/cachedpath.py which are a set of replacement functions for these operations which use cached stat data rather than hitting the kernel each time. It gives a nice performance improvement halving the build time of the kernel do_package. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-15package.bbclass: add debug-without-src PACKAGE_DEBUG_SPLIT_STYLEMartin Jansa1
* same as original and default version, but does not package source files in PN-dbg Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-03-05package.bbclass: Ensure all .so files get strippedRichard Purdie1
It was realised that .so files which were not marked as executable were not gettings stripped. This was wasting space in images. This patch ensures they do get processed by the code correctly. [YOCTO #3973] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05package.bbclass: Fix do_package variable dependenciesRichard Purdie1
When the do_package function was split into different sections, the vardeps were not updated to match meaning some function changes weren't reflected in the sstate checksums. Fix this. [YOCTO #3980] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-22pakcage.bbclass:use a better way to cut -dev/-dbg suffixSong.Li1
In pakcage.bbclass,function replace('-dev', '') is used to cut '-dev' suffix. but it will cause the name "lib32-device-dev" to "lib32ice", so change it to [:-4]. Signed-off-by: Song.Li <Song.Li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-17classes: Drop none package specific packaging variable accessesRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>