summaryrefslogtreecommitdiff
path: root/scripts/oe-pkgdata-util
AgeCommit message (Collapse)AuthorFiles
2016-09-03scripts: ensure tinfoil is shut down correctlyPaul Eggleton1
We should always shut down tinfoil when we're finished with it, either by explicitly calling the shutdown() method or by using it as a context manager ("with ..."). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-12oe-pkgdata-util: new option to provide full info for binary package(s)Alexander D. Kanevskiy1
New option can be used for displaying full information about binary package(s), including name, full version, recipe name, recipe full version and package size. This information can be useful inside custom buildhistory class to produce detailed image manifest for further analysis. List of packages can be specified as command line arguments or can be read from file (e.g. from existing image manifest). Warning: In case of image manifest is used as input outside of build process, be aware that pkgdata might change since the time image was built. Output format: {PKG} [PKGE:]{PKGV}[-{PKGR}] {PN} [PE:]{PV}[-{PR}] {PKGSIZE} Signed-off-by: Alexander D. Kanevskiy <kad@kad.name> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-06-07oe-pkgdata-util: fix AttributeErrorEd Bartosh1
The default arparse behaviour in Python 2 and early versions of Python 3 was to print usage message if no subcommands provided by user. In Python >= 3.2.3 subparser has to have attribute 'required' set to True to behave similar way. By default subparser.required attribute is set to False, usage message is not printed and code continues to be executed. oe-pkgdata-util code raises AttributeError exception in this case: AttributeError: 'Namespace' object has no attribute 'func' Fixed this by setting subparser.required to True. [YOCTO #9711] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02scripts: python3: convert iterables to listsEd Bartosh1
Converted return value of items() keys() and values() to lists when dictionary is modified in the loop and when the result is added to the list. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02scripts: use python3 in shebangEd Bartosh1
As most of oe-test cases work for devtool and recipetool it makes sense to switch both tools to python 3 by explicitly referring to python3 in their shebangs. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2016-04-04classes/buildhistory: optimise getting package size listPaul Eggleton1
Invoking oe-pkgdata-util in turn for every package in the list was slow with a large image. Modify oe-pkgdata-util's read-value command to take an option to read the list of packages from a file, as well as prefix the value with the package name; we can then use this to get all of the package sizes at once. This reduces the time to gather this information from minutes to just a second or two. Fixes [YOCTO #7339]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22scripts: print usage in argparse-using scripts when a command-line error occursPaul Eggleton1
For scripts that use Python's standard argparse module to parse command-line arguments, create a subclass which will show the usage the usage information when a command-line parsing error occurs. The most common case would be when the script is run with no arguments; at least then the user immediately gets to see what arguments they might need to pass instead of just an error message. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08scripts/oe-pkgdata-util: sort the packages in list-pkg-filesRoss Burton1
Sort the list of packages in list-pkg-files to make the output easier to read. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24scripts/oe-pkgdata-util: Fix variable name in error handlingRichard Purdie1
Fix: logger.error('Unable to find pkgdata directory %s' % pkgdata_dir) NameError: global name 'pkgdata_dir' is not defined Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-24oe-pkgdata-util: avoid returning skipped packagesRobert Yang1
The skipped packages may be pulled in by another package, for example, when libc6-dbg is already installed and should be skipped, but it would be pulled in by libsegfault, this patch fixes the issue. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-17lib/oe/package_manager: support exclusion from complementary glob process by ↵Paul Eggleton1
regex Sometimes you do not want certain packages to be installed when installing complementary packages, e.g. when using dev-pkgs in IMAGE_FEATURES you may not want to install all packages from a particular multilib. This introduces a new PACKAGE_EXCLUDE_COMPLEMENTARY variable to allow specifying regexes to match packages to exclude. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14oe-pkgdata-util: add list-pkg-files subcommandPaul Eggleton1
Adds a subcommand to list the files in a package, or list the files in all packages for a recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: add list-pkgs subcommandPaul Eggleton1
Add a subcommand to list packages, with options to list packages matching a specification, and packages produced by a particular recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: allow reverse package name lookupsPaul Eggleton1
Add a -r/--reverse option to the lookup-pkg subcommand to enable looking up the recipe-space package name for one or more runtime package names. Also make this subcommand into a function that can be reused elsewhere. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: fix read-value to handle data with colons in the valuePaul Eggleton1
The read-value subcommand was truncating the value if it contained colons, for example FILES_INFO. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: make find-path show a proper error if no package foundPaul Eggleton1
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: improve command-line usagePaul Eggleton1
* Use argparse instead of optparse for standardised help output, options and a much cleaner code structure * Look up pkgdata directory automatically so the user doesn't have to specify it * Use standard logging NOTE: this does mean a slight change in syntax - if you do want to specify the pkgdata directory (usually only necessary if you're calling it from within the build process) you need to use the parameter -p (or --pkgdata-dir) and specify this before the command, not after it. Examples: oe-pkgdata-util find-path /sbin/mke2fs oe-pkgdata-util lookup-recipe libelf1 oe-pkgdata-util read-value PKGSIZE libc6 oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-06-25oe-pkgdata-util: fix help textPaul Eggleton1
This was copy/pasted from another script and not corrected. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-12-03classes/package: record PKGSIZE as total file size in pkgdataPaul Eggleton1
We were using "du -sk" to collect the total size of all files in each package for writing out to PKGSIZE in each pkgdata file; however this reports the total space used on disk not the total size of all files, which means it is dependent on the block size and filesystem being used for TMPDIR on the build host. Instead, take the total of the size reported by lstat() for each packaged file, which we are already collecting for FILES_INFO in any case. Note: this changes PKGSIZE to be reported in bytes instead of kilobytes since this is what lstat reports, but this is really what we should be storing anyway so that we have the precision if we need it. Fixes [YOCTO #5334] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03classes/package: fix FILES_INFO serialisation in pkgdataPaul Eggleton1
The FILES_INFO entry in each pkgdata file stores the list of files for each package. Make the following improvements to how this is stored: * Store paths as they would be seen on the target rather than erroneously including the full path to PKGDEST (which is specific to the build host the package was built on) * For simplicity when loading the data, store complete paths for each entry instead of trying to break off the first part and use it as the dict key * Record sizes for each file (as needed by Toaster) * Serialise the value explicitly using json rather than just passing it through str(). Fixes [YOCTO #5443]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-29scripts/oe-pkgdata-util: fix global name 'debug' is not definedPaul Eggleton1
This global variable is no longer present, so pass in the value specified via the command line. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-28scripts/oe-pkgdata-util: add ability to find a recipe from a target packagePaul Eggleton1
Add a "lookup-recipe" command to show which recipe produced a particular package. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: add ability to look up runtime package namesPaul Eggleton1
Add a "lookup-pkg" command to oe-pkgdata-util that can be used to find the runtime name of a package (after e.g. Debian library package renaming). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: add ability to search for a target pathPaul Eggleton1
Add ability to search for a target path in produced packages, in order to find which package provides a specific file. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: check path arguments to ensure they existPaul Eggleton1
Show an error if the specified paths don't exist. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: improve help text and command line parsingPaul Eggleton1
* Use optparse to parse command line * Make help text actually helpful by describing what each command does * Drop comment at the top listing the commands which is now superfluous Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: remove remnants of former pkgdata structurePaul Eggleton1
OE-Core commit 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379 removed the vendor-os argument from the command line, and the code using the package architectures, so clean these items up. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-09-13bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific ↵Richard Purdie1
directory Currently we have a hierarchy of pkgdata directories and the code has to put together a search path and look through each in turn until it finds the data it needs. This has lead to a number of hardcoded paths and file globing which is unpredictable and undesirable. Worse, certain tricks that should be easy like a GL specific package architecture become problematic with the curretn search paths. With the modern sstate code, we can do better and construct a single pkgdata directory for each machine in just the same way as we do for the sysroot. This is already tried and well tested. With such a single directory, all the code that iterated through multiple pkgdata directories and simply be removed and give a significant simplification of the code. Even existing build directories adapt to the change well since the package contents doesn't change, just the location they're installed to and the stamp for them. The only complication is the we need a different shlibs directory for each multilib. These are only used by package.bbclass and the simple fix is to add MLPREFIX to the shlib directory name. This means the multilib packages will repackage and the sstate checksum will change but an existing build directory will adapt to the changes safely. It is close to release however I believe the benefits this patch give us are worth consideration for inclusion and give us more options for dealing with problems like the GL one. It also sets the ground work well for shlibs improvements in 1.6. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-16classes/buildhistory: record size of installed package not compressed archiveMartin Jansa1
* usually it's more important to know how much space will each package take on target device then size of compressed package * example for libewebkit0 with 4 different architectures, interesting that om_gta02 .ipk is bigger but it's smaller when installed before: MACHINE DEFAULTTUNE SIZE (.ipk file) om_gta04 cortexa8t-neon 15996 KiB libewebkit0 qemux86_64 x86-64 16992 KiB libewebkit0 spitz xscale 16148 KiB libewebkit0 om_gta02 arm920t 16260 KiB libewebkit0 after: MACHINE DEFAULTTUNE SIZE (installed) om_gta04 cortexa8t-neon 60544 KiB libewebkit0 qemux86_64 x86-64 63720 KiB libewebkit0 spitz xscale 60588 KiB libewebkit0 om_gta02 arm920t 56268 KiB libewebkit0 Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-08scripts/oe-pkgdata-util: find complementary packages for split packagesPaul Eggleton1
Check after getting the original package name (e.g. undoing Debian renaming) if there is a complementary package for that name, e.g. if the glob is *-dev, then libudev0 -> libudev -> libudev-dev. Fixes [YOCTO #4136]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31scripts/oe-pkgdata-util: exclude unpackaged in glob outputPaul Eggleton1
Check for .packaged file and exclude packages if this file does not exist - this avoids attempting to install empty packages during complementary package installation within do_rootfs that didn't end up being created (and failing as a result). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-28Rework installation of dev, dbg, doc, and locale packagesPaul Eggleton1
Use a similar mechanism that was previously used to install locales at rootfs generation time to install other "complementary" packages (e.g. *-dev packages) - i.e. install all of the explicitly requested packages and their dependencies, then get a list of the packages that were installed, and use that list to install the complementary packages. This has been implemented by using a list of globs which should make it easier to extend in future. The previous locale package installation code assumed that the locale packages did not have any dependencies that were not already installed; now that we are installing non-locale packages this is no longer correct. In practice only the rpm backend actually made use of this assumption, so it needed to be changed to call into the existing package backend code to do the complementary package installation rather than calling rpm directly. This fixes the doc-pkgs IMAGE_FEATURES feature to work correctly, and also ensures that all dev/dbg packages get installed for dev-pkgs/dbg-pkgs respectively even if the dependency chains between those packages was not ensuring that already. The code has also been adapted to work correctly with the new SDK-from-image functionality. To that end, an SDKIMAGE_FEATURES variable has been added to allow specifying what extra image features should go into the SDK (extra, because by virtue of installing all of the packages in the image into the target part of the SDK, we already include all of IMAGE_FEATURES) with a default value of "dev-pkgs dbg-pkgs". Fixes [YOCTO #2614]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>