summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2013-12-03classes/package: fix FILES_INFO serialisation in pkgdataPaul Eggleton1
The FILES_INFO entry in each pkgdata file stores the list of files for each package. Make the following improvements to how this is stored: * Store paths as they would be seen on the target rather than erroneously including the full path to PKGDEST (which is specific to the build host the package was built on) * For simplicity when loading the data, store complete paths for each entry instead of trying to break off the first part and use it as the dict key * Record sizes for each file (as needed by Toaster) * Serialise the value explicitly using json rather than just passing it through str(). Fixes [YOCTO #5443]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26classes/image: write image manifestPaul Eggleton1
Write a list of installed packages to a .manifest file next to the image, so we can find out what went into the image after it has been constructed without necessarily having to have buildhistory enabled (although that will provide more detail.) We can make use of this for example in the testimage class associated code that checks for installed packages for determining whether or not to run specific tests. Note: this replaces the previous ipk-specific manifest code with something that works for ipk, rpm and deb, and instead of a pruned status file, packages are listed one per line, in the following format: <packagename> <packagearch> <version> Tests for all three backends have shown that the performance impact of this change is negligible (about 1.5s max). Implements [YOCTO #5410] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Add option --minutes to show time in minutesPeter Kjellerstedt2
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Add a color for the package_write_* tasksPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Reorder the legend to match the task execution orderPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Make bars without a specified color whitePeter Kjellerstedt1
Previously they were transparent. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Make "Show more" show all processesPeter Kjellerstedt4
While "Show more" is enabled, all processes are shown, regardless of --mintime. This also has the added benefit of making the first shown bar start at its correct offset from the start time, rather than always starting at 0. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Draw a lot less ticksPeter Kjellerstedt1
With this, one second ticks are only enabled if the width of a second is five pixels or more. It is also possible to distinguish 1, 5 and 30 second ticks. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Disable options that do not make sensePeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: No need to do a double list comprehension over filesPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Avoid having overlapping process barsPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Use correct header heightPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Correct the X offset for the chartPeter Kjellerstedt1
This will make the first bar actually start within the graph. It will also move the graph to the right so the names of the first tasks are more likely to be visible. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Set the initial state of "Show more" correctlyPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Make the horizontal scaling stay within boundsPeter Kjellerstedt1
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18pybootchartgui: Import pybootchartgui 0.14.5Peter Kjellerstedt16
This update the pybootchartgui code to the latest release from its new location at "https://github.com/mmeeks/bootchart". This only imports the relevant parts, and not all of bootchart2. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-14scripts/create-recipe: fix handling of --helpPaul Eggleton1
If --help is specified as the first argument, show the standard help text instead of trying to process it as a URL. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-30wic: Remove selinux_check()Tom Zanussi1
This seems to be an obsolete check - we don't have any problems with image creation under selinux, so remove it. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-10-29scripts/oe-pkgdata-util: fix global name 'debug' is not definedPaul Eggleton1
This global variable is no longer present, so pass in the value specified via the command line. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-29scripts: Add ksize.py and dirsize.pyDarren Hart2
Fixes [YOCTO #5388] These scripts can be useful when working to reduce the size of the Linux kernel and the root filesystem. ksize.py displays the kernel build size by the built-in.o files. dirsize.py displays the various sizes of the components of the root directory. Signed-off-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-28scripts/oe-pkgdata-util: add ability to find a recipe from a target packagePaul Eggleton1
Add a "lookup-recipe" command to show which recipe produced a particular package. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: add ability to look up runtime package namesPaul Eggleton1
Add a "lookup-pkg" command to oe-pkgdata-util that can be used to find the runtime name of a package (after e.g. Debian library package renaming). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: add ability to search for a target pathPaul Eggleton1
Add ability to search for a target path in produced packages, in order to find which package provides a specific file. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: check path arguments to ensure they existPaul Eggleton1
Show an error if the specified paths don't exist. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: improve help text and command line parsingPaul Eggleton1
* Use optparse to parse command line * Make help text actually helpful by describing what each command does * Drop comment at the top listing the commands which is now superfluous Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-28scripts/oe-pkgdata-util: remove remnants of former pkgdata structurePaul Eggleton1
OE-Core commit 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379 removed the vendor-os argument from the command line, and the code using the package architectures, so clean these items up. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2013-10-26relocate_sdk.py: Allow script to work with Python 2.4 and 3.Konrad Scherer1
Python 2.4 does not support the 'b' string literal or the keyword 'as' in exception handling. Python 3 does not accept the old method of exception handling and defaults to unicode. The b() function converts strings to bytes on Python 3 and using sys.exc_info() avoids the exception handling syntax. Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-10-26runqemu-extract-sdk: add --numeric-owner option to tar commandChen Qi1
If the same username exists on both target and the build host, but the uids differ, and we start target via NFS, then the uid for the user will be incorrect on target. For example, if postfix's uid on host is 119 and on target is 1024, then if we start target via NFS, the uid for postfix will be 119. The root cause is that when we use runqemu-extract-sdk to generate the NFS rootfs for later use, the tar command will respect the username instead of uid. So if PSEUDO_PASSWD environment is not set correctly, the host /etc/passwd will be used, resulting in wrong uids. The situation for gid is completely analogous to that of uid. It's almost impossible for the runqemu-extract-sdk to guess the correct location of the needed password file merely based on the target tarball name. This patch solves this problem by adding the '--numeric-owner' option to the tar command so that the uid/gid will be used when extracting the tarball using runqemu-extract-sdk. In this situation, we'll always get the correct uid/gid after extracting the tarball. [YOCTO #5364] Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-10-26buildhistory-diff: add ability to report version changesPaul Eggleton1
Add a -v/--report-ver option to report changes in PKGE/PKGV/PKGR even if the value is the same as the default from PE/PV/PR. Also add a -a/--report-all option to report all changes instead of just the default significant ones. Addresses [YOCTO #5263]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-10-23wic: Update and generalize pseudo setup for rootfs generationTom Zanussi1
Remove unnecessary pseudo exports i.e. PSEUDO_DISABLED and move the setup to the top-level prepare_rootfs(). Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Make find_binary_path() more user-friendlyTom Zanussi1
find_binary_path() is useful, but if the binary isn't found, it prints a stacktrace and a less-than-useful message. Users complain when they get stacktraces for things they can act on, so remove the stacktrace and tell the user what the problem is. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove binary dependenciesTom Zanussi1
Current functionality doesn't make use of kpartx, mount, or unmount, and we use native mkswap, so remove the binary checks for those. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove rpmmisc call from livecdTom Zanussi1
We don't currently use LiveCDImageCreator, but it makes calls when initialized via the plugin interface to rpmmisc module functions, which we don't want the dependency on. To make it (and LiveUSBImageCreator) happy, we give it the dummy "i386" value for now. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove rpm and grabber dependencies from BaseImageCreatorTom Zanussi1
BaseImageCreator is a base class for DirectImageCreator and others, and imports rpm and grabber (which imports rpm). The various plugins e.g. DirectPlugin import the creators and therefore these dependencies, which manifest at run-time as e.g.: Warning: Failed to load plugin imager/direct_plugin: No module named rpm Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: remove rpm warning code from BackendPluginTom Zanussi1
We don't currently use rpm functionality, so we don't need to silence rpm warnings. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove dependency on myurlgrab moduleTom Zanussi1
mylrlgrab is in grabber, which imports rpm. For current functionality, we don't need to grab urls or import rpm, so remove the dependency. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove dependency on rpmmiscTom Zanussi3
rpmmisc imports rpm and contains misc rpm utilities related to packaging and determining arches based on the packaging. We should never run across this in the initial version of wic, so remove the dependency. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: eliminate module checksTom Zanussi1
We're removing all external dependencies including rpm and urlgrabber, so we don't need this check. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-23wic: Remove dependency on rt_util moduleTom Zanussi7
rt_util contains bootstrap_mic(), which imports rpm and other things we don't need because we don't do bootstrap i.e. runtime (set in wic.conf) is always set to 'native', which means use what's on the local host. bootstrap mode is for downloading and installing rpms that wic needs, which we may want to implement later; for now, we just want to use what's local. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18wic: add pseudo to the populate-extfs stepTom Zanussi1
Without this, files in the generated filesystem pick up the wrong ownership. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18wic: Initialize return values in find_artifacts()Tom Zanussi1
If one of these isn't found, it won't be initialized and will throw an UnboundLocalError. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18wic: Check for the existence/correctness of build artifactsTom Zanussi1
If a user uses the -e option and specifies a machine that hasn't been built or uses the wrong .wks script for the build artifacts pointed to by the current machine, we should point that out for obvious cases. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-16wic: Check for external modulesDarren Hart1
Since eight unique files import rpm, perform a check at the top level for the existence of the rpm module print a sensible error message if it is not. This may be able to be removed if some of the core rpm dependencies are removed from the mic libs. Also check for urlgrabber. This avoids a bracktrace in the event the modules are not installed which can be very off-putting to would-be users. Signed-off-by: Darren Hart <dvhart@linux.intel.com> Cc: Tom Zanussi <tom.zanussi@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-16wic: Force lba off for FAT16 partitionsDarren Hart1
If fat16 is specified to the mkpart parted command, parted will default to setting the lba flag which causes certain EFI firmware to fail to detect the filesystem. lba shouldn't be necessary for FAT16 filesystems anyway, explicitly disable it. Signed-off-by: Darren Hart <dvhart@linux.intel.com> Cc: Tom Zanussi <tom.zanussi@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-16wic: check passed-in build artifact directoriesTom Zanussi1
Make sure they exist - complain if they don't. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-16wic: check for build artifactsTom Zanussi1
wic needs to be given one form of build artifacts or another - complain if the user doesn't do that. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-01wic: Add OpenEmbedded-specific implementationTom Zanussi20
Reuses the mic/livecd infrastructure but heavily subclasses and modifies it to adapt to the special needs of building images from existing OpenEmbedded build artifacts. In addition to the OE-specific mic objects and modifications to the underlying infrastructure, this adds a mechanism to allow OE kickstart files to be 'canned' and made available to users via the 'wic list images' command. Two initial OE kickstart files have been added as canned .wks files: directdisk, which implements the same thing as the images created by directdisk.bbclass, and mkefidisk, which can essentially be used as a replacement for mkefidisk.sh. Of course, since creation of these images are now driven by .wks files rather than being hard-coded into class files or scripts, they can be easily modified to generate different variations on those images. They also don't require root priveleges, since they don't use mount to create the images. They don't however write to media like mkefidisk.sh does, but rather create images that can be written onto media. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-01wic: Add mic w/pykickstartTom Zanussi135
This is the starting point for the implemention described in [YOCTO 3847] which came to the conclusion that it would make sense to use kickstart syntax to implement image creation in OpenEmbedded. I subsequently realized that there was an existing tool that already implemented image creation using kickstart syntax, the Tizen/Meego mic tool. As such, it made sense to use that as a starting point - this commit essentially just copies the relevant Python code from the MIC tool to the scripts/lib dir, where it can be accessed by the previously created wic tool. Most of this will be removed or renamed by later commits, since we're initially focusing on partitioning only. Care should be taken so that we can easily add back any additional functionality should we decide later to expand the tool, though (we may also want to contribute our local changes to the mic tool to the Tizen project if it makes sense, and therefore should avoid gratuitous changes to the original code if possible). Added the /mic subdir from Tizen mic repo as a starting point: git clone git://review.tizen.org/tools/mic.git For reference, the top commit: commit 20164175ddc234a17b8a12c33d04b012347b1530 Author: Gui Chen <gui.chen@intel.com> Date: Sun Jun 30 22:32:16 2013 -0400 bump up to 0.19.2 Also added the /plugins subdir, moved to under the /mic subdir (to match the default plugin_dir location in mic.conf.in, which was renamed to yocto-image.conf (moved and renamed by later patches) and put into /scripts. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-01wic: Initial code for wic (OpenEmbedded Image Creator)Tom Zanussi4
Initial implementation of the 'wic' command. The 'wic' command generates partitioned images from existing OpenEmbedded build artifacts. Image generation is driven by partitioning commands contained in an 'Openembedded kickstart' (.wks) file specified either directly on the command-line or as one of a selection of canned .wks files (see 'wic list images'). When applied to a given set of build artifacts, the result is an image or set of images that can be directly written onto media and used on a particular system. 'wic' is based loosely on the 'mic' (Meego Image Creator) framework, but heavily modified to make direct use of OpenEmbedded build artifacts instead of package installation and configuration, things already incorporated int the OE artifacts. The name 'wic' comes from 'oeic' with the 'oe' diphthong promoted to the letter 'w', because 'oeic' is impossible to remember or pronounce. This covers the mechanics of invoking and providing help for the command and sub-commands; it contains hooks for future commits to connect with the actual functionality, once implemented. Help is integrated into the 'wic' command - see that for details on usage. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-30mkefidisk.sh: Allow using a loopback mounted fileJason Wessel1
It should be possible to generate a disk to a file using a loopback device with mkefidisk.sh, which is useful for booting simulators. To make this possible the partitions for the loop back need to work similarly to the mmc devices. The mkfs.vfat also requires and additional argument to force it to write to something other then a real disk. Example: qemu-img create -f raw bigdisk 4G dev=`sudo losetup -f` sudo losetup $dev bigdisk mkefidisk.sh $dev tmp-eglibc/deploy/images/qemux86/core-image-minimal-qemux86.hddimg /dev/sda sudo losetup -d $dev Note: Also a bug was fixed in the mkefidisk.sh where if the disk you are writing to initially has an invalid label the size of the first partition will be computed incorrectly. For the simulator disk creation this is generally always the case, but this can happen with real hardware as well. Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>