summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2016-09-30scripts/buildstats-diff: make logger msg format a bit more readableMarkus Lehtonen1
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: use exception for internal error handlingMarkus Lehtonen1
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: add walltime to --diff-attrMarkus Lehtonen1
For comparing the elapsed wall clock time of tests. Default values for --min-val and --min-absdiff are 5 seconds and 2 seconds. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: add read_ops and write_ops to --diff-attrMarkus Lehtonen1
Two new options, making it possible to compare the number of filesystem operations of tasks. Defaults for --min-val and --min-absdiff are set to more or less arbitrary 500 and 50 operations, respectively. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: add read_bytes and write_bytes to --diff-attrMarkus Lehtonen1
These are I/O counter values from /proc/<pid>/io and represent the number of bytes read from / written to the storage layer. Default values for --min-val and --min-absdiff limits are set to 512kB and 128kB, respectively. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: introduce --diff-attrMarkus Lehtonen1
A new command line option for choosing which "attribute" of the buildstats to compare. At first, the already supported 'cputime' is the only available option. But, refactoring done in this patch should make it easy to add new attribute types. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: do not hardcode field widths in outputMarkus Lehtonen1
Dynamically adjust the width of all fields in task diff output. Makes it easier to print other units than cputime, too. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: implement BSTask classMarkus Lehtonen1
New class representing buildstats data of a single task. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: rename --min-time and --min-timediff argsMarkus Lehtonen1
Rename these arguments to --min-val and --min-absdiff in preparation for supporting other "quantities" than just cputime. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30scripts/buildstats-diff: check that the given directory existsMarkus Lehtonen1
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-30wic: rename and amend systemd-boot wks fileJianxun Zhang1
Rename wks for systemd-boot per the suggestion from community. Also amend description to distinguish it from others when running "wic list images". Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-28runqemu: explicitly set image formatEd Bartosh1
QEMU produces a warning if drive format is not specified: WARNING: Image format was not specified for 'tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.wic' and probing guessed raw. Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. Specify the 'raw' format explicitly to remove the restrictions. Set image format to 'vmdk', 'qcow2' or 'vdi' for correspondent image types. Set it to 'raw' for the rest of image types. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28recipetool: newappend: drop _provide_to_pnChristopher Larson1
This function was broken by the multi-config changes, and isn't needed anymore now that recipeutils.pn_to_recipe can handle provides. Without this, the newappend sub-command fails. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28scripts/cleanup-workdir: Adapt to SDK_ARCH -> SDK_SYS chanages for crosssdkRichard Purdie1
With the change of crosssdk to use SDK_SYS, we need to update the script to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28devtool: Add a line break to generated READMEJoe MacDonald1
When devtool creates a new workspace, it produced a README with one very long line and no space following 'bblayers.conf'. Add a line break as was intended. Signed-off-by: Joe MacDonald <joe_macdonald@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23scripts/runqemu: provide better error message on runqemu ifup failStephano Cetola1
If runqemu-ifup fails hen running testimage, a rather cryptic error regarding "no tty present" is displayed. If this step fails, we should at least point the user at runqemu-gen-tapdevs. A quick search of this term in the manual will lead them to "Enabling Runtime Tests on QEMU" which should give them all the info they need. Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23wic: remove partition imagesEd Bartosh1
Preserving images for every partition doubles disk space consumed by an image build. As those images are not used, so it's better to remove them after assembling final image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23scripts/runqemu: Using a cpio* rootfs has no special networkNathan Rossi1
When booting a system with the rootfs being of cpio* type the networking setup should still work the same as for all other root filesystem types. This change removes the clearing of the NETWORK_CMD variable allowing for the slirp/tap setup to be provided to QEMU. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-23runqemu: Move virtio RNG to machine configurationNathan Rossi1
Not all QEMU machines (outside of those available in OE-Core) are capable of using the virtio-rng-pci device due to various machine models not having a pci/virtio bus. This makes it such that the use of the '-device virtio-rng-pci' flag to QEMU is machine specific. This patch removes the general addition of the flag to all runqemu targets and adds the flag into the QB_OPT_APPEND for all the qemu* machines in OE-Core that support its use (which is all of them). Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-22directdisk*.wks: add serial console supportEd Bartosh3
Added serial console to kernel command line to to make it easier to boot wic images on devices without display. Tested on MinnowBoard MAX. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-21runqemu: don't fail during check_arg_machine()Joshua Lock1
If DEPLOY_DIR_IMAGE doesn't exist during check_arg_machine() we will attempt to guess a suitable value later when check_and_set() calls validate_paths(), therefore this shouldn't raise an exception Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21runqemu: don't try and invoke bitbake when running in a toolchain envJoshua Lock1
If a MACHINE value is passed we can't validate it by running bitbake as the toolchain environment doesn't include the build system, we must assume that the passed value for MACHINE is correct. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21runqemu: try and guess qemu-system binary when MACHINE isn't setJoshua Lock1
Emulate some logic from the prior, shell based, version of runqemu to try and infer the correct setting for MACHINE from the kernel and rootfs filenames. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21runqemu: validate paths and attempt to infer unset pathsJoshua Lock1
We need to validate and ensure all paths are set regardless of whether runqemu was invoked with a .qemuboot.conf file or otherwise. Split this logic out into a separate method called during check_and_set() Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20runqemu: improve finding of rootfs, kernel and dtbRobert Yang1
* Search rootfs in the following order: - IMAGE_NAME*.FSTYPE - IMAGE_LINK_NAME*.FSTYPE * Search kernel in the following order: - QB_DEFAULT_KERNEL - KERNEL_IMAGETYPE - KERNEL_IMAGETYPE* * Search dtb in the following order: - QB_DTB - QB_DTB* - *.dtb * Fix DTB, it should only work with "-kernel" option. [YOCTO #10265] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-20runqemu-gen-tapdevs: remove /etc/runqemu-nosudo when remove tapsRobert Yang1
It creats /etc/runqemu-nosudo when creats taps, so should remove it when remove taps. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-19runqemu: use OECORE_NATIVE_SYSROOT from sdkRobert Yang1
There is no STAGING_DIR_NATIVE or bitbake in a extracted sdk, so check OECORE_NATIVE_SYSROOT and use it. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19runqemu: work even if a *.qemuboot.conf isn't foundJoshua Lock1
A qemuboot conf file is a convenience but it should still be possible to invoke runqemu without them, especially for examples such as using the SDK with an extracted rootfs via NFS. As read_qemuboot() is always called we need to be sure that function can return cleanly, without throwing Exceptions, even if a qemuboot conf file isn't found. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19runqemu: try symlinks when kernel or rootfs can't be foundJoshua Lock1
If the kernel or rootfs names written to the qemuboot.conf can't be found, try and find the symlinked variant of the filename. This will help usability of runqemu, for example where a user downloads an image and associated files as the symlinked names yet the qemuboot.conf variables point to the full, non-linked, file names. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19runqemu: clarify an INFO messageJoshua Lock1
Make it clearer that we are looking for a file which ends with qemuboot.conf Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19runqemu: add guidance to resolve issues with missing filesJoshua Lock1
When a required binary cannot be found print some guidance pointing to using a sourced OE build environment or a qemuboot.conf file, based on a similar message from the previous shell-based runqemu. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19devtool: add: drop superfluous validation for recipe namePaul Eggleton1
Now that recipeutils.validate_pn() properly validates characters used in the name, we can drop this bit checking for '/' since that's not permitted by validate_pn(). (The FIXME comment here - that I myself apparently wrote - is questionable since that function was clearly never intended to allow '/', perhaps I was misled because it was broken and did so). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: support git short form URLsPaul Eggleton3
In keeping with making recipetool create / devtool add as easy to use as possible, users shouldn't have to know how to reformat git short form ssh URLs for consumption by BitBake's fetcher (for example user@git.example.com:repo.git should be expressed as git://user@git.example.com/repo.git;protocol=ssh ) - instead we should just take care of that automatically. Add some logic in the appropriate places to do that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: tweak license crunchingPaul Eggleton1
Filter out a plain "Licensed under the XXXX license" statement, as seen in the capnproto project (and no doubt others). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: pick up AC_PROG_SWIGPaul Eggleton1
AX_PKG_SWIG is not the only commonly-used macro for detecting swig - there's also AC_PROG_SWIG. As per AX_PKG_SWIG, add swig-native to DEPENDS if AC_PROG_SWIG is found in configure.ac. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: detect python autoconf macrosPaul Eggleton1
If python is required then we need to inherit pythonnative (or python3native) otherwise do_configure will probably fail since it won't be able to find python. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: fix error with git tree and no networkPaul Eggleton1
When creating a recipe for an existing local git clone, we attempt to use the fetcher to determine if it supports the SRCREV variable. Unfortunately running this code does a network check to get the latest revision as a direct result of us using '${AUTOREV}' as a default value. If you don't have a network connection this will of course fail. Rather than have this block creating the recipe, catch the exception and just guess from the URL. Ultimately this should probably be fixed in the fetcher but for now this will at least resolve the issue on this end. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: fix name/version extraction from filenamePaul Eggleton1
I ran into an example where recipetool was getting the name/version completely wrong: https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz >From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb file (likely because it couldn't split the file name and therefore took all of it, then got the version from one of the files inside the tarball). When this happens it's just irritating because you then have to delete the recipe / run devtool reset and then run recipetool create / devtool add again and specify the version manually. This patch is the result of systematically running the determine_from_filename() function over the files on the Yocto Project source mirror and my local downloads directory and fixing as many of the generic issues as reasonably practical - it now gets the name and version correct much more often. There are still cases where it won't, but they are now in the minority. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: improve python recipe license handlingPaul Eggleton1
Try to ensure that for Apache, GPL and LGPL where the values extracted from the "Classifiers" field may not be version-specific, if there is a versioned license in the free-form license field then use that instead. Also insert the free-form license field as a comment in the recipe for the user's reference. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19recipetool: create: fix picking up name from local python source treePaul Eggleton1
Make use of the extravalues dict to send back other variable values from the python handling plugin, and enable passing back PV and PN. This not only places variable values in the final recipe a bit more consistently with other types of source, it also allows the name and version to be picked up fron a local source tree and not just when the recipe is fetched from a remote URL that happens to have those in it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-19runqemu: acquire_lock() should fail when failed to open the fileRobert Yang1
The open(self.lock, 'w') may fail when the lock is created by other users, return false for this case to let it try other devices. Fixed: runqemu - INFO - Running /sbin/ip link... runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap0.lock... Traceback (most recent call last): File "/buildarea/lyang1/poky/scripts/runqemu", line 972, in <module> ret = main() File "/buildarea/lyang1/poky/scripts/runqemu", line 963, in main config.setup_network() File "/buildarea/lyang1/poky/scripts/runqemu", line 810, in setup_network self.setup_tap() File "/buildarea/lyang1/poky/scripts/runqemu", line 761, in setup_tap if self.acquire_lock(): File "/buildarea/lyang1/poky/scripts/runqemu", line 182, in acquire_lock lock_descriptor = open(self.lock, 'w') PermissionError: [Errno 13] Permission denied: '/tmp/qemu-tap-locks/tap0.lock' Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16scripts: add tool to scan for bashisms recipe shell scriptsRoss Burton1
Shell functions in bitbake are executed with /bin/sh so should be POSIX compliant and not use Bash extensions, or at least only use extensions that are implemented in both dash and ash (busybox). This tool will extract all of the shell scripts from all recipes and run them through checkbashisms (it assumes that checkbashisms is on $PATH). There is a whitelist to filter out false-positives such as the use of $HOSTNAME (a bashism) in functions where we have defined it, or using the 'type' builtin which is supported by ash/dash. [ YOCTO #8851 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16scripts: introduce buildstats-diffMarkus Lehtonen1
New script for comparing buildstats from two separate builds. The script has two modes: normally it prints the differences in task execution (cpu) times but using --ver-diff option makes it just print the recipe version differences without any cpu time data. Other command line options are provided to alter the sort criteria of the data and to filter out insignificant differences and/or short tasks. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-15oe-selftest: check for coverage version before starting testsHumberto Ibarra1
python coverage versions lower than 4.x have problems with some distros. Adding the 4.x version as requirement to continue with coverage tracking. [YOCTO #10207] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14create-pull-request: remove output directoryEd Bartosh1
When 'git request-pull' fails it makes sense to remove output directory. Otherwise create-pull-request will complain that output directory already exists on the next run. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14runqemu: fix a race issue on lockdirRobert Yang1
There might be a race issue when multi runqemu processess are running at the same time: | Traceback (most recent call last): | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 920, in <module> | ret = main() | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 911, in main | config.setup_network() | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 760, in setup_network | self.setup_tap() | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-ipk/build/scripts/runqemu", line 697, in setup_tap | os.mkdir(lockdir) | FileExistsError: [Errno 17] File exists: '/tmp/qemu-tap-locks' Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09scripts/runqemu: Add snapshot supportRichard Purdie1
Allow access to the snapshot option of qemu to simplify some of our runtime testing to avoid copying images. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09runqemu: Enable virtio RNG for all platformsRichard Purdie1
We have problems where systems simply stop booting and hang. This is due to a lack of entropy which means ssh keys and networking can't be brought up. Adding in the virtio-rng passthrough support allows host entropy to pass into the guess and avoids these hangs. This is particularly problematic after the gnutls upgrade which starts using /dev/random instead of /dev/urandom but was an issue we'd occasionally seem before that. It particualrly affected x86 and ppc machines for some reason. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09runqemu: Update to modern prefrerred net syntaxRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09runqemu: Allow unique network interface MAC addressesRichard Purdie1
Current qemu instances all share the same MAC address. This shouldn't be an issue as they are all on separate network interfaces, however on the slight chance this is causing problems, its easy enough to ensure we use unique MAC addresses based on the IP numbers we assign. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-18 15:08:09 +0000