summaryrefslogtreecommitdiff
path: root/scripts/lib
AgeCommit message (Collapse)AuthorFiles
2017-04-05wic: improve error messageChen Qi1
When using `wic create mkefidisk -e core-image-minimal', the following error message appeared. Please bake it with 'bitbake parted-native' and try again. However, following this command doesn't do any help. The same problem still appeared. The problem is that when we 'bitbake parted-native', it doesn't have anything to do with core-image-minimal. And the required tool 'parted' is not under core-image-minimal's recipe-sysroot-native directory. Improve the error message so that following it could get things done. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05Revert "filemap: remove FilemapSeek class"Ed Bartosh1
FIEMAP API is not supported by tmpfs file system, but SEEK_HOLE/SEEK_DATA is supported. Returned back FilemapSeek class that implements support of SEEK_HOLE/SEEK_DATA API to make sparse_copy API working on tmpfs again. This reverts commit 6b80c13f7a82a312a3b981de5a56c66466ba1fac. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05scripts/oe-build-perf-report: fix 'charts ready' console messageMarkus Lehtonen2
The javascript console log messages are used in scraping, when converting an html test report to html email. Before this patch a console message indicating that all charts have been drawn was not correctly sent if the last test failed (or didn't have chart data for some other reason) which, in turn, caused oe-build-perf-report-email.py script to fail with a timeout. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05yocto-compat-layer.py: Fix the signature validationMark Hatle1
The initial signatures need to be collected -after- the dependency layers have been added to the system. Otherwise changes that happen due to dependencies, outside of the layer being scanned, will show up as signature problems. The add_layer function was split into two pieces so that we can process the dependencies first, and then add the layer itself for the comparison. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05compatlayer/__init__.py: Allow add_layer to process recursive depsMark Hatle1
When processing a layer for dependencies, you have to process the layer itself, it's dependencies, the dependencies dependencies and so forth until all items have been processed. i.e.: LayerA requires LayerB requires LayerC requires layerD The end result should be LayerB, LayerC and LayerD are all dependencies of LayerA. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05devtool: point runqemu to correct native bindirbrian avery1
devtool/runqemu.py was relying on STAGING_BINDIR_NATIVE to find the host tools it needed like qemu-system-<arch>. In the post RSS world, this no longer exists. This patch points it to {STAGING_DIR}/{BUILD_ARCH}/{bindir_native}. [YOCTO #11223] Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01scripts/contrib: add oe-build-perf-report-emailMarkus Lehtonen1
Script for sending build perf test reports as an email. Mangles an html report, generated by oe-build-perf-report, into a format suitable for html emails. Supports multipart emails where a plaintext alternative can be included in the same email. Dependencies required to be installed on the host: - phantomjs - optipng [YOCTO #10931] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01scripts: add oe-build-perf-report scriptMarkus Lehtonen5
A new tool for pretty-printing build perf test results stored in a Git repository. The scripts is able to produce either simple plaintext report showing the difference between two commits, or, an html report that also displays trendcharts of the test results. The script uses Jinja2 templates for generating HTML reports so it requires python3-jinja2 to be installed on the system. [YOCTO #10931] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31wic: partition: Update fsck parametersDaniel Schultz1
These parameters are copied from the ext image class. -D will let fsck perform further directory optimizations -v might be helpful for debugging purpose Signed-off-by: Daniel Schultz <d.schultz@phytec.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31wic: remove prepare_empty_partition_squashfsEd Bartosh1
There is not much sense in creation of empty squashfs partition. It's also not possible to create empty squashfs partition of specified size. Even more, prepare_empty_partition_squashfs method is absolutely broken. It raises exception when called and even its signature differs from the rest of of similar methods. It means that nobody uses it and it's safe to remove it. Removed prepare_empty_partition_squashfs method and testing of empty squashfs partition. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: fix list of supported fstypes in help contentEd Bartosh1
Added vfat and msdos to the list of supported fstypes in 'wic help kickstart' output. [YOCTO #11137] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: set correct system id for msdos partitionsEd Bartosh1
Explicitly set system id 0x6(FAT16) for msdos partitions. Removed old code that attempts to achieve the same result using 'parted ... lba off'. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: set FAT 16 for msdos partitionsEd Bartosh1
Used '-F 16' parameter for mkdosfs to create FAT16 partitions for 'msdos' partition type. [YOCTO #11137] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: support 'msdos' fstypeEd Bartosh1
Added prepare_empty_partition_msdos and prepare_rootfs_msdos methods to support 'msdos' filesystem type. Created aliases prepare_empty_partition_vfat and prepare_rootfs_vfat to continue supporting creation of vfat patitiions. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: don't silently skip unknown fstypesEd Bartosh1
Fixed wic code that loops through hard-coded list of known fstypes to find prepare_rootfs_<fstype> or prepare_empty_partition_<fstype> methods and silently skipping unknown fstypes. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: allow only supported fstypesEd Bartosh3
Restricted possible values of --fstype to the list of supported types. This should catch incorrect values when .wks file is being parsed. Removed checks for empty fstype and mentioning of unsupported fstype 'ontrackdm6aux3'. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: remove unused code from runner moduleEd Bartosh2
Removed unused APIs 'outs' and 'quiet'. Removed 'catch' parameter from runner.runtool API as wic uses only one value of it. Removed the code that handles unused values of 'catch' parameter. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: remove runner.show APIEd Bartosh2
Replaced runner.show call to exec_cmd call in bootimg-pcbios plugin. Removed runner.show API as it's not used anywhere else. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: use wic-tools STAGING_DATADIR as bootimg_dirEd Bartosh1
If bootloader artifacts are not found in default bootimg_dir use wic-tools sysroot for the same purpose. This should prevent wic from failing if bootloader artifacts can't be found in image native sysroot. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31wic: remove fsimage pluginEd Bartosh2
Removed fsimage plugin and prepare_rootfs_from_fs_image API as they duplicate functionality of rawcopy plugin. Fsimage plugin makes wic to remove the image artifact from deployment directory, which can cause problems too. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-31filemap: remove FilemapSeek classEd Bartosh1
FIEMAP API was added to Linux kernel 2.6.28 back in 2008 SEEK_HOLE and SEEK_DATA API was added much letter. As FIEMAP is used by filemap module as a default API it's safe to remove FileMpSeek class as it's never used. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-30scripts/lib/compatlayer: detect_layers always use realpath'sAníbal Limón1
If you are using relative paths and change to other folder for execution it will fail, so use realpaths always. [YOCTO #11164] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28scripts/yocto-compat-layer.py: Add dependency validation in add_layerAníbal Limón1
Some layers don't have dependencies so add a validation to avoid exception when trying to None.split(). Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28devtool: open kconfig fragment in binary modeStephano Cetola1
When devtool writes to the kconfig fragment, it writes the output of the diff command returned from pipe.communicate(). This function returns binary objects. We should open the kconfig fragment file in binary mode if we expect to write binary objects to it. [YOCTO #11171] Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27wic: fix generation of partition UUIDEd Bartosh1
Partition UUID for msdos partitioning is based on a 4 bytes long system identifier. Wic uses random number to generate system identifier. For the numbers starting with 0 partition uuid is shorter, which makes wic images non-bootable as kernel expects 8 charactest in PARTUUID parameter. Padded system identifier with '0' when generating partition UUID to make it always 8 characters long. This should fix the boot issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-24wic: use kernel_dir to find systemd-efi bootloaderEd Bartosh1
Fixed test failure caused by using bootimg_dir path instead of kernel_dir to find systemd-efi bootloader: FAIL: test_systemd_bootdisk (oeqa.selftest.wic.Wic) ... FileNotFoundError: [Errno 2] No such file or directory: 'tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share' Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: improve getting syslinux pathEd Bartosh1
Used wic-tools STAGING_DATADIR if syslinux can't be found in default bootimg_dir. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: remove empty plugin methodsEd Bartosh3
Removed do_install_disk and do_configure_partition methods from bootimage-partition, fsimage and rawcopy plugins as they're empy and not used. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: fix bug in sparse_copyEd Bartosh1
sparse_copy creates output file that is smaller than input file when input file ends with unmapped blocks. Used truncate(<input file size>) when output file is created to ensure the size of output file will always be equal to the size of input file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: use kernel_dir instead of bootimg_dirEd Bartosh3
bootimg_dir is usually set to the value of STAGING_DATADIR and kernel_dir - to the value of DEPLOY_DIR_IMAGE, so usage of kernel_dir is more logical in bootimg-efi, bootimg-partition and rawcopy plugins. Replaced usage of bootimg_dir to kernel_dir in 3 above mentioned plugins that use DEPLOY_DIR_IMAGE as a default artifact location. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: don't set creator.bootimg_dir in pluginsEd Bartosh2
Changing bootimg_dir in plugins is a dangerous hack as this value comes from wic command line and can be used by multiple plugins. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22wic: remove rootfs_pcbios_ext pluginEd Bartosh1
Removed broken unused plugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21scripts/yocto-compat-layer.py: Handle layer dependencies when testAníbal Limón1
If some layer depends on other tries to find layer dependency, if the layer dependency isn't found avoid to test the layer and notice the user. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21scripts/yocto-compat-layer.py: Add option to disable layer autodiscoveryAníbal Limón1
Sometimes there is a need to only analyze the layer specified by the command line, the new option -n will disable autodiscovery of layers and only will try to test specified layers. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21scripts/lib/compatlayer: Remove require of meta- in layer dir nameAníbal Limón1
The layers isn't required to have a dirctory name start with meta- so remove the validation. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21scripts/compatlayer: Add exclude of meta-world-pkgdata on get_signaturesAníbal Limón1
The meta-world-pkgdata recipe can be modified when a layer is added may be can add recipes to world target, so exclude by default. [YOCTO #11162] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21scriptutils: Add support for specify stream on logger_createAníbal Limón1
It is a good idea to let the script to choose what stream wants to dump the logging output. [YOCTO #11160] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21wic: partition.py: Add fsck to avoid corrupt EXT file systemsDaniel Schultz1
This patch avoids the creation of a corrupt EXT file system. Since there are no checks if a EXT file system was successfully created, this should add to prevent possible system failures. Signed-off-by: Daniel Schultz <d.schultz@phytec.de> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21devtool: tidy up handling of parse failuresPaul Eggleton1
Since the tinfoil2 refactoring, if an error occurred during parsing, we were showing a traceback and not correctly exiting (since we weren't calling shutdown()). Fix both of these issues. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21wic/partionedfs: Avoid reserving space for non-existing ext. partitionKristian Amlie1
We don't need the gap that the extended partition occupies if we already know that we have less than five partitions. Saves up to one full alignment of space. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21devtool/recipetill: npm install of devDependenciesAnders Darander3
Web applications built using e.g. angular2, usually requires that the packages in devDependencies are available. Thus, add an option '--fetch-dev' to both devtool add and recipetool, to add npm packages in devDependencies to DEPENDS. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-16yocto-compat-layer: improve error handling in signature creationPatrick Ohly1
When "bitbake -k -S none world" failed, the error printed by yocto-compat-layer.py contained the stack trace multiple times and did not contain the stderr output from bitbake, making the error hard to understand and debug: INFO: ====================================================================== INFO: ERROR: test_signatures (common.CommonCompatLayer) INFO: ---------------------------------------------------------------------- INFO: Traceback (most recent call last): File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures stderr=subprocess.PIPE) File "/usr/lib/python3.4/subprocess.py", line 620, in check_output raise CalledProcessError(retcode, process.args, output=output) subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures curr_sigs = get_signatures(self.td['builddir'], failsafe=True) File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 149, in get_signatures raise RuntimeError(msg) RuntimeError: Traceback (most recent call last): File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 144, in get_signatures stderr=subprocess.PIPE) File "/usr/lib/python3.4/subprocess.py", line 620, in check_output raise CalledProcessError(retcode, process.args, output=output) subprocess.CalledProcessError: Command 'bitbake -k -S none world' returned non-zero exit status 1 Loading cache...done. Loaded 1328 entries from dependency cache. NOTE: Resolving any missing task queue dependencies NOTE: Runtime target 'zlib-qat' is unbuildable, removing... Missing or unbuildable dependency chain was: ['zlib-qat'] ... Summary: There were 5 ERROR messages shown, returning a non-zero exit code. The yocto-compat-layer.log was incomplete, it only had the first part without the command output. stderr was missing due to stderr=subprocess.PIPE. Instead of the complicated try/except construct it's better to check the return code ourselves and raise just a single exception. The output (both on stderr and in the yocto-compat-layer.log) now is: INFO: ====================================================================== INFO: ERROR: test_signatures (common.CommonCompatLayer) INFO: ---------------------------------------------------------------------- INFO: Traceback (most recent call last): File "/fast/work/poky/scripts/lib/compatlayer/cases/common.py", line 51, in test_signatures curr_sigs = get_signatures(self.td['builddir'], failsafe=True) File "/fast/work/poky/scripts/lib/compatlayer/__init__.py", line 147, in get_signatures raise RuntimeError(msg) RuntimeError: Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities. Command: bitbake -k -S none world Output: Loading cache...done. Loaded 1328 entries from dependency cache. NOTE: Resolving any missing task queue dependencies ERROR: Nothing PROVIDES 'qat16' (but /fast/work/meta-intel/common/recipes-extended/openssl-qat/openssl-qat_0.4.9-009.bb DEPENDS on or otherwise requires it) ERROR: qat16 was skipped: incompatible with machine qemux86 (not in COMPATIBLE_MACHINE) ... Missing or unbuildable dependency chain was: ['openssl-qat-dev'] ... Summary: There were 5 ERROR messages shown, returning a non-zero exit code. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-11wic/direct.py: Avoid exception if using multiple rawcopy/no-table entries.Kristian Amlie1
If we are both having a bootloader and a U-Boot environment file, we can end up with two entries using "--source rawcopy" and "--no-table", and since they reuse the same file [1], their cleanup handlers will try to delete the same file twice. So make sure we only do it once. [1] Although they reuse the same file, the resulting output is correct, so it appears the file is accessed in properly sequential order. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-10wic: filemap: Fixed spared_copy skipDaniel Schultz1
This patches removes the empty space in front of the copied file which was skipped. Without this reduction it's not possible to place a partition with rawcopy and skip parameter on a desired alignment. Signed-off-by: Daniel Schultz <d.schultz@phytec.de> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-10wic: plugins: rawcopy: Fixed wrong variable typeDaniel Schultz1
Without the int() function this variable will be a string. This will led to a error in Filemap on line 545 due wrong types. > [...] > File > ".../poky/scripts/lib/wic/filemap.py", line 545, in sparse_copy > if start < skip < end: > TypeError: unorderable types: int() < str() Signed-off-by: Daniel Schultz <d.schultz@phytec.de> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04yocto-compat-layer.py: Add script to YP Compatible Layer validationAníbal Limón7
The yocto-compat-layer script serves as a tool to validate the alignament of a layer with YP Compatible Layers Programme [1], is based on an RFC sent to the ML to enable automatic testing of layers [2] that wants to be YP Compatible. The tool takes an layer (or set of layers) via command line option -l and detects what kind of layer is distro, machine or software and then executes a set of tests against the layer in order to validate the compatibility. The tests currently implemented are: common.test_readme: Test if a README file exists in the layer and isn't empty. common.test_parse: Test for execute bitbake -p without errors. common.test_show_environment: Test for execute bitbake -e without errors. common.test_signatures: Test executed in BSP and DISTRO layers to review doesn't comes with recipes that changes the signatures. bsp.test_bsp_defines_machines: Test if a BSP layers has machines configurations. bsp.test_bsp_no_set_machine: Test the BSP layer to doesn't set machine at adding layer. distro.test_distro_defines_distros: Test if a DISTRO layers has distro configurations. distro.test_distro_no_set_distro: Test the DISTRO layer to doesn't set distro at adding layer. Example of usage: $ source oe-init-build-env $ yocto-compat-layer.py LAYER_DIR [YOCTO #10596] [1] https://www.yoctoproject.org/webform/yocto-project-compatible-registration [2] https://lists.yoctoproject.org/pipermail/yocto-ab/2016-October/001801.html Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04scripts/lib/create_npm: handle Public Domain licensesAnders Darander1
Rewrite Public Domain as PD, as that's what the place holder in meta/files/common_licenses is called. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04scripts/lib/create_npm: handle AND and OR in licensesAnders Darander1
Handle npm packages with multiple licenses (AND and OR). Prior to this, AND and OR were treated as licensed in their own. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04scripts/lib/create_npm: rewrite see license in eulaAnders Darander1
Rewrite the 'SEE LICENSE IN EULA' to a single string (without spaces), to avoid splitting the string later on. (Otherwise, each word gets split, and assumed to be a license on it's own. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04create_npm.py: convert MIT/X11 to MITAnders Darander1
Quite a few npm packages declare MIT/X11 as their license. This is equal to a pure MIT license. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-26 23:07:49 +0000