summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2017-03-04wic: raise WicError instead of ImageError and CreatorErrorEd Bartosh4
There is no need to raise special exceptions. Raising WicError should be enough. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: raise WicError in wic pluginsEd Bartosh9
Replaced sys.exit with raising WicError in wic plugins. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: raise WicError in core modulesEd Bartosh4
Replaced sys.exit with raising WicError in the core wic modules. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: raise WicError in main moduleEd Bartosh1
Replaced sys.exit with raising WicError in main module. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: move errors moduleEd Bartosh6
Moved from lib/wic/utils/ to lib/wic as this is a core module. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: remove msger moduleEd Bartosh2
Removed custom logger module msger as it's replaced by wic logger. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: use wic logger in wic source pluginsEd Bartosh8
Replaced msger with wic logger in wic source plugins. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: use wic logger in imager direct pluginEd Bartosh1
Replaced msger with wic logger in the direct plugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: use wic logger in core modulesEd Bartosh8
Replaced msger with wic logger in the core wic modules. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-04wic: setup logging in the main wic moduleEd Bartosh1
Set up wic logger using standerd logging module. This is going to replace custom msger module. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-01devtool/sdk: Run build-sysroots after installing new thingsRichard Purdie1
After running sdk-install we need to ensure that the standalone sysroots are updated as done when the eSDK is originally built. Add such a call so this happens automatically and the envrionment scripts in the SDK work correctly after updates. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01generate-manifest-2.7.py: fix python-tests rdependsMing Liu1
A potential flaw was introduced by commit a8dc4bc0: [ python: fix python-tests rdepends ] It made python-tests RDEPENDS on python-modules which is correct, but this should be done in generate-manifest-2.7.py, and re-generate the manifest. Fixed by adding this dependency in generate-manifest-2.7.py as well. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01generate-manifest-3.5.py: add logic to generate native manifestMing Liu1
python3-native supposes to RPROVIDE all native packages as added in generate-manifest-3.5.py, but it does not so far, this leads a problem that sometimes bitbake cant find a runtime provider for a python3-*-native when a new runtime dependency on it being required, this usualy happens after a new native python3-* recipe is created or the old native python3-* recipes are upgraded. To avoid manually extending RPROVIDE every time when a new runtime dependency is introduced, an argument '-n/--native' is added to the manifest generator, allowing it create a native python3 manifest, with a RPROVIDE line only, the RPROVIDE should contain all the sub-packages. The generated python-native-3.5-manifest.inc is also added which is included by python3-native recipe. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01generate-manifest-2.7.py: add logic to generate native manifestMing Liu1
python-native supposes to RPROVIDE all native packages as added in generate-manifest-2.7.py, but it does not so far, this leads a problem that sometimes bitbake cant find a runtime provider for a python-*-native when a new runtime dependency on it being required, this usualy happens after a new native python-* recipe is created or the old native python-* recipes are upgraded. To give a example, the following commit is trying to address such a issue: commit 4583cd1bb15306e8f0ab7bcd80732e6f35aa4533: [ python-native: Make python-native also RPROVIDE python-unittest-native ] To avoid manually extending RPROVIDE every time when a new runtime dependency is introduced, an argument '-n/--native' is added to the manifest generator, allowing it create a native python manifest, with a RPROVIDE line only, the RPROVIDE should contain all the sub-packages. The generated python-native-2.7-manifest.inc is also added which is included by python-native recipe. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01scripts/lib/devtool/deploy.py: add --port/-P argument for target connectionTim Orling1
Enable using, e.g. host port 2222 for connection to qemu target. Defaults to 22 for standard ssh/scp port. [YOCTO #11079] Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-28runqemu: support UEFI with OVMF firmwarePatrick Ohly1
In the simplest case, "runqemu qemux86 <some-image> qcow2 ovmf" for an EFI-enabled image in the qcow2 format will locate the ovmf.qcow2 firmware file deployed by the ovmf recipe in the image deploy directory, override the graphics hardware with "-vga std" because that is all that OVMF supports, and boot with UEFI enabled. ovmf is not built by default. Either do it explicitly ("bitbake ovmf") or make it a part of the normal build ("MACHINE_ESSENTIAL_EXTRA_RDEPENDS_append = ' ovmf'"). The firmware file is activated as a flash drive instead of using the qemu BIOS parameters, because that is the recommended method (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764918#47) as it allows storing UEFI variables in the file. Instead of just "ovmf", a full path to an existing file can also be used, just as with the rootfs. That may be useful when making a permanent copy of the virtual machine data files. It is possible to specify "ovmf*" parameters more than once, then each parameter creates a separate flash drive. This way it is possible to use separate flash drives for firmware code and variables: $ runqemu qemux86 <some-image> qcow2 ovmf.code ovmf.vars" Note that rebuilding ovmf will overwrite the ovmf.vars.qcow2 file in the image deploy directory. So when the goal is to update the firmware while keeping variables, make a copy of the variable file and use that: $ mkdir my-machine $ cp tmp/deploy/images/qemux86/ovmf.vars.qcow2 my-machine/ $ runqemu qemux86 <some-image> qcow2 ovmf.code my-machine/ovmf.vars.qcow2 When Secure Boot was enabled in ovmf, one can pick that instead of the non-Secure-Boot enabled ovmf.code: $ runqemu qemux86 <some-image> qcow2 ovmf.secboot.code my-machine/ovmf.vars.qcow2 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-28runqemu: also accept -image suffix for rootfs parameterPatrick Ohly1
The magic detection of the rootfs parameter only worked for image recipes which embedd the "image" string in the middle, as in "core-image-minimal". Sometimes it is more natural to call an image "something-image". To get such an image detected by runqemu, "-image" at the end of a parameter must also cause that parameter to be treated as the rootfs parameter. Inside the image directory, "something-image" has an -<arch> suffix and thus no change is needed for those usages of re.search('-image-'). However, while at it also enhance those string searches a bit (no need for re; any()+map() a bit closer to the intended logic). Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-28runqemu: fix undefined variable reference in check_arg_path()Patrick Ohly1
'arg' isn't defined, the right name there is 'p'. This fixes a rather obscure error message when that code path ends up being taken: $ runqemu some/existing-file-name runqemu - ERROR - name 'arg' is not defined runqemu - ERROR - Try 'runqemu help' on how to use it Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-23devtool: upgrade: fix error of import recipeutilsLuck Hoang1
upgrade.py imports oe.recipeutils in meta/lib/ but path to oe.recipeutils is not provided. This fails populate_sdk_ext. Signed-off-by: Luck Hoang <huyht1205@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23scripts/buildstats-diff: fix epoch numbers in --ver-diffMarkus Lehtonen1
Incorrect data was printed (recipe name instead of epoch number) when displaying changes in epoch. (From OE-Core rev: 5e2b1bfb684dc76963f692172f7457c2249c3266) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15wic: direct: move creation of PartitionedImage to __init__Ed Bartosh1
Moved creation of PartitionedImage object from DirectPlugin.create method to init. It makes the code a bit more readable and logical. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: move generation of part.realnum to PartitionedImageEd Bartosh1
Moved the code that generates real partition numbers from DirectPlugin to PartitionedImage. The idea is to have all code that works with partitions in PartitionedImage class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: add PartitionedImage.prepare methodEd Bartosh1
Moved code that calls prepare method of Partition objects from DirectPlugin to PartitionedImage.prepare. The idea is to have all code that works with partitions in PartitionedImage class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: set bootloader.source in the __init__Ed Bartosh1
Moved setting of bootloader source from do_create method to __init__ as it doesn't have anything to do with image creation. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: move UUID generation to PartitionedImageEd Bartosh1
Moved code that generates partition UUIDs from DirectPlugin to PartitionedImage class as it's more logical to have it there. This allows the code to be reused by other imager plugins. Got rid of having yet another list of partitions in PartitionedImage. Reused the list passed from DirectPlugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: add 'realnum' attribute to partitionEd Bartosh1
Replaced call of _get_part_num method with an attribute. This eliminates the need to call the method and loop over partitions every time we need to know realnum for partition. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: remove useless codeEd Bartosh1
Removed catching CreatorError and raising it again. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: don't catch ImagerErrorEd Bartosh1
Don't transform ImagerError exception into warning. Let wic to catch it on the upper level. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: remove set_bootimg_dir setterEd Bartosh3
Removed java-like setter set_bootimg_dir. It's more pythonic to access public attributes directly. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: ksparser: set default disk to 'sda'Ed Bartosh1
Set default value of --ondisk to 'sda' to ensure we always have disk name for the partition. This is a first step of replacing --ondisk with disk <name> attribute of .wks. This is better as all partitions share the same disk. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: remove utils/oe/__init__.pyEd Bartosh1
This file and utils/oe folder are not needed anymore as all modules were removed or moved out of this directory. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: move PartitionedImage class to direct.pyEd Bartosh2
As PartitionedImage is only used in direct.py it makes sense to move it there. It's easier to maintain (and refactor) it this way. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: move disk operations to PartitionImage classEd Bartosh5
Disk operations were spread over DirectPlugin, DiskImage and Image code making the code hard to understand. Renamed Image class to PartitionedImage. Removed DiskImage class. Moved disk operations to PartitionedImage. There was an implicit support for multiple disks: if different devices were specified in .wks file (e.g. --ondisk sda and --ondisk sdb), wic would theoretically generate multiple images. This is quite confusing option and the code supporting it was broken for a long time. The same effect (multiple output images) can be achieved in obvious and clear way - by using multiple .wks files. This functionality was removed. PartitionedImage works only with one image. This makes the code less complex and easier to maintain. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: Add --exclude-path option to rootfs source plugin.Kristian Amlie5
It will omit the given path from the resulting partition, and if the given path ends in a slash, it will only delete the content, and keep the directory. Since mkfs only accepts whole directories as input, we need to copy the rootfs directory to the workdir so that we can selectively delete files from it. Since we want to use the copyhardlinktree() function, we need to put the generic oe lib in the module search path. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: remove unused argument scripts_pathEd Bartosh3
There is no need to pass scripts_path from main wic module down the stack as it's not used there. Removed scripts_path argument from DirectPlugin class and wic_create function. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: remove unused importEd Bartosh1
Removed unused import of wic.errors module. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: remove unused plugin attributesEd Bartosh1
Removed unused _disks, _disk_format and _disk_names attributes from DirectPlugin class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: make sure layout_partitions is called onceEd Bartosh1
Removed artificial _partitions_layed_out attribute and unneeded call of layout_partitions method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: use the same partition object in direct and partitionedfsEd Bartosh3
Partition attributes were copied to the dictionary in partitionedfs code, which makes the code hard to follow. Used partition object passed from direct.py module as is in partitionedfs. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: get rid of _get_parts getterEd Bartosh1
Replaced _get_parts getter with direct attribute access to self.parts Removed code that implicitly created partition if there are no partitions mentioned in .wks file Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: rename __create_partition and __add_diskEd Bartosh1
Renamed private methods with leading double underscores: __create_partition -> _create_partition __add_disk -> _add_disk There is no point to have those names mangled, one underscore is enough. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: get rid of __add_partitionEd Bartosh1
3 lines long private method __add_partition is called only from add_partition method. Merged them together to increase readability. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: merged __format_disks and createEd Bartosh1
Private method __format_disks is called only from create method making the code less readable. Merged the code into one method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15build-perf-test-wrapper.sh: fix issues with non-existing path argumentsMarkus Lehtonen1
Without the '-s' option realpath will error out if the given path does not exist. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oe-build-perf-test: drop --commit-resultsMarkus Lehtonen1
Drop support for committing results into Git repository. The functionality was not directly related to testing and feels unnecessary complication of the script. The functionality has been moved into a separate oe-git-archive script. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15build-perf-test-wrapper.sh: use oe-git-archiveMarkus Lehtonen1
Start to use the new helper script for archiving results data in Git. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15scripts/oe-git-archive: support creating bare repositoriesMarkus Lehtonen1
[YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15scripts: Implement oe-git-archiveMarkus Lehtonen1
A helper script for committing data to git and pushing it upstream. The motivation for the script stems from the need to archiving QA test results in git, but it could be used elsewhere, of course. The script needs to be run under an initialized build environment because it utilizes bitbake configuration information. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15classes: Drop now unneeded update_data callsRichard Purdie1
Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07devtool: improve parse failure handlingPaul Eggleton2
With the move to tinfoil2, the behaviour when parsing failed has changed a bit - exceptions are now raised, so handle these appropriately. Specifically when if parsing the recipe created when running devtool add fails, rename it to .bb.parsefailed so that the user can run bitbake afterwards without parsing being interrupted. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>