summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2017-02-28runqemu: support UEFI with OVMF firmwarePatrick Ohly1
In the simplest case, "runqemu qemux86 <some-image> qcow2 ovmf" for an EFI-enabled image in the qcow2 format will locate the ovmf.qcow2 firmware file deployed by the ovmf recipe in the image deploy directory, override the graphics hardware with "-vga std" because that is all that OVMF supports, and boot with UEFI enabled. ovmf is not built by default. Either do it explicitly ("bitbake ovmf") or make it a part of the normal build ("MACHINE_ESSENTIAL_EXTRA_RDEPENDS_append = ' ovmf'"). The firmware file is activated as a flash drive instead of using the qemu BIOS parameters, because that is the recommended method (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764918#47) as it allows storing UEFI variables in the file. Instead of just "ovmf", a full path to an existing file can also be used, just as with the rootfs. That may be useful when making a permanent copy of the virtual machine data files. It is possible to specify "ovmf*" parameters more than once, then each parameter creates a separate flash drive. This way it is possible to use separate flash drives for firmware code and variables: $ runqemu qemux86 <some-image> qcow2 ovmf.code ovmf.vars" Note that rebuilding ovmf will overwrite the ovmf.vars.qcow2 file in the image deploy directory. So when the goal is to update the firmware while keeping variables, make a copy of the variable file and use that: $ mkdir my-machine $ cp tmp/deploy/images/qemux86/ovmf.vars.qcow2 my-machine/ $ runqemu qemux86 <some-image> qcow2 ovmf.code my-machine/ovmf.vars.qcow2 When Secure Boot was enabled in ovmf, one can pick that instead of the non-Secure-Boot enabled ovmf.code: $ runqemu qemux86 <some-image> qcow2 ovmf.secboot.code my-machine/ovmf.vars.qcow2 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-28runqemu: also accept -image suffix for rootfs parameterPatrick Ohly1
The magic detection of the rootfs parameter only worked for image recipes which embedd the "image" string in the middle, as in "core-image-minimal". Sometimes it is more natural to call an image "something-image". To get such an image detected by runqemu, "-image" at the end of a parameter must also cause that parameter to be treated as the rootfs parameter. Inside the image directory, "something-image" has an -<arch> suffix and thus no change is needed for those usages of re.search('-image-'). However, while at it also enhance those string searches a bit (no need for re; any()+map() a bit closer to the intended logic). Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-28runqemu: fix undefined variable reference in check_arg_path()Patrick Ohly1
'arg' isn't defined, the right name there is 'p'. This fixes a rather obscure error message when that code path ends up being taken: $ runqemu some/existing-file-name runqemu - ERROR - name 'arg' is not defined runqemu - ERROR - Try 'runqemu help' on how to use it Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2017-02-23devtool: upgrade: fix error of import recipeutilsLuck Hoang1
upgrade.py imports oe.recipeutils in meta/lib/ but path to oe.recipeutils is not provided. This fails populate_sdk_ext. Signed-off-by: Luck Hoang <huyht1205@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23scripts/buildstats-diff: fix epoch numbers in --ver-diffMarkus Lehtonen1
Incorrect data was printed (recipe name instead of epoch number) when displaying changes in epoch. (From OE-Core rev: 5e2b1bfb684dc76963f692172f7457c2249c3266) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15wic: direct: move creation of PartitionedImage to __init__Ed Bartosh1
Moved creation of PartitionedImage object from DirectPlugin.create method to init. It makes the code a bit more readable and logical. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: move generation of part.realnum to PartitionedImageEd Bartosh1
Moved the code that generates real partition numbers from DirectPlugin to PartitionedImage. The idea is to have all code that works with partitions in PartitionedImage class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: add PartitionedImage.prepare methodEd Bartosh1
Moved code that calls prepare method of Partition objects from DirectPlugin to PartitionedImage.prepare. The idea is to have all code that works with partitions in PartitionedImage class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: set bootloader.source in the __init__Ed Bartosh1
Moved setting of bootloader source from do_create method to __init__ as it doesn't have anything to do with image creation. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: move UUID generation to PartitionedImageEd Bartosh1
Moved code that generates partition UUIDs from DirectPlugin to PartitionedImage class as it's more logical to have it there. This allows the code to be reused by other imager plugins. Got rid of having yet another list of partitions in PartitionedImage. Reused the list passed from DirectPlugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: add 'realnum' attribute to partitionEd Bartosh1
Replaced call of _get_part_num method with an attribute. This eliminates the need to call the method and loop over partitions every time we need to know realnum for partition. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: remove useless codeEd Bartosh1
Removed catching CreatorError and raising it again. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: don't catch ImagerErrorEd Bartosh1
Don't transform ImagerError exception into warning. Let wic to catch it on the upper level. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: direct: remove set_bootimg_dir setterEd Bartosh3
Removed java-like setter set_bootimg_dir. It's more pythonic to access public attributes directly. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: ksparser: set default disk to 'sda'Ed Bartosh1
Set default value of --ondisk to 'sda' to ensure we always have disk name for the partition. This is a first step of replacing --ondisk with disk <name> attribute of .wks. This is better as all partitions share the same disk. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: remove utils/oe/__init__.pyEd Bartosh1
This file and utils/oe folder are not needed anymore as all modules were removed or moved out of this directory. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: move PartitionedImage class to direct.pyEd Bartosh2
As PartitionedImage is only used in direct.py it makes sense to move it there. It's easier to maintain (and refactor) it this way. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: move disk operations to PartitionImage classEd Bartosh5
Disk operations were spread over DirectPlugin, DiskImage and Image code making the code hard to understand. Renamed Image class to PartitionedImage. Removed DiskImage class. Moved disk operations to PartitionedImage. There was an implicit support for multiple disks: if different devices were specified in .wks file (e.g. --ondisk sda and --ondisk sdb), wic would theoretically generate multiple images. This is quite confusing option and the code supporting it was broken for a long time. The same effect (multiple output images) can be achieved in obvious and clear way - by using multiple .wks files. This functionality was removed. PartitionedImage works only with one image. This makes the code less complex and easier to maintain. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-15wic: Add --exclude-path option to rootfs source plugin.Kristian Amlie5
It will omit the given path from the resulting partition, and if the given path ends in a slash, it will only delete the content, and keep the directory. Since mkfs only accepts whole directories as input, we need to copy the rootfs directory to the workdir so that we can selectively delete files from it. Since we want to use the copyhardlinktree() function, we need to put the generic oe lib in the module search path. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: remove unused argument scripts_pathEd Bartosh3
There is no need to pass scripts_path from main wic module down the stack as it's not used there. Removed scripts_path argument from DirectPlugin class and wic_create function. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: remove unused importEd Bartosh1
Removed unused import of wic.errors module. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: remove unused plugin attributesEd Bartosh1
Removed unused _disks, _disk_format and _disk_names attributes from DirectPlugin class. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: make sure layout_partitions is called onceEd Bartosh1
Removed artificial _partitions_layed_out attribute and unneeded call of layout_partitions method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: use the same partition object in direct and partitionedfsEd Bartosh3
Partition attributes were copied to the dictionary in partitionedfs code, which makes the code hard to follow. Used partition object passed from direct.py module as is in partitionedfs. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: direct: get rid of _get_parts getterEd Bartosh1
Replaced _get_parts getter with direct attribute access to self.parts Removed code that implicitly created partition if there are no partitions mentioned in .wks file Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: rename __create_partition and __add_diskEd Bartosh1
Renamed private methods with leading double underscores: __create_partition -> _create_partition __add_disk -> _add_disk There is no point to have those names mangled, one underscore is enough. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: get rid of __add_partitionEd Bartosh1
3 lines long private method __add_partition is called only from add_partition method. Merged them together to increase readability. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15wic: partitionedfs: merged __format_disks and createEd Bartosh1
Private method __format_disks is called only from create method making the code less readable. Merged the code into one method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15build-perf-test-wrapper.sh: fix issues with non-existing path argumentsMarkus Lehtonen1
Without the '-s' option realpath will error out if the given path does not exist. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oe-build-perf-test: drop --commit-resultsMarkus Lehtonen1
Drop support for committing results into Git repository. The functionality was not directly related to testing and feels unnecessary complication of the script. The functionality has been moved into a separate oe-git-archive script. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15build-perf-test-wrapper.sh: use oe-git-archiveMarkus Lehtonen1
Start to use the new helper script for archiving results data in Git. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15scripts/oe-git-archive: support creating bare repositoriesMarkus Lehtonen1
[YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15scripts: Implement oe-git-archiveMarkus Lehtonen1
A helper script for committing data to git and pushing it upstream. The motivation for the script stems from the need to archiving QA test results in git, but it could be used elsewhere, of course. The script needs to be run under an initialized build environment because it utilizes bitbake configuration information. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15classes: Drop now unneeded update_data callsRichard Purdie1
Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07devtool: improve parse failure handlingPaul Eggleton2
With the move to tinfoil2, the behaviour when parsing failed has changed a bit - exceptions are now raised, so handle these appropriately. Specifically when if parsing the recipe created when running devtool add fails, rename it to .bb.parsefailed so that the user can run bitbake afterwards without parsing being interrupted. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07recipetool: create: do not treat numbers in SCM URLs as versionsPaul Eggleton1
Numbers within SCM (e.g. git) URLs are extremely unlikely to be valid version numbers - more likely they are just part of the name, thus don't try to extract them and use them as the version - doing so causes pretty bad behaviour within devtool: --------- snip --------- $ devtool add https://github.com/inhedron/libtr50 NOTE: Fetching git://github.com/inhedron/libtr50;protocol=https... ... NOTE: Using default source tree path .../build/workspace/sources/libtr ... RecursionError: maximum recursion depth exceeded while calling a Python object --------- snip --------- (This was because ${PV} was being substituted into the URL, but PV's value was being set to include ${SRCPV}, so there was a circular reference.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07recipetool: create: properly handle npm optional dependenciesPaul Eggleton1
npm's package.json supports two types of dependencies - optionalDependencies and dependencies; in the code for creating a recipe from a non-npm source (e.g. a git repository) we were not handling optionalDependencies and thus when pointed at a node.js application outside of npm we weren't taking care of all dependencies. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05wic: fix call of serial_console_form_kargsEd Bartosh1
As syslinux module has been recently removed and serial_consloe_form_kargs became local API in rootfs_pcbios_ext plugin it should be called without syslinux. prefix. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05wic: isoimage-isohybrid: use wic-tools to get syslinux pathEd Bartosh1
wic-tools recipe specific sysroot contains syslinux as wic-tools depends on it. Used wic-tools target to get syslinux path should guarantee that syslinux is installed there and can be used. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05wic: flatten imager class hierarchyEd Bartosh4
wic code is hard to follow due to deep and twiggy class inheritance tree. Flatten imager tree: wic -> wic_create -> Creator -> DirectPlugin -> DirectImageCreator to wic -> wic_create -> DirectPlugin by removing Creator class and creator module merging DirectImageCreator into DirectPlugin Changed APIs to use the same parameters names. Passed parsed command line options as an object down the stack. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: split out typingAnders Darander1
This allows us to use typing.py without having to add the whole python3-misc package. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add ipaddress to python3-ioAnders Darander1
This allows us to use ipaddress without requiring the add the whole python3-misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add _compat_pickle to python3-pickleAnders Darander1
This allows us to depend on _compat_pickle.* wihtout having to add the whole python3-misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add socketserver to netserverAnders Darander1
socketserver.* should be part of python3-netserver. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add argparse to RDEPENDS for netclientAnders Darander1
http/server.py requires argparse. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python3-manifest: move htlm.py to python3-htmlAnders Darander1
This allows us to use html.py without importing misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: fix adding imp to importlibAnders Darander1
Commit: 512334f102a33833d39af53467894315f0715d07 "python-3.5-manifest: Add imp to importlib" added imp to importlib in the generated manifest, but not in the generator script. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: get rid of image_output_dir variableEd Bartosh1
Used options.outdir instead of image_output_dir. There is no sense to use extra variable for this. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: direct: fix creation of work directoryEd Bartosh1
It was a typo in current code: mktemp was used instead of mkdtemp to create work directory. This is fixed by using mkdtemp. Create work directory as a subdirectory of output directory to make sure both are on the same partition to make moving of result image faster. This also fixes possible disk space issues as mkdtemp uses TMPDIR, TEMP or TMP environment variables to get default value of its 'dir' parameter. Those variables are usually pointing to /tmp, which is not the best location to create huge images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: engine: create output dirEd Bartosh1
Make sure output directory exists before creating an image. Create it if it doesn't exist. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>