summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2016-02-21devtool: undeploy-target: support undeploying all recipesPaul Eggleton1
If you want to put the target device back to exactly how it was before devtool deploy-target started poking things into it, then it would make things easier if you didn't have to figure out which recipes were deployed. Now that we have the list stored on the target we can determine this reliably, so add a -a/--all option to undeploy-target to undeploy everything that has been deployed. One of the side-effects of this is that the dry-run functionality for undeploy-target had to be reimplemented to actually run the script on the target, since we have no way of knowing what's been deployed from the host side. We don't need to do the same for deploy-target though since we know exactly which files will be deployed without referring to the target. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: deploy-target: write deployed files list to targetPaul Eggleton1
When running devtool deploy-target, we save a list of deployed files, and this list is used by devtool undeploy-target (or the next time deploy-target is run if the list is present, in case any files have been renamed or deleted since the first time). We were writing this file to the host, but it makes more sense to write the list to the target instead, so that if we for example swap in a different board, or switch hosts, things will work as expected. In order to do this properly we have to construct a shell script and ship it over to the target so we can run it. The manifest is written out to a hidden directory in the root (/.devtool). Fixes [YOCTO #7908]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: sdk-update: tweak command-line handling of updateserverPaul Eggleton1
Get the default value for updateserver from the configuration file and show it in the help; also only make the parameter optional if it's specified. This means we can also drop the check in the function as argparse will then ensure it's specified if there's no config setting. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: (un)deploy-target: add help descriptionsPaul Eggleton1
Add a long description used when running --help on the specific command. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21scripts/lib/argparse_oe: tweak title above optionsPaul Eggleton1
Naming these as "optional arguments" is perhaps slightly confusing since some of the positional arguments might also be optional; in addition it's rare (though possible) for options to be mandatory - up until recently we had a recipetool option (-o) that was mandatory. It's not perfect, but change it to "options" so it's at least a bit more appropriate. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: categorise and order subcommands in help outputPaul Eggleton12
The listing of subcommands in the --help output for devtool was starting to get difficult to follow, with commands appearing in no particular order (due to some being in separate modules and the order of those modules being parsed). Logically grouping the subcommands as well as being able to exercise some control over the order of the subcommands and groups would help, if we do so without losing the dynamic nature of the list (i.e. that it comes from the plugins). Argparse provides no built-in way to handle this and really, really makes it a pain to add, but with some subclassing and hacking it's now possible, and can be extended by any plugin as desired. To put a subcommand into a group, all you need to do is specify a group= parameter in the call to subparsers.add_parser(). you can also specify an order= parameter to make the subcommand sort higher or lower in the list (higher order numbers appear first, so use negative numbers to force items to the end if that's what you want). To add a new group, use subparsers.add_subparser_group(), supplying the name, description and optionally an order number for the group itself (again, higher numbers appear first). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: update-recipe: don't show workspace recipe warning if no updatePaul Eggleton1
If we didn't make any changes to the file then there's no point warning the user that we have done. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: reset: fix preserving patches/other files next to recipesPaul Eggleton1
If files had been created next to the recipe (for example devtool add, edit the source and commit and then devtool update-recipe), running devtool reset failed to preserve those files and gave an error due to trying to rmdir the directory containing them which wasn't empty. Fix the preservation of files in the "attic" directory properly so we catch anything under the directory for the recipe, and replicate the same structure in the attic directory rather than slightly flattening it as we were before. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool / recipetool: use common code for launching editorPaul Eggleton3
Looking at Chris Larson's code for starting the user's editor for "recipetool newappend" it was slightly better than what I wrote for "devtool edit-recipe" in that it checks VISUAL as well as EDITOR and defaults to vi if neither are set, so break this out to its own function and call it from both places. The broken out version passes shell=True however in case it's a more complicated command rather than just a name of an executable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: minor fix for error messagePaul Eggleton1
There is no -N/--name option for devtool, that's a recipetool option - with devtool you just specify the name as a positional argument. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: Don't recursively look for .devtoolbase in --basepathRandy Witt1
If the user specifies --basepath on the commandline, only the directory specified should be searched for .devtoolbase. Otherwise when --basepath is a child of the sdk directory, .devtoolbase will always be found and devtool will only show options meant to be used within an sdk. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15linux-yocto: Work around PAT issue on qemux86Richard Purdie1
We have an issue with PAT handling on older processes with limited PAT bits, see the patch description for the full problem. This replaces the runqemu workaround with a kernel patch until we can get the kernel trees sorted out and discuss a proper fix with upstream. It should be safe everywhere so is applied unconditionally. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15wic: fix processing of --use-uuidEd Bartosh1
Processing of this option was lost during recent change of wks parsting. It was discovered during the work on booting wic images under qemu. Now, when -use-uuid is fixed it's possible to specify root partition by partition uuid. This will be done in the following commit. (From OE-Core rev: b4882e0b84d7fd4c85ee95386e94722485eafc2b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: support path/to/<image>-<machine>.wicEd Bartosh1
Supported providing wic image path to runqemu: runquemu path/to/<image>-<machine>.wic [YOCTO #8691] (From OE-Core rev: 58a3bfb1e4b493200820cdf0bf3fc79e31e792de) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: don't set KERNEL for wic imagesEd Bartosh2
Wic images should be boot as is, without pointing qemu to the kernel binary. Current code doesn't use kernel, but sets KERNEL variable and shows kernel path in the console output. This can confuse users. Changed runqemu and runqemu-internal code to avoid setting KERNEL variable and show kernel path. (From OE-Core rev: 474caa7ed5ff05caa5d49d270b283882fa616ed1) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: add support for wic imagesEd Bartosh2
Quemu should be able to run wic images this way: runqemu <machine> <image recipe> wic Tested with 'runqemu qemux86-64 wic-image-minimal wic' (From OE-Core rev: 8716be799949cb8bde7fa49cbea61312a3a93bb7) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15scripts/sstate-cache-management.sh: Change wordingJan Sarenik1
Use "out of" rather than "from" in the output. Signed-off-by: Jan Sarenik <jajomojo@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-12scripts/runqemu-internal: Work around qemux86 PAT bugs in linux 4.4.1Richard Purdie1
Since the upgrade of qemux86 to 4.4.1 we're seeing PAT issues when starting the X server. We need to fix the problem but the failing sanity tests mask out other issues and we need a workaround. Merge this for now until we can figure out the full issue. This is better than changing the kernel defconfig or reverting to old kernel versions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: set S when we set SRC_URI from local git repoPaul Eggleton1
If you specify a local directory which happens to be a git repository with an origin remote (and it is in fact remote), we can use that for SRC_URI as implemented by OE-Core revision b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S if the recipe is going to be of any use fetching from that SRC_URI later. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: convert http git URLs that don't end in .git but contain ↵Paul Eggleton1
/git/ When recipetool create is given a URL that starts with http(s):// and contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe to assume it's a git repository and not something that should be fetched with wget, so rewrite the URL. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: ensure URL parameters don't make it into the namePaul Eggleton1
When auto-detecting the name for a recipe from the URL, strip off any parameters (";name=value...") before parsing the URL, otherwise this just ends up in the recipe name which is undesirable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: add: fix adding from a local source directoryPaul Eggleton1
Fix a regression introduced in in OE-Core revision aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source tree without specifying a name resulted in a traceback. Fixes [YOCTO #9086]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: modify: make -x the default behaviourPaul Eggleton1
It's going to be more common for users not to have the prepared source tree for a recipe already, so the default behaviour ought to be to extract it for them from the recipe. Change the default to extract (effectively making the -x option a no-op) and add a --no-extract/-n option to disable it. Later we can look at trying to be smart and reusing an existing source tree instead of erroring out if it exists; for now this is just the default reversal. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: determine name/version from github/bitbucket URLsPaul Eggleton1
Both BitBucket and GitHub provide "release" downloads that unfortunately don't include the actual project name. For recipetool this means we have to have special handling for these URLs or else the current code assumes the name of the project is something like "v0.9.1". Borrow and adapt some code from autospec to do this, and put it in its own function for clarity. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: support cmake find_library directivePaul Eggleton1
CMake supports a find_library() directive to find named libraries, so detect dependencies from this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: commit for extra tasks that modify source when extractingPaul Eggleton1
When extracting source for a recipe, if there are additional custom tasks run that make changes to the source, create a commit in the generated git branch so they are contained. This is particularly useful for tasks that come before do_patch since otherwise the changes might get incorporated in the first applied patch, but otherwise it helps avoid the tree being dirty at any point. Fixes [YOCTO #7626]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool.newappend: add -e/--edit argumentChristopher Larson1
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-11python: merge python-elementtree into python-xmlRoss Burton2
python-xml depends on python-elementtree as the latter just contains a C library used by the former. However there's no point to this split apart from increasing the number of packages, so merge -elementtree into python-xml. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07isoimage-isohybrid.py: fix cpio working directoryIoan-Adrian Ratiu1
Take `pwd` to be <initial-dir>. The %s path is relative to it. The value of %s is "output_folder/build". The current code works as follows: Changing directory to %s and finding the sources (after cd'ing) to cpio with output redirection to %s/initrd.cpio triggers the following error "Error: exec_cmd: cd output_folder/build/INITRD && find . | cpio -o -H newc >output_folder/build/initrd.cpio returned '1' instead of 0" This happens because after the cd, `pwd` is <initial-dir>/%s and by the redirect we write the result to to <initial-dir>/%s/%s/initrd.cpio which obviously does not exist. Fix this by getting the sources with "find %s" instead of "cd && find ." Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-06scripts/devtool: Add getVarFlag expand argumentRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04python-distutils: add missing dependency on python-emailDominique Hunziker2
Signed-off-by: Dominique Hunziker <dominique.hunziker@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-04wic: isoimage-isohybrid: check for syslinux-nativeMihaly Varga1
.iso image creation fails if during the image creation syslinux is baked and syslinux-native is not. Added new check to verify if both syslinux and syslinux-native are baked and bake them if these are not installed. Signed-off-by: Mihaly Varga <mihaly.varga@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-04wic: default to empty bootloader configMaciej Borzecki1
A kickstart file for non-x86 boards may have no 'bootloader' stanza. It is the usual case if bootloader is setup using other mechanism than through wic, and is for instance a part of u-boot configuration. In such case the 'bootloader' field in the KickStart class will be uninitialized. Instead of adding an empty bootloader line in every kickstart file call the bootloader parser with empty argument list to get defaults namespace. Signed-off-by: Maciej Borzecki <maciej.borzecki@open-rnd.pl> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-29bitbake-whatchanged: avoid double do_ task name prefixOlof Johansson1
When used with --verbose, the heading for each task looks like === The verbose changes of example.do_do_compile: This should instead be === The verbose changes of example.do_compile: Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-26gen-lockedsig-cache: fix bad destination path joiningPaul Eggleton1
When copying the sstate-cache into the extensible SDK, if the source path had a trailing / and the destination path did not, there would be a missing / between the path and the subdirectory name, and you'd end up with subdirectories like "sstate-cacheCentOS-6.7". There are functions in os.path for this sort of thing so let's just use them and avoid the problem. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-26scripts/wipe-sysroot: also delete uninative sysrootRoss Burton1
The uninative sysroot is in ${STAGING_DIR}-uninative so delete that alongwith $STAGING_DIR. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-26devtool: extract: use the correct datastore for builddirMarkus Lehtonen1
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-26devtool: add configure-help subcommandPaul Eggleton2
When you need to set EXTRA_OECONF for a recipe, you need to know what options the configure script actually supports; the configure script however is only accessible from within a devshell and (at least in the case of autotooled software fetched from an SCM repository) may not actually exist until do_configure has run. Thus, provide a "devtool configure-help" subcommand that runs the configure script for a recipe with --help and shows you the output through a pager (e.g. less), prefaced by a header describing the current options being specified. There is basic support for autotools, cmake and bare configure scripts. The cmake support is a little hacky since cmake doesn't really have a concise help option that lists user-defined knobs (without actually running through the configure process), however that being a design feature of cmake there's not much I can think of to do about that at the moment. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26devtool: properly handle bb.build.FuncFailed when extracting sourcePaul Eggleton1
When we run the tasks required to extract the source for a recipe (e.g. within "devtool modify" or "devtool extract") if one of those tasks fails you get a bb.build.FuncFailed exception; handle this properly so you don't see a traceback. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26devtool: add: warn if modified recipe found in attic directoryPaul Eggleton1
If a recipe generated by "devtool add" has been modified since then when you run "devtool reset", it will be moved into the "attic" subdirectory of the workspace in case those modifications need to be preserved. It seems natural that if those modifications were worth preserving we should warn the user if such a file exists when they run "devtool add" to create the same recipe again, so they can pick up where they left off if they want to. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26devtool: build-image: allow specifying packages to add to imagePaul Eggleton1
Provide an option to devtool build-image to specify the list of packages instead of taking the list of packages produced by recipes in the workspace. Sometimes you don't want all of these packages; other times you want to add more. This is the most immediate fix for [YOCTO #8855], though it is a little crude so I would like to provide better means of customising the image contents later. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26devtool: move edit-recipe to a separate modulePaul Eggleton2
standard.py is getting a bit large; move the "utility" commands to another module. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24build-perf-test.sh: add eSDK testingMarkus Lehtonen1
Add simple initial eSDK test. Currently, only download size and installation time of eSDK is measured. The eSDK to be tested is generated from the same image that the other tests are run for. This patch will add two new fields to the global results log and that needs to be taken into account when examining the results. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24build-perf-test.sh: more generic timing functionMarkus Lehtonen1
Make it possible to time also other than bitbake commands. The name of the log file is changed from bitbake.log to commands.log. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22recipetool: create: better fix for fetch error handlingPaul Eggleton1
I was a little bit hasty in OE-Core revision c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's fetcher code is not consistent in whether it logs something useful or not; when fetching from an http URL it does but with a git repository it doesn't. In advance of any major reworking of fetch error handling in BitBake, let's just print the text of the exception and then we know we have shown something to the user. Additionally, we were only catching FetchException here but there are several other classes of exception that the fetcher can raise (e.g. MalformedUrl); catch the parent BBFetchException class instead so we avoid tracebacks for those other classes as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22recipetool: create: fix extraction of name from URLs ending in /Paul Eggleton1
If the URL ends in a / then we want to strip that off the path we split out of the URL before calling os.path.basename() on it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22recipetool: create: extract SRC_URI from local git repositoriesPaul Eggleton1
If you specify a local directory which happens to be a git repository with an origin remote (and it is in fact remote), we can use that for SRC_URI rather than leaving it blank in the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22devtool / recipetool: support specifying a subdirectory within the fetched ↵Paul Eggleton2
source Sometimes you don't want to build an entire project, just a subdirectory of it; add a --src-subdir option to make that easier. (We still look for a single subdirectory in what gets unpacked, e.g. what you might find within a tarball, so whatever you specify with this option is added onto the end of that.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22recipetool: create: strip quotes from values extracted from CMakeLists.txtPaul Eggleton1
Quoting is optional in CMakeLists.txt and is occasionally used, so strip out quotes if they are present. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22gen-lockedsig-cache: copy correct native sstate into ext SDKPaul Eggleton1
When constructing the sstate-cache directory for the extensible SDK, we were copying in any matching native sstate packages, and as the signature doesn't actually change when the distro changes (since NATIVELSBSTRING is just a path separator for the artifacts and is not part of the signature) we ended up copying duplicated packages when the distro changed e.g. upon host distro upgrade. Only search in the NATIVELSBSTRING-named subdirectory for native packages and the issue goes away. Fixes [YOCTO #8885]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>