summaryrefslogtreecommitdiff
path: root/scripts/lib
AgeCommit message (Collapse)AuthorFiles
2016-01-19wic: removed unused importsEd Bartosh4
Removed imports of wic.kickstart from plugins as they're not used in the code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19wic: improve processing of parseing errorsEd Bartosh1
Caught argparse.ArgumentError Included .ks file name and line number into the error messages. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19wic: catch KickStartErrorEd Bartosh1
Catch parsing errors and output them using msger. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19wic: add custom exception KickStartErrorEd Bartosh1
This exception will be raised by kickstart parser on parsing errors and processed in the code which calls parser to produce meaningful error output. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic/help.py: document requirements for valid fstab generationJoshua Lock1
Without one of the --ondrive, --ondisk or --use-uuid options for a partition with a mountpoint specified the automatically generated fstab entry will be invalid. [YOCTO #8844] Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: get rid of 2 gettersEd Bartosh3
Used partitions and configfile bootloader attributes instead of using getters get_bootloader_file and get_partitions. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: get rid of set_size and set_source_file settersEd Bartosh3
Used size and source_file attributes instead of using setters. It's more pythonic, clear an consistent. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: get rid of get_rootfs and set_rootfsEd Bartosh4
Got rid of get_rootfs and set_rootfs java-like getter and setter. Renamed rootfs to rootfs_dir to be consistent with the name of kickstart parameter --rootfs-dir. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: get rid of get_timeout getterEd Bartosh4
Used bootloader.timeout instead of kickstart.get_timeout getter. Accessing attributes instead of getting them with getters is more pythonic, shorter and readable. It also more consistent as most of partition and bootloader attributes are used this way. This change also takes care of appendLine bootloader attribute: it's renamed to bootloader.append attribute provided by new parser. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: adjust code for new data structureEd Bartosh4
New data structure is less nested than old one. Adjusted bootloader and partitions fields: self.ks.handler.bootloader -> self.ks.bootoader self.ks.handler.partitions -> self.ks.partitions Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: remove pykickstart codeEd Bartosh19
Removed pykickstart-related code as it's replaced by new kickstart parser. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: use new kickstart parserEd Bartosh2
Used KickStart parser class instead of pykickstart API. This commit breaks wic as data structures and field names provided by new API are a bit different from old ones. This issue will be addressed in the following commits. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: add kickstart parser moduleEd Bartosh1
This module will replace existing pykickstart machinery it contains only option used by wic, it's simple and clear. And It will allow to remove a lot of old complex code from 3rdparty/pykickstart/ and kickstart/custom_commands. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-17wic: add partition moduleEd Bartosh1
Copied partition-related API from kickstart/custom_commands/partition.py to separate module in preparation for removal of all pykickstart related code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11populate_sdk_ext: Use new --setscene-only option to bitbake instead of ↵Richard Purdie1
workarounds Rather than horrible workarounds, use the new --setscene-only option of bitbake to pre-populate the SDK structures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: build: support using BBCLASSEXTENDed namesPaul Eggleton2
It's logical that you would want to build BBCLASSEXTENDed items separately through devtool build, so simply allow that - we're just passing the name verbatim to bitbake, so all it means is adjusting the validation. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: reset: support recipes with BBCLASSEXTENDPaul Eggleton2
If the recipe file itself was created in the workspace, and it uses BBCLASSEXTEND (e.g. through devtool add --also-native), then we need to clean the other variants as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: refactor code for getting local recipe filePaul Eggleton1
We're doing this in a couple of places, let's just find the recipe file if it exists within the workspace (which it will if it's been added through "devtool add") when we read in the workspace. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: add: support adding a native variantPaul Eggleton2
Sometimes you need to build a variant of a recipe for the build host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a --also-native command line option to "recipetool create" that enables this and plumb it through from an identical option for "devtool add". (We could conceivably do the same for nativesdk, but I felt it might be confusing within the context of the extensible SDK, where nativesdk isn't really relevant to the user.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: reset: do clean for multiple recipes at once with -aPaul Eggleton1
We need to run the clean for all recipes that are being reset before we start deleting things from the workspace; if we don't, recipes providing dependencies may be missing when we come to clean a recipe later (since we don't and couldn't practically reset them in dependency order). This also improves performance since we have the startup startup time for the clean just once rather than for every recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11recipetool: create: support creating standalone native/nativesdk recipesPaul Eggleton1
If the recipe name ends with -native then we should inherit native; likewise if it starts with nativesdk- then inherit nativesdk. (Note that the recipe name must actually be specified by the user in order to trigger this - we won't do it based on any name auto-detected from e.g. the tarball name.) Since we're doing this based on the name, "devtool add" will also gain this functionality automatically. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11recipetool: create: lower case name when determining from filenamePaul Eggleton1
As a matter of general convention we expect recipe names to be lower case; in fact some of the packaging backends insist upon it. Since in this part of the code we're auto-determining the name, we should convert the name to lowercase if it's not already so that we're following convention and avoiding any validation issues later on. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: sdk-update: add option to skip preparation stepPaul Eggleton1
For debugging purposes it's useful to be able to skip the preparation step so you can inspect what the state of the build system is first. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: sdk-update: fix error checkingPaul Eggleton1
Running "raise" with no arguments here is invalid, we're not in exception handling context. Rather than also adding code to catch the exception I just moved the check out to the parent function from which we can just exit. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: sdk-update: fix metadata update stepPaul Eggleton1
* Clone the correct path - we need .git on the end * Pull from the specified path instead of expecting a remote to be set * up in the repo already (it isn't by default) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11devtool: sdk-update: fix not using updateserver config file optionPaul Eggleton1
We read the updateserver setting from the config file but we never actually used that value - the code then went on to use only the value supplied on the command line. Fix courtesy of Dmitry Rozhkov <dmitry.rozhkov@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07wic: rawcopy: Copy source file to build folderNoor Ahsan1
When a file is given using --sourceparams then wic directly use that file instead of copying them to build folder. At time of assembling it os.rename is called which renames all the files to name. In that process the original file is renamed. When image recipe is rebuilt then wic complains about missing file which was renamed in previous build. [YOCTO #8854] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-28devtool: use cp instead of shutil.copytreeEd Bartosh1
Copied layers with 'cp -a' instead of calling shutil.copytree as copytree fails to copy broken symlinks. More pythonic fix would be to use copytree with 'ignore' parameter, but this could slow down copying complex directory structures. [YOCTO #8825] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22recipetool: create: fix error when extracting source to a specified directoryPaul Eggleton1
Having fetched the source and unpacked it to a temporary directory, we then move part of it to the destination directory, or if the source is at the top level we move the whole temporary directory, but in the latter case we were later attempting to delete the temporary directory which no longer existed. Clear out the variable so that doesn't happen. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: improve autotools supportPaul Eggleton1
* tar and binutils we can assume are there * libsocket is only relevant on BSD systems, so we can ignore it. * Detect more things implying gettext/intltool is needed * Detect glib-2.0 requirement. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: sync: tweak help / messagesPaul Eggleton1
Much of this was copy/pasted from the extract subcommand code; make it specific to sync. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: reset: print message about leaving source tree behindPaul Eggleton1
We deliberately leave the source tree alone when resetting in case it contains any work in progress belonging to the user; tell them that we're doing this so they aren't surprised about it still existing later on. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: status: list recipe file within workspace if one existsPaul Eggleton1
If a recipe in the workspace actually exists as a file within the workspace (e.g. after doing "devtool add" or "devtool upgrade") then show the path to the recipe file on the status line for the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: modify: default source tree pathPaul Eggleton1
As per the changes to "devtool add", make the source tree path optional and use the default path if none is specified. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: allow specifying URL as positional argumentPaul Eggleton2
Having to specify -f is a little bit ugly when a URI is distinctive enough to recognise amongst the other positional parameters, so take it as an optional positional parameter. -f/--fetch is still supported, but deprecated. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: figure out recipe name from recipetoolPaul Eggleton1
recipetool create now has all the logic in it for auto-detecting the name and version, and using those in the file name - so we can make the name an optional parameter for devtool add and we pick up the file name that recipetool has used after the fact. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: allow source tree to be omittedPaul Eggleton1
Assuming we're fetching source remotely (from a URI) we can default the source tree that will be extracted from it to a "sources" directory under the workspace in order to save the user specifying it if they don't have a preferred location. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22scripts/lib/argparse_oe: handle intermixing of optional positional argumentsPaul Eggleton1
Python's argparse module can't handle when several optional positional arguments (set with nargs='?') are intermixed with other options. If the positional arguments aren't optional then this isn't an issue; thus when changing positional arguments to optional (as we are doing with devtool) we need this workaround. This is a pretty horrible hack, but we don't want this flexibility of ordering to disappear simply because we made some arguments optional. Unfortunately the corresponding bug remains unresolved upstream even in Python 3, and argparse is not really designed to be subclassed so it doesn't make things like this easy. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: update-recipe: use correct method to get bbappend filenamePaul Eggleton1
The bbappend already exists at this point, so we know what its path is - there's no need to figure it out from scratch here. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: split out function for naming bbappendPaul Eggleton2
We're repeating this in a couple of places, so we might as well have a function to do it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: tweak help textPaul Eggleton1
Add a few clarifying words. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: edit-recipe: add new subcommandPaul Eggleton1
Add an "edit-recipe" subcommand that runs your default editor (as specified by the EDITOR environment variable) on the specified recipe in the workspace. Note that by default the recipe file itself must be in the workspace - i.e. as a result of "devtool add" or "devtool upgrade"; however there is a -a/--any-recipe option to override this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: basic extraction of name/version from filenamePaul Eggleton1
Often the filename (e.g. source tarball) contains the name and version of the software it contains. (This isn't intended to be exhaustive, just to catch the common case.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: support extracting name and version from build scriptsPaul Eggleton3
Some build systems (notably autotools) support declaring the name and version of the program being built; since we need those for the recipe we can attempt to extract them. It's a little fuzzy as they are often omitted or may not be appropriately formatted for our purposes, but it does work on a reasonable number of software packages to be useful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: set up priority system for recipe handlersPaul Eggleton1
Sometimes we want to force one handler to run before another; if the two handlers are in different plugins that's difficult without some kind of priority number, so add one and sort by it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: detect when specified URL returns a web pagePaul Eggleton1
If the user specifies a URL that just returns a web page, then it's probably incorrect (or broken); attempt to detect this and show an error if it's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: prevent attempting to unpack entire DL_DIRPaul Eggleton1
If you specify a URL ending in /, BitBake's fetcher returns a localpath of ${DL_DIR}, and if you then try to unpack that it will attempt to copy the entire DL_DIR contents to the destination - which at least on my system filled my entire /tmp. Obviously we should fix the fetcher, but at least detect and stop that from happening here for now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: minor fix for potential issue in python handlingPaul Eggleton1
If SRC_URI happened not to be in the pre-generated lines then this code would error out. This is unlikely to happen with the way the create code is structured at the moment, but handle it just in case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: fix do_install handling for makefile-only softwarePaul Eggleton1
In my testing here it appears make -qn returns an error (exit code 2) whereas make -n doesn't; I can't immediately tell why based on the documentation. We don't actually care for it to be quiet since we're capturing the output, so let's just leave -q off and have this work properly as a result. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: avoid traceback on fetch errorPaul Eggleton1
If a fetch error occurs, the fetcher already prints a reasonable error - we don't need the traceback as well, so catch that and exit if it occurs. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>