summaryrefslogtreecommitdiff
path: root/scripts/lib
AgeCommit message (Collapse)AuthorFiles
2015-12-01recipetool: make plugin registration function name consistent with devtoolPaul Eggleton4
This should have been register_commands rather than register_command; I used register_commands in devtool so lets change this here to be consistent with that. (Since this is extensible through layers though we need to remain compatible with the old name, so fall back to that if the new function name isn't there.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01recipetool: add setvar subcommandPaul Eggleton1
Add a recipetool subcommand "setvar" to set a variable in a file. This uses our existing logic such that it doesn't matter if the variable is already set in the recipe, if it's set in the recipe or some inc file, and if the variable is not currently set that the line setting the variable gets inserted in the right place in the file. Implements [YOCTO #7676]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()Paul Eggleton1
Use bb.utils.edit_metadata() to replace some of the logic in this function; this avoids us effectively having two implementations of the same thing. In the process fix the following issues: * Insert values before any leading comments for the next variable instead of after them * Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place * Properly handle replacing varflag settings (e.g. SRC_URI[md5sum]) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: clarify help textPaul Eggleton3
* Make some minor clarifications to help text * Drop ArgumentDefaultsHelpFormatter and just put the defaults in the text itself where needed (because otherwise you get defaults shown for store_true options which is somewhat confusing). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: build: enable showing default task in helpPaul Eggleton1
Enable access to the configuration object in register_commands() so that we can read configuration values there; this allows us to show the task that will be run in the command line help for the build subcommand. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: build: use bbappend to set PARALLEL_MAKEPaul Eggleton1
Use a bbappend file to set PARALLEL_MAKE instead of a postfile; this is a bit neater and only affects the specified recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: ensure we change back to the original dir on errorPaul Eggleton1
This is just belt-and-braces but we ought to use try..finally in this kind of situation, so just do it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: search: print SUMMARY valuePaul Eggleton1
Print the SUMMARY value for each matched item assuming it's not the default. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: drop unused plugin_init() functionsPaul Eggleton5
This function is no longer required to be defined for a plugin, so drop it where it's a no-op. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: package: use DEPLOY_DIR_<pkgtype> to get deploy directoryPaul Eggleton1
Rather than reconstructing the output path for packages, use the proper variable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: disable creating workspace for extract and search subcommandsPaul Eggleton2
For subcommands that don't actually involve the workspace, don't auto-create the workspace. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: provide a means to update the source branchPaul Eggleton1
If you're upgrading a git recipe to a revision on a release branch that's different to the branch for the current revision, then you'll need to update the branch parameter in SRC_URI, so add a --srcbranch/-B command-line parameter to let you do that easily. It handles both when the branch is stated verbatim in the recipe, and when a reference to another variable is used (a common convention is to use a SRCBRANCH variable for this, though the code doesn't care what variable is used if any). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fetch remote repository before checking out new revisionPaul Eggleton1
If we're upgrading a recipe that fetches from git, and we've simply fetched a tarball of the repo instead of directly from the upstream repo (this can happen if you have PREMIRRORS set up as in poky with a core recipe, e.g. kernelshark) then we won't have any new revisions, and the checkout will fail with "fatal: reference is not a tree: <hash>". To avoid this, do a "git fetch" before checking out the new revision. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: remove erroneous error when not renaming recipePaul Eggleton1
If we're upgrading a git recipe the recipe file usually won't need renaming; for some unknown reason we were throwing an error here which isn't correct. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fix updating PV and SRCREVPaul Eggleton1
This code was clearly never tested. Fix the following issues: * Actually set SRCREV if it's been specified * Enable history tracking and reparse so that we handle if variables are set in an inc file next to the recipe * Use a more accurate check for PV being in the recipe which will work if it's in an inc file next to the recipe Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fix removing other recipes from workspace on resetPaul Eggleton1
If you did a "devtool add" followed by "devtool upgrade" and then did a "devtool reset" on the recipe you upgraded, the first recipe would also be deleted from the workspace - this was because we were erroneously adding the entire "recipes" subdirectory and its contents to be tracked for removal on reset. Remove the unnecessary call to os.path.dirname() that caused this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: include do_patch in SRCTREECOVEREDTASKSTzu-Jung Lee1
The external source of kernel has been patched during the construction of git repository. Include the do_patch task in the SRCTREECOVEREDTASKS. Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01wic: exec_native_cmd: implement support for pseudoEd Bartosh2
Wic runs some tools through pseudo, which makes exec_native_cmd to fail and throw cryptic error message when tool is not baked: For example: Error: exec_cmd: 'export PSEUDO_PREFIX=/media/ssd/poky-build/tmp/sysroots/x86_64-linux/usr;export PSEUDO_LOCALSTATEDIR=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/ ... PSEUDO_PASSWD=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/ ... PSEUDO_NOSYMLINKEXP=1;/media/ssd/poky-build/tmp/sysroots/ ... mkfs.ext4 -F -i 8192 /var/tmp/wic/build/rootfs_platform.7.ext4 -L platform -d /media/ssd/poky-build/tmp/work/qemux86-poky-linux/core-image-minimal/... returned '1' instead of 0 Made exec_native_cmd aware of pseudo and properly report errors when command is not found. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-25devtool: add sync commandTzu-Jung Lee2
The sync command is similar to the extract command, except it fetches the sync'ed and patched branch to an existing git repository. This enables users to keep track the upstream development while maintaining their own local git repository at the same time. Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-25recipetool.append: don't choke on a trailing ; in a urlChristopher Larson1
Apparently bb.fetch.URI can't handle this at the moment. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24wic/utils/oe/misc.py: Preserve PATH when running native toolsRandy Witt1
Previously exec_native_cmd() would remove all items from PATH except for the native sysroot. This can cause issues for the tools that are created using create_wrapper(). Now instead of wiping out the PATH, run a sanity check to check if the command is in the native sysroot. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton2
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19devtool: extract: fix error handlingPaul Eggleton1
If recipe parsing fails we need to exit out instead of attempting to use the data object that's set to None, which resulted in a traceback. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03recipetool: create: fix change in path structure if --extract-to path existsPaul Eggleton1
If the directory specified by --extract-to exists, because we were using shutil.move() to move the temporary extracted directory to the specified path, a subdirectory was being created under that directory instead of moving the contents, which was a different result than if the directory didn't previously exist. We could try to always move the contents but that's complicated when any symlinks are involved; the simplest thing is just to remove the directory (which should be empty anyway) before moving the temporary directory across in its place. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-03devtool: update-recipe: avoid updating patches that have not changedPaul Eggleton1
Use "git cherry" against the original tag that we made when we extracted the source in order to find the revisions that are definitely new. This allows you to modify a commit in the middle of the series and then run devtool update-recipe and not have the subsequent patches unnecessarily modified. Fixes [YOCTO #8388]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: modify: use correct local files directory nameMarkus Lehtonen1
The name of the directory for local source files under srctree is 'oe-local-files', not 'local-files'. Fixes a bug that slipped through in b7ab82485e4514e07ab8a76e554da27ddc92e6c0. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: update-recipe: enable var history trackingMarkus Lehtonen2
Enable variable history tracking so that the variables are updated in the correct file - i.e. in the file they are already defined. [YOCTO #7715] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: modify: make bitbake use local files from srctreeMarkus Lehtonen1
This change makes it possible to have local files (non-remote SRC_URI files, i.e. files that are located in the "recipe space") under the srctree even if S!=WORKDIR. The files must be placed under the 'local-files' subdirectory. Complements the previous patch that imports local files into srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: better support for local source filesMarkus Lehtonen2
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: file mover function that creates target dirMarkus Lehtonen1
Helper function for replacing a pattern like: target_dir = os.path.dirname(target) bb.utils.mkdirhier(target_dir) shutil.move(source, target) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update_recipe: refactor patch generationMarkus Lehtonen1
Implement new function that handles patch file generation. The new function also does the discovery of new, updated and deleted patches. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29devtool: update-recipe: add new patches in correct orderMarkus Lehtonen1
When adding multiple new patches append them to SRC_URI in correct order so that they apply correctly. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-23devtool: upgrade: use shutil.move instead of os.renameMarkus Lehtonen1
Rename fails over filesystem boundaries. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: runqemu: avoid recipe parsePaul Eggleton1
We only need the base configuration to get the variable values we want to get here, there's no need to parse recipes / load the cache. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23devtool: second fix for running from a different directoryMarkus Lehtonen8
Do not change change current working directory permanently, but, only for the duration of tinfoil initialization instead. The previous fix caused very unintuitive behavior where using relative paths were solved with respect to the builddir instead of the current working directory. E.g. calling "devtool extract zlib ./zlib" would always create create srctree in ${TOPDIR}/zlib, independent of the users cwd. (From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add search commandPaul Eggleton1
Adds a subcommand to search to find the target recipe name providing some file or capability. This is implemented by searching on recipe name, package name, description, package contents (file names), and runtime file provides. For example: $ devtool search libGL mesa $ devtool search X11 xextproto libxxf86vm xf86driproto xf86vidmodeproto libxfixes xproto libx11 ... $ devtool search /bin/sed busybox sed This is particularly useful within the extensible SDK but is also made available in devtool alongside the build system. Note of course that because this searches pkgdata, useful results depend upon do_packagedata(_setscene) having executed for the recipe being searched for. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add basic means of running runqemu within the extensible SDKPaul Eggleton1
We ship the runqemu script and if we build QEMU itself within the extensible SDK, then it would be nice to be able to run it. This is a very thin wrapper around runqemu, supplying the machine and image name so the user doesn't need to. (This subcommand is only available within the extensible SDK since it only really makes sense there where it is otherwise hard to run runqemu directly.) Implements [YOCTO #6657]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool / recipetool: add handling for binary-only packagesPaul Eggleton2
Add a means of creating recipes for package files or archives that contain a directory structure to be installed verbatim, for example an rpm file. (We mostly just re-use bin_package here and skip some of the normal build system checks.) This support is available in "recipetool create" and "devtool add" which wraps the former. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton1
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: improve image recipe handlingPaul Eggleton1
* Make image optional for the extensible SDK (auto-determine it based on the targets the SDK was built for) * Check that specified recipe is in fact an image Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: tell user where to find output filesPaul Eggleton1
If the user is running "devtool build-image" within the extensible SDK then they probably won't know where to find the resulting output files, so we should tell them explicitly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: fix recipe/package terminologyPaul Eggleton1
We build recipes and include packages into the image, adjust the terminology used in code and messages accordingly. Also fix a few typos. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: move important "recipe created" message to the endPaul Eggleton1
If we end up printing a message about the build directory being the same as the source, we should print that first and then print the message about the recipe file possibly needing to be edited to the end so that it has slightly more impact. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: set up fetched source as a git repository by defaultPaul Eggleton2
If the fetched source isn't already a git repository, initialise it as one and then branch and tag, just as we do with "devtool modify". This makes it easier to make changes, commit them and then use the "devtool update-recipe" command to turn those commits into patches on the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: better handling for recipes that don't unpack sourcePaul Eggleton1
Some recipes don't extract any source (for example, opkg-keyrings). We were producing a traceback in this case because we weren't checking if the directory existed. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: fix extracting source for work-shared recipesPaul Eggleton1
Recipes that use work-shared (such as libgcc) are capable of unpacking the source, but it doesn't necessarily unpack to ${WORKDIR}/${BP}. Use the last part of the actual S value instead which is more likely to work. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: show proper error when extracting source for recipes with disabled ↵Paul Eggleton1
unpack task If you try to use "devtool modify -x" or "devtool extract" on a recipe where do_unpack has been set as noexec (e.g. glibc-locale), then we get an error because the expected source wasn't ever unpacked. Do a check up front for noexec being set on do_unpack and error out with a reasonable message if that's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix handling of URIs containing #Paul Eggleton1
The # character in a URI denotes a fragment; we don't care about this since it is never supposed to be sent to the server, so remove it from the URI before actually trying to fetch it or use it in SRC_URI within the recipe. (This has come up because download links on pypi.python.org seem to have a fragment containing the md5sum of the download; without stripping this off the fetcher will choke on it.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22recipetool: create: fix creating empty shell functionsPaul Eggleton1
The shell considers empty functions to be a syntax error, so for template shell functions that contain only comments (or no lines at all) then add a : to act as a no-op which avoids the syntax error. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: add: properly handle separate build directoryPaul Eggleton3
When we were adding a recipe for software that would typically be built in the same directory as the source, we were always using a separate build directory unless the user explicitly specified not to, leading to errors for software that doesn't expect to be built that way (such as Python modules using distutils). Split out the code that makes this determination automatically from the "devtool modify" and "devtool upgrade" code and re-use that here so the behaviour is consistent. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>