summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)AuthorFiles
2016-09-03scripts: ensure tinfoil is shut down correctlyPaul Eggleton10
We should always shut down tinfoil when we're finished with it, either by explicitly calling the shutdown() method or by using it as a context manager ("with ..."). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03scripts/contrib: update scripts for changes to internal APIPaul Eggleton3
The multiconfig changes altered some of the functions being called here, so update the calls. Make use of the new Tinfoil.parse_recipe_file() function to make parsing easier. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03lib/oe/patch: commit with a dummy user/email when PATCHTOOL=gitPaul Eggleton3
When using PATCHTOOL = "git", the user of the system is not really the committer - it's the build system itself. Thus, specify "dummy" values for username and email instead of using the user's configured values. Various parts of the devtool code that need to make commits have also been updated to use the same logic. This allows PATCHTOOL = "git" and devtool to be used on systems where git user.name / user.email has not been set (on versions of git where it doesn't default a value under this circumstance). If you want to return to the old behaviour where the externally configured user name / email are used, set the following in your local.conf: PATCH_GIT_USER_NAME = "" PATCH_GIT_USER_EMAIL = "" Fixes [YOCTO #8703]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oe-build-perf-test: rename log file and implement --log-fileMarkus Lehtonen1
Rename the (main) log file of the oe-build-perf-test script from 'output.log' to 'oe-build-perf-test.log'. Also, add a new command line option --log-file which makes it possible to use an alternative log file name/path, if needed. Note that the file name/path is relative to the output directory. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oe-build-perf-test: update globalres and git even if tests failedMarkus Lehtonen1
Write globalres log file and commit results to Git even if some tests failed. Now that tests do not depend on each other there should be no risk of bogus results caused by test failures. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25build-perf-test-wrapper.sh: make workdir configurableMarkus Lehtonen1
New command line argument '-w' may be used to specify work dir other than the default <GIT_DIR>/build-perf-test. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25build-perf-test-wrapper.sh: make archive dir configurableMarkus Lehtonen1
Add new command line argument '-a' that can be used to define the directory where results (tarballs) are archived. Giving an empty string disables archiving which makes sense if you store results in Git. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25build-perf-test-wrapper.sh: allow saving results in GitMarkus Lehtonen1
Add new command line argument '-C' that allows saving results in a Git repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25build-perf-test-wrapper.sh: parse args with getoptsMarkus Lehtonen1
Use getopts for parsing the command line. This changes the usage so that if a commit (to-be-tested) is defined it must be given by using '-c', instead of a positional argument. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: add {git_commit_count} keyword for --commit-results-tagMarkus Lehtonen1
Makes it possible to create easily sortable tags. Also, the default tag format is updated to use the new keyword. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: new {tag_num} keyword for --commit-results-tagMarkus Lehtonen1
This makes it possible to create numbered tags, where the "basename" of the tag is the same and the only difference is an (automatically) increasing index number. This is useful if you do multiple test runs on the same commit. For example, using: --commit-results-tag {tester_host}/{git_commit}/{tag_num} would give you tags something like: myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/0 myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/1 ... The default tag format is updated to use this new keyword in order to prevent unintentional tag name clashes. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: tag results committed to GitMarkus Lehtonen1
Create a Git tag when committing results to a Git repository. This patch also implements --commit-results-tag command line option for controlling the tag name. The value is a format string where the following fields may be used: - {git_branch} - target branch being tested - {git_commit} - target commit being tested - {tester_host} - hostname of the tester machine Tagging can be disabled by giving an empty string to --commit-results-tag. The option has no effect if --commit-results is not defined. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: pre-check Git repo when using --commit-resultsMarkus Lehtonen1
Do a pre-check on the path that is specified with --commit-results before running any tests. The script will create and/or initialize a fresh Git repository if the given directory does not exist or if it is an empty directory. It fails if it finds a non-empty directory that is not a Git repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: implement --commit-results-branchMarkus Lehtonen1
A new command line option for defining the branch where results are commited. The value is actually a format string accepting two field names: - {git_branch} expands to the name of the target branch being tested - {tester_host} expands to the hostname of the tester machine The option has no effect if --commit-results is not used. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: support committing results data to GitMarkus Lehtonen1
Implement a new command line option '--commit-results' which commits the test results data into a Git repository. The given path must be an existing initialized local Git repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: use absolute paths in cmdline argsMarkus Lehtonen1
This is safer as the current working directory may change. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25oe-build-perf-test: implement --run-tests optionMarkus Lehtonen1
Makes it possible to run only a subset of tests. NOTE: The tests currently have (unwritten) dependencies on each other so use this option with care. Mainly for debugging. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25python-3.5-manifest: Add argparse moduleFabio Berton1
Adding argparse module from Python's standard library. This allow use argparse without installing all python-misc modules. For compatibility, add python3-argparse as RDEPENDS to python3-misc. Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25python-3.5-manifest: Rename Queue module to queueFabio Berton1
The Queue module has been renamed to queue in Python 3. Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25devtool: build_image: Fix recipe filterOla x Nilsson1
The missing split() causes dev and dbg packages to match. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25devtool: Use the wildcard flag in update_recipe_patchOla x Nilsson1
The --wilcard-version flag was only used in the srcrev variant of the update-recipe command. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-24combo-layer: python3: fix UnicodeDecodeErrorEd Bartosh1
check_patch function opens patch file in text mode. This causes python3 to throw exception when calling readline(): UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa7 in position NNNN: invalid start byte Opening file in binary mode and using binary type instead of strings should fix this. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-23python-3.5-manifest.inc: the signal module RDEPENDS on enumRobert Yang1
Fixed: $ python3 >>> import signal Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/path/to/sdk/sysroots/x86_64-pokysdk-linux/usr/lib/python3.5/signal.py", line 4, in <module> from enum import IntEnum as _IntEnum ImportError: No module named 'enum' Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-20oe-build-perf-test: align log message format with testrunner outputMarkus Lehtonen1
The previous attempt on this was a bit erroneous, dropping time stamps completely although only the timestamp format should've been changed. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18devtool/recipetool/meta: Adapt to bitbake API changes for ↵Richard Purdie6
multi-configuration builds Unfortunately to implenent multiconfig support in bitbake some APIs had to change. This updates code in OE to match the changes in bitbake. Its mostly periperhal changes around devtool/recipetool [Will need a bitbake version requirement bump which I'll make when merging] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17oe-build-perf-test: simplify stderr log formatMarkus Lehtonen1
Remove timestamps from the stderr log in order to make the console output more readable, i.e. more in line with the output from unittest runner. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17oe-build-perf-test: set-up file logging as early as possibleMarkus Lehtonen1
So that the log file would not miss any records. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17oe-build-perf-test: suppress logger output when tests are being runMarkus Lehtonen1
Prevent logger from writing to stderr when the tests are being run by the TestRunner. During this time the logger output is only written to the log file. This way the console output from the script is cleaner and not mixed with possible logger records. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17oe-build-perf-test: use new unittest based frameworkMarkus Lehtonen1
Convert scripts/oe-build-perf-test to be compatible with the new Python unittest based buildperf test framework. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17oeqa.buildperf: add BuildPerfTestResult classMarkus Lehtonen1
The new class is derived from unittest.TextTestResult class. It is actually implemented by modifying the old BuildPerfTestRunner class which, in turn, is replaced by a totally new simple implementation derived from unittest.TestRunner. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17python3.5-manifest: Fixes several dependencies on the newest python3Alejandro Hernandez1
This patch adds the following packages: python3-enum (needed by python3-git), python3-selectors (needed by python3-subprocess), python3-signal (needed by python3-subprocess), and it also fixes the following ones with missing dependencies: python3-subprocess, python3-compression, python3-datetime [YOCTO #10127] [YOCTO #10124] [YOCTO #10122] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17graph-tool: update to new networkx API, be iterativeRoss Burton1
Update the dot parser to the new networkx API (using pydotplus to parse). Also, switch the path display to output the paths as they are found instead of collecting them into a list, so output appears sooner. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17python-3.5-manifest: Add some missing RDEPENDSKyle Russell1
ctype's util.py needs subprocess lang's inspect.py needs importlib.machinery math's random.py needs crypt's hashlib subprocess imports threading Signed-off-by: Kyle Russell <bkylerussell@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17gen-lockedsig-cache: ensure symlinks are dereferencedPaul Eggleton1
If you set up a local mirror in SSTATE_MIRRORS then you can end up with symlinks in SSTATE_DIR rather than real files. We don't want these symlinks in the sstate-cache prodcued by gen-lockedsig-cache, so dereference any symlinks before copying. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-17oe-buildenv-internal: hint at specifying bitbake path in error messagePaul Eggleton1
If you check out OE-Core and then run oe-init-build-env you get an error about not having bitbake checked out in a "bitbake" subdirectory, however it's possible to specify the bitbake path on the oe-init-build-env command line, so hint at that in the error message rather than implying it has to be in the default location. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-10runqemu: qemuzynqmp: Add Linux boot supportAlistair Francis2
Add support to direct boot Linux instead of just booting u-boot. Signed-off-by: Alistair Francis <alistair.francis@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-04create-pull-request: set subject automatically for cover latterRobert Yang1
Set cover letter's subject automatically as the patch's subject when there is only one patch. [YOCTO #9410] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-04create-pull-request: read remote from env var CPR_CONTRIB_REMOTERobert Yang1
So that we don't have specify "-u <contrib>" everytime, and CPR_CONTRIB_REMOTE can be overrided by -u. [YOCTO #9409] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-04create-pull-request: add option -a to auto pushRobert Yang1
Before this patch, we need two steps to create PULL: * Step 1, create branch: $ git push <contrib> <local_branch>:<remote_branch> * Step 2, create PULL: $ create-pull-request -u <contrib> -l <local_branch> -b <remote_branch> -r <local_branch>~<n> We can see that the args used in step 1 are in step 2, so we can use "create-pull-request -a" or set CPR_CONTRIB_AUTO_PUSH in to create the branch to simplify the steps. [YOCTO #9408] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-01uncovered: list uncovered python modulesEd Bartosh1
This bash script prints list of modules uncovered by oe-selftest or any other test that produces coverage report. It expects coverage report on its stdin and a directory to look for python modules as a command line parameter, e.g. coverage report --rcfile=build/.coveragerc | ./scripts/contrib/uncovered bitbake/ should print list of uncovered python modules from bitbake/ directory tree to stdout. [YOCTO #9809] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-28oe-selftest: simplifying log filenamesBenjamin Esquivel1
avoiding characters like ':' and making a clearer separation of the fields that compose the filename. Changing from: oe-selftest-2016-07-20_16:05:27.log to: oe-selftest-20160720-160527.log Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-28oe-selftest: export test results via xmlrunnerBenjamin Esquivel1
if available, use the xmlrunner for exporting the test results to a dir named the same than the log where the text results are stored. this means creating a dir with the name of the log (without the .log) and dumping there the xml files that indicate the results of each of the tests. if xmlrunner is not available then it will behave the same as before, no xml exports. [YOCTO#9682] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-28recipetool: create: fix greedy regex that broke support for github tarballsPaul Eggleton1
The regex here needs to be anchored to the end or it'll match longer URLs, which was exactly what I was trying to avoid. This regression was introduced in OE-Core revision 7998dc3597657229507e5c140fceef1e485ac402. Fixes [YOCTO #10023]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-25recipetool: record unknown license filesPaul Eggleton1
Add a comment to the recipe listing license files that were found but not able to be identified, so that the user can find and examine them by hand fairly easily. Fixes [YOCTO #9882]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25classes/populate_sdk_ext: filter sstate within the extensible SDKPaul Eggleton1
Use the new oe-check-sstate to filter the sstate artifacts shipped with the extensible SDK by effectively running bitbake within the produced eSDK and and getting it to tell us which tasks it will restore from sstate. This has several benefits: 1) We drop the *-initial artifacts from the minimal + toolchain eSDK. This still leaves us with a reasonably large SDK for this configuration, however it does pave the way for future reductions since we are actually filtering by what will be expected to be there on install rather than hoping that whatever cuts we make will match. 2) We verify bitbake's basic operation within the eSDK, i.e. that we haven't messed up the configuration 3) We verify that the sstate artifacts we expect to be present are present (at least in the sstate cache for the build producing the eSDK). Outside deletion of sstate artifacts has been a problem up to now, and this should at least catch that earlier i.e. during the build rather than when someone tries to install the eSDK. This does add a couple of minutes to the do_populate_sdk_ext time, but it seems like the most appropriate way to handle this. Should mostly address [YOCTO #9083] and [YOCTO #9626]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-25scripts: add oe-check-sstate scriptPaul Eggleton1
Add a script to check which sstate artifacts would be installed by building a given target - by default this is done with a separate TMPDIR to ensure we get the "from scratch" result. The script produces a list of tasks that will be restored from the sstate cache. This can also be combined with BB_SETSCENE_ENFORCE* to check if sstate artifacts are available. The implementation is a little crude - we're running bitbake -n and looking at the output. In future when we have the ability to execute tasks from tinfoil-based scripts we can look at rewriting that part of it to use that instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21wic: rawcopy: make source filenames uniqueEd Bartosh1
Rawcopy plugin copies source files to build folder before using them to assemble result image. After assembling the image wic renames source files to <image>.p<partition number>. If the same source file is used in multiple partitions wic breaks trying to rename file that doesn't exist. Added <line number> suffix to the files when copying them to the build dir. This should make filename unique even if the same source file is used for multiple partitions. [YOCTO #9826] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20oe-selftest: print errors when failed to find testRobert Yang1
For example: $ oe-selftest --run-tests-by name hello world 2016-07-12 00:33:28,678 - selftest - ERROR - Failed to find test: hello 2016-07-12 00:33:28,679 - selftest - ERROR - Failed to find test: world Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-20oe-git-proxy: don't depend on syslinuxAndré Draszik1
gethostip comes from syslinux. It seems odd to depend on a bootloader to clone a git repository. Switch to using getent from the c-library, which should be available on every system. We now also support the case where a hostname resolves to more than one IP address. Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-07-20devtool: add finish subcommandPaul Eggleton1
Add a subcommand which will "finish" the work on a recipe. This is effectively the same as update-recipe followed by reset, except that the destination layer is required and it will do the right thing depending on the situation - if the recipe file itself is in the workspace (e.g. as a result of devtool add), the recipe file and any associated files will be moved to the destination layer; or if the destination layer is the one containing the original recipe, the recipe will be overwritten; otherwise a bbappend will be created to apply the changes. In all cases the layer path can be loosely specified - it could be a layer name, or a partial path into a recipe. In the case of upgrades, devtool finish will also take care of deleting the old recipe. This avoids the user having to figure out the correct actions when they're done - they just do "devtool finish recipename layername" and it saves their work and then removes the recipe from the workspace. Addresses [YOCTO #8594]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-28 08:40:42 +0000