summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2017-03-21oe-selftest: tinfoil: add tests for recently enabled datastore operationsPaul Eggleton1
A recent patch to bitbake fixes these datastore operations so that they actually affect the server end, so we should test that they work. (For full disclosure, some of these tests would probably pass without those fixes, since the operation would be done on the client side instead - but we are at least exercising the code paths.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21meta/lib/oeqa: change sdk test from cvs to cpiobrian avery2
We currently fetch,configure,build, and install cvs as our test for the sdk. cvs unfortunately, requires a default editor in order to run. The change in 94790a8254d6 that checks to see if you have something like vi installed is fragile since you may have a different default editor. This patch switches from using cvs as a test to using cpio. cpio also uses autotools so the functionality tested is equivalent. Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21selftest/devtool: Add test to verify "modify virtual/kernel"Jose Perez Carranza1
The purpose of this test case is to verify that devtool modify works correctly when building the kernel. [YOCTO #10817] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21qemurunner: configure guest networkingEd Bartosh1
Configured guest network interface through serial connection when kernel is not run by qemu. This should make it possible to test wic images with testimage. [YOCTO #10833] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21qemurunner: get network params from runqemu outputEd Bartosh1
Parsed runqemu output to get guest network configuration if it's not present in runqemu command line. [YOCTO #10833] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17dnf: add /usr/bin/dnf symlink that points to /usr/bin/dnf-2Alexander Kanavin1
All documentation refers to dnf binary as 'dnf' yet make install does not create one - it's done by Fedora's spec file when building the rpm. Let's replicate this behavior. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17selftest/signing.py: verify that images can be created from signed packagesAlexander Kanavin1
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17rpm: add support for remote package feeds via PACKAGE_FEED_URIS variableAlexander Kanavin1
I've used a previous patch (which was never merged) by Humberto Ibarra <humberto.ibarra.lopez@intel.com> as a model for how to do runtime testing of this feature (e.g. we need to boot an image, run dnf on it, and check that it is indeed able to access the remote repo over http). Here's his original commit message: ===== Testing that feeds specified with PACKAGE_FEED_URIS var are set correctly has two parts. First a build with this var set is required, and then smart update needs to be issued in the running taget. The previous is not a common selftest practice because this is a simple test, but requires building and running a specific image, which takes a lot of time. testimage is not a good fit either, since the images tested there do not have the PACKAGE_FEED_URIS var set. For this test, the runtime-test module is being used, which is a selftest module but runs a testimage command. The var and test environment were set in runtime-perf.py and the actual test is done in a new testcase added to meta-selftest layer. ===== [YOCTO #10872] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16selftest: Disable SSTATE_MIRRORS for sstate signing testRichard Purdie1
Building ed into an sstate mirror, then leaving it enabled for oe-selftest -r signing.Signing.test_signing_sstate_archive results in: NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Started WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Failed to fetch URL file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig, attempting MIRRORS if available ERROR: ed-1.14.1-r0 do_populate_lic_setscene: Fetcher failure: Unable to find file file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig anywhere. The paths that were searched were: /media/build1/poky/build/test-sstate /media/build1/poky/build/test-sstate WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Cannot verify signature on sstate package /media/build1/poky/build/test-sstate/29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Succeeded so we need to disable SSTATE_MIRRORS for this test. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14wic: selftest: account for occasional newline in debugfs file namesMaciej Borzecki1
Debugfs output may contain a newline in file names in 'ls -p' output. Make sure that output is correctly split into lines by matching '/\n' and newlines are removed from file names. Fixes the following error appearing in AB tests: Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 388, in test_exclude_path files = [line.split('/')[5] for line in res.output.split('\n')] File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 388, in <listcomp> files = [line.split('/')[5] for line in res.output.split('\n')] IndexError: list index out of range Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13oe-selftest: no need to parse gpg output to determine the name of the ↵Alexander Kanavin1
signing key The parsing fails on my machine, and we use a key with a hardcoded name, and so can provide it directly. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13gpg_sign.py: fix signing of rpm files using gpgAlexander Kanavin1
This means a) calling rpmkeys and rpmsign instead of rpm b) instructing gpg to run non-interactively; otherwise on my machine it pops up windows requesting a key passphrase Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13selftest: update output checks in incremental image test to match what dnf ↵Alexander Kanavin1
prints Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13selftest: do not perform a full build in test_continueAlexander Kanavin1
This was fetching and building the toolchain and everything else against empty download dir and sstate cache, and so was enormously slow. The test does not need that, it only checks that one fetch task fails and another succeeds when using bitbake's -k option. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13testimage.bbclass: fix runtime test for rpm, port smart tests to dnfAlexander Kanavin5
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-10lib/oeqa/selftest/bbtests: update test_non_gplv3 to use selftest-edJoshua Lock1
GPLv2 recipes have been moved to a new layer (meta-gplv2), instead of readline perform this test on the selftest-ed recipe in meta-selftest which has gplv2 and gplv3 variants. Tested with oe-selftest -r bbtests.BitbakeTests.test_non_gplv3 Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-10lib/oeqa/selftest/oescripts: make test_cleanup_workdir use selftest-edJoshua Lock1
Use a recipe which is bundled in the meta-selftest layer for this test, rather than relying on OE-Core remaining static (or updating the tests when OE-Core changes recipes). Tested with oe-selftest -r oescripts.TestScripts.test_cleanup_workdir Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08selftest: remove extra backslashes from debug outputEd Bartosh1
Remove unneeded backslashes from the format strings that caused debug output to look confusing: 2017-03-06 16:52:42,428 - selftest.base - DEBUG - Removing from: ... \IMAGE_FSTYPES = "wic" WKS_FILE = "mkefidisk.wks" Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08qemurunner.py: ignore decode errorsEd Bartosh1
qemu output can contain control characters. This cause qemurunner API to crash when decoding the output to utf-8: Traceback (most recent call last): File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner self.run() File "meta/lib/oeqa/utils/qemurunner.py", line 472, in run threading.Thread.run(self) File "/usr/lib64/python3.4/threading.py", line 859, in run self._target(*self._args, **self._kwargs) File "meta/lib/oeqa/utils/qemurunner.py", line 465, in threadtarget self.eventloop() File "meta/lib/oeqa/utils/qemurunner.py", line 526, in eventloop self.logfunc(data) File "meta/lib/oeqa/utils/qemurunner.py", line 77, in log msg = msg.decode("utf-8") UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: unexpected end of data Added errors='ignore' to decode call to fix this. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08selftest: test wic efi image in qemuEd Bartosh1
Added test_qemu_efi test case to wic test suite. It uses ovmf qemu extention to test mkefidisk image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08targetcontrol: add image_fstype argument to commands.runqemuEd Bartosh2
qemu runner picks up first fsimage type from the hard-coded list of supported types. This makes it impossible to test particular image type unless it's not ext4(first type in the hardcoded list of types). Added image_fstypes argument to commands.runqemu and QemuTarget __init__ to specify type of the image to run qemu with. This will be used to pass wic image type to test efi wic images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08qemurunner: add runqemuparams argument to commands.runqemuEd Bartosh4
Added possibility to pass additional runqemu parameters down the stack of APIs: commands.runqemu -> QemuTarget.start -> QemuRunner.start This will be used to pass ovmf parameter in testing of efi wic images under qemu. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-03oeqa/sdkext/devtool: Ensure dependencies for ↵Richard Purdie1
test_extend_autotools_recipe_creation are present test_extend_autotools_recipe_creation needs libxml2 so ensure this is installed/present as it may not be in the minimal eSDK case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/sdkext: don't skip tests if there isn't a toolchainRoss Burton1
Skipping the tests if a toolchain wasn't installed out of the box (for example, a minimal eSDK) doesn't make sense as the first thing the tests should do is install a toolchain. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/sdkext/devtool: use finally instead of repeating cleanupRoss Burton1
Use the finally: block to always to cleanup. Now that the test harness in testsdk.bbclass has monkey-patched CalledProcessException to display the output we don't need to do that in the test case. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa: add output to subprocess exceptionsRoss Burton2
Out of the box subprocess.CalledProcessError.__str__() just displays the command and exit code, which isn't very useful for debugging. Add a function to oeqa.utils.subprocesstweak to monkey-patch __str__() so that it can also display the value of stdout and stderr. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/esdk/devtool: clean setUpClass/tearDownClassRoss Burton1
These methods are class not instance methods, so the argument should be cls not self. Also don't put variables into cls that we don't need there. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/selftest: Drop http sstate sharingRichard Purdie1
Using httpServer from python for sharing sstate is known to be buggy, it can't cope with the number/type of requests coming from bitbake and quietly fails to share files. This causes intermittent build failures which are hard to debug. We can use a file:// url for the sstate mirror instead, removing the need for the http server. The sdk-update test is simply dropped since the SDK is never published to this location and hence it would never have any update. Its equiavalent to pointing at an empty web server. There is a better eSDK update test in testsdk so rather than improve this one, lets drop it and concentrate on the one there. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/sdkext/context: Work around broken dependency checks to get sdk tests ↵Richard Purdie1
running This is admitted a bit of a hack but it does allow a number of significant sdk tests to run successfully and hence improves testing of eSDK which is good. I'm therefore proposing we do this until we come up with a better solution since the current lack of testing is worrying and would have caught other issues had it been present. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqs/sdk*/case: Use universal_newlines for subprocess callsRichard Purdie2
This removes the need for some of the ugly decode calls with hardcoded locales. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/sdkext: Ensure we run a deterministic set of testsRichard Purdie1
The directory list of sdk tests to run can vary so this code effectively selects a random set of SDK tests to run in the eSDK. We want to attemp all the SDK tests so remove the element selection. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/sdkext/devtool: rename recipe name to match upstreamRoss Burton1
Call the generated recipe librdfa instead of bb-example to make it clearer what is happening. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oelib/buildhistory.py: Add unittest for buildhistory_analysisHumberto Ibarra1
The buildhistory_analysis module (in which buildhistory-diff is based) was lacking unittest for its functions. Created selftest module for this and a few testcases to cover basic cases. [YOCTO #10727] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oeqa/core/loader.py: Avoid importing tests with built-ins nameMariano Lopez1
If importing a test with the same name as a built-in module, it will silently import the built-in and check for tests in built-in module. This happened with syslog module in debian based machines, so add a raise to avoid this behavior. [YOCTO #10978] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oeqa/core/loader.py: Give meaningful error when failed to load classesMariano Lopez1
With this we get the class that is actually having the problem, not just a TypeError with an unknown class causing the error. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01selftest: Optimize get_bb_var useMariano Lopez17
get_bb_var calls bitbake every time it is used and every call would take about 7 seconds. There are tests that calls get_bb_var several times when they can use get_bb_vars. Also there are tests that calls it to fetch the same variable over and over again. This will optimize the use of get_bb_var and get_bb_vars for a little speed up in the tests. [YOCTO #11037] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01selftest: Avoid sstate corruption by calling cleansstateMariano Lopez8
Currently selftest doesn't use sstates because some tests clean sstate cache; using sstates would give a performance boost instead of building everything from scratch. With this sstates are not corrupted using different methods depending on tests: devtool: These tests needed to delete the cache so SSTATE_DIR as SSTATE_MIRRORS and set a temporal SSTATE_DIR. sstatetests: This module already used a temporal SSTATE_DIR, so just set up the SSTATE_MIRRORS. Rest: Removed cleansstate, some of them required to force a certain task, others were just removed or changed for another task. [YOCTO #10929] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23selftest/eSDK.py: Cleanup when there is an error in setUpClassMariano Lopez1
Lately autobuilders are experiencing hangs with selftest, it seems it is cause if an error happens in setUpClass method of oeSDKExtSelfTest class because HTTP server keeps running in background. This patch will ensure tearDownClass will be run if there is an error in setUpClass. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23oeqa/runtime/context.py: Fix use of getTarget() with testexportMariano Lopez2
The idea on getTarget is to use kwargs to send custom variables to different targets, instead of this, a new variable was added (just used for custom targets) and this broke testexport. So in order to fix it, just add the custom variable to kwargs. This fixes the use of getTarget() in testexport class that was introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23oeqa: fix typoHumberto Ibarra3
The word 'uninstall' was spelled incorrectly in various parts of oeqa files. (From OE-Core rev: af365e025030436f83b233fa51ecc9c58c58ce3c) Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-19selftest/containerimage.py: Add container IMAGE_FSTYPES testRandy Witt1
This test checks to make sure only the files expected exist in a container image. Currently only ROOTFS_BOOTSTRAP_INSTALL, gets added to all images without the user specifying it. But this test should help if a developer in the future ever silently adds more than just ROOTFS_BOOTSTRAP_INSTALL, and that the developer can make sure it also gets removed from a container image. [YOCTO #9502] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-19oeqa/selftest/bbtests: Update after changes to dependency graph file generationRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15selftest/wic: Add tests for --exclude-dir option.Kristian Amlie1
Based partially on an earlier patch by Maciej Borzecki. Note that because tools now reside in recipe specific sysroots, we need to import the path from bitbake and apply it during the test. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15selftest: remove result file log awkward checkingBenjamin Esquivel1
because of the way that the test cases are looked in the results, the file logger is incompatible with extended implementations of the unittest runner. as the xml runner extends the unittest runner, it shares the id() method which returns the full name of the test, not only the test function name. With that, a single check of the full name reviews at the same time the class name as well as the function name. [YOCTO#11012] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oeqa.utils.git: support bare reposMarkus Lehtonen1
[YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oeqa.utils.git: add git_dir propertyMarkus Lehtonen1
[YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oeqa/buildperf: don't archive stdout/stderr of commandsMarkus Lehtonen2
Stop capturing output of the shell commands into <test>/commands.log. Redirecting output into a file prevented the unittest framework from capturing it, causing useless errors (with empty output) like: oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']' returned non-zero exit status 1 with output: In general, the console output of commands is only interesting when something fails. Also, dropping the commands.log file is a huge saving in disk space, and thus, repository size when results are archived in Git. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15testimage: Implement custom target loadingJuro Bystricky1
This patch implements custom target loading for testimage, currently missing due to major changes to the test framework. Custom targets can be defined in various meta-layers, so we need an extra path information in order to find them. Any other additional info is retrieved as usual via the variables TEST_TARGET and TEST_SUITES Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15oeqa/loader.py: Avoid error in TypeError exceptionJuro Bystricky1
Without this an exception would occur Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15qemu.py: add 'elf' as supported fstypeJuro Bystricky1
Add 'elf' as also supported by OEQemuTarget. Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-05 18:35:08 +0000