summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)AuthorFiles
2015-11-16oeqa/selftest/layerappend: fix test if build directory is not inside COREBASEPaul Eggleton1
Fix test_layer_appends to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/devtool: fix test if build directory is not inside COREBASEPaul Eggleton1
Fix test_devtool_update_recipe_git to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24oeqa/selftest: Added testcase decorators.Daniel Istrate4
Added testcase decorators for testopia integration. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: improve config writing and cleanupRoss Burton3
The selftest.inc configuration file is deleted in both tearDown() and setUp() so there's no need to use addCleanup() to remove statements from it. Use write_config instead of append_config if the intention is to start from an empty config file, for clarity. Finally remove some misleading comments that claim that append_config() writes to local.conf when it doesn't. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: remove numbers from test namesRoss Burton1
There isn't any required ordering of tests, and the numbers meant that the tests didn't appear in oe-selftest --list-classes. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: clean up selftest.inc in teardownRoss Burton1
Test cases may want to do call bitbake in setUpClass() but at that point the previous selftest.inc is still present which could change the build configuration and result in any built artifacts being removed in the next bitbake invocation as part of the sysroot clean up. Resolve this by cleaning selftest.inc in the tearDown, the clean in setUp should be considered a safety net. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: fix cleaningRoss Burton1
In some situations the native tools built in setUpClass() are wiped from the sysroot by the time the tests are executed, likely due to the cleanup performed in the base setUp() method. Avoid this by doing all of the preparatory building in setUpLocal. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: corrected testcase decorator for test18_iso_imageDaniel Istrate1
Changed testcase decorator for TC test18_iso_image from 1264 to 1346. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: verify that devtool can use plugins in other layersRoss Burton1
In selftest we want to exercise devtool's ability to use plugins from other layers, so add a basic command to reverse a string to meta-selftest and a test in oe-selftest to call it. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/buildoptions: Use the correct script for cleaning the workdirLeonardo Sandoval1
The script 'cleanupworkdir' does not exit, so changing it to 'cleanup-workdir'. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: Updated bitbake TCsDaniel Istrate1
- Added new TC test_force_task_1 (1354); check that do_package_write_rpm() re-executes upon changes in package image. - Updated TC test_force_task_2 (163); changed test recipe to zlib and added do_package() to the task execution list. - Removed unnecessary imports. Fix for bug [YOCTO #5875]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: clean up local DL_DIR/SSTATE_DIR safelyRoss Burton1
Now that the selftests clean selftest.inc in the tearDown using bitbake in a function passed to addCleanup (which happens after teardown) will use the default configuration and not the customised one. As the intention was to clean away the temporary DL_DIR and SSTATE_DIR, simple use track_for_cleanup to delete the entire directory instead. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/manifest.py: Test support for manifestsBenjamin Esquivel1
adding support for tests to verify that manifest contents contain entries that exist in an specified pkgdata dir tests added: - manifest.VerifyManifest.test_image_manifest_entries - manifest.VerifyManifest.test_SDK_manifest_entries test support written for future tests: -adding a setUpClass that supports other manifest tests -a get dir from bb var function that verifies if the dir exists -a ManifestEntry class defined with missing items list -check for the paths and fail gracefully if not there -debug prints for failure analysis [YOCTO#8028] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton1
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03oe-selftest: wic: fix LocalSetupEd Bartosh1
Appended required features to build config every time LocalSetup is run to ensure that core-image-minimal is not rebuilt due to configuration changes. This should fix test22_mkgummidisk and test23_mkefidisk testcases as they depend on specific features set in LocalSetup. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/wic: Use SetupLocal instead of SetupRichard Purdie1
If we replace the code Setup method, we don't get the cleanup it performs and this just resulted in failures on the autobuilder due to a stale config fragment. Setup will call SetupLocal so this should be a safe and easy fix to resolve the auobuilder failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add test that MACHINE doesn't change target sigsRichard Purdie1
When we change between two machines with the same tune, we shouldn't see rebuilds of binaries. This adds a test for this using the qemux86copy machine. We also extend the test to cover multilibs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add check for same sigs for SDKMACHINERichard Purdie1
Extend one of the sstate tests to also ensure that changing SDKMACHINE doesn't change the target task checksums. Also fix a typo and improve debugging by turning the diff filtering off in all cases (if the test fails, we want to full list). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/selftest/wic: Added testcase decorator to all testcases + fixed minor ↵Daniel Istrate1
typos. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: better support for local source filesMarkus Lehtonen1
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking repo statusMarkus Lehtonen1
New method for checking the status of the working tree of a repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking srctree repoMarkus Lehtonen1
Removes some code duplication. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add setup() methodMarkus Lehtonen1
In order to remove some code duplication. The method currently only does checking of workspace dir. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-23oeqa/selftest/archiver: Test that archiver filters on recipe nameDaniel Istrate1
[YOCTO #6929] this test validates the feature introduced in bug 6929 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton1
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22oeqa/selftest: buildoptions.py Removed unused importsDaniel Istrate1
Removed unused imports: unittest, logging, pexpect Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-16oeqa/sstatetests: Add test for nativesdk stamp invariance with MACHINERichard Purdie1
nativesdk-glbic should not rebuild when you change MACHINE but it was. We've fixed that, now add tests to ensure this doesn't happen again. Rather than add yet another stamps test, extend one of the existing ones to cover this instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09oeqa/sstatettests: Improve output of assertRichard Purdie1
Currently if this fails you see: "AssertionError: 1 != 0" which is useless. Add the output of the failed command so we can stand some chance of debugging what went wrong. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: update-recipe: better 'auto' modeMarkus Lehtonen1
Enhance the logic behind the 'auto' mode a bit by only updating the SRCREV if the changes are already found upstream. The logic is simple: update SRCREV only if the current local HEAD commit is found in the remote branch (i.e. 'origin/<branch_name>'). Otherwise resort to patching. This affects a couple of the oe-selftest tests so update those as well. [YOCTO #7907] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: add upgrade featureLeonardo Sandoval1
Upgrades a recipe to a particular version and downloads the source code into a folder. User can avoid patching the source code. These are the general steps of the upgrade function: - Extract current recipe source code into srctree and create a branch - Extract upgrade recipe source code into srctree and rebase with previous branch. In case the rebase is not correctly applied, source code will not be deleted, so user correct the patches - Creates the new recipe under the workspace [YOCTO #7642] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02wic: add test cases for 3 imagesEd Bartosh1
Added test cases to build qemux86_directdisk, mkgummidisk and mkefdisk images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02oe-selftest: wic: configure a buildEd Bartosh1
Added MACHINE_FEATURES and IMAGE_FSTYPES variables to the build config to ensure that bootimg and efi artifacts are built. This is needed to build canned wic images that are using those artifacts. Introduced class variable Wic.image_is_ready to avoid building image more than once. It would be better to build image in setUpClass, which is called only once, but add_config method can't be called in class method, so we have to use this trick with class variable in setUp method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-31oe-selftest: test devtool build-image pluginEd Bartosh1
Added test case to test functionaly of build-image plugin: Add two packages to workspace and run 'devtool build-image core-image-minimal'. Checked if command is successful. Checked if expected package is added to generated core-image-minimal.bbappend file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2015-08-31oe-selftest: wic: fix test19_image_env test caseEd Bartosh1
This test case fails on qemu machines as not all expected bitbake variables are present in .env file. Fixed by filtering out optional variables. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-31oeqa: Add allarch sstate sig testRichard Purdie1
"allarch" targets should be MACHINE invariant. This means their sstate signature should not change regardless of which MACHINE is built. Errors keep creeping in around this area so automating this as part of self test seems the best way to maintain this. The "do_build" stamps are known to differ and are harmless so those are excluded from the test. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30oe-selftest: test building wic image by bitbakeEd Bartosh1
Added test case to verify building of wic-image-minimal recipe and produced artifacts: manifest and bzipped partitioned image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30oe-selftest: test generation of <image>.envEd Bartosh1
Added test case to check if <image>.env file is generated and contains bitbake variables used in wic code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30oeqa: Optimise sstate testsRichard Purdie1
We can use "none" here instead of printdiff for some small performance gains. This also means we can remove the ignore exit code hack which leads to safer code. Also drop an unused variable. (From OE-Core rev: e2b8d6fa45b1ae530be2ffabf48ea61d4c467066) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-24oe-selftest: replace assertTrue(False, ...) with fail(...)Paul Eggleton2
I'd somehow missed the existence of fail() when I wrote these. It's preferable here so you don't get the somewhat useless "false is not true" message in the case of failure. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-24wic: Test creation of iso imageMihaly Varga1
Added new wic testcase, for testing the creation of the hybrid iso image with isoimage-isohybrid plugin. Signed-off-by: Mihaly Varga <mihaly.varga@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-19devtool: fix handling of BBCLASSEXTENDed recipesPaul Eggleton1
If a recipe is BBCLASSEXTENDed (e.g. to -native), its PN value and the name of the bbappend will be different; we were assuming them to be the same when reading in the workspace, leading to us seeing the base recipe name everywhere afterwards. Also add a test so we ensure this doesn't regress in future. Fixes [YOCTO #8157]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-11oeqa/selftest: remove tests for TCs that are no longer validMihail Stanciu1
Several toaster backend tests are no longer valid due to design changes. Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-31oeqa/sstatetests.py: add annotator to test_sstate_32_64_same_hash [YOCTO #6006]Costin Constantin1
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-31oeqa/sstatetests.py: add decorator to test_sstate_nativelsbstring_same_hashCostin Constantin1
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-31oe-selftest: devtool: fix teardown warning in test_devtool_update_recipe_appendPaul Eggleton1
We don't need to run bitbake -c cleansstate on this because it's being cleaned out as part of "devtool reset". Addresses [YOCTO #8031]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/devtool: use new runqemu functionPaul Eggleton1
Use the common code we now have to run QEMU instead of running it ourselves, avoiding reliance on the machine showing up at 192.168.7.2. This also makes a copy of the image rather than using -snapshot so if we need to inspect the image after a failure, we can. Fixes [YOCTO #7928]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: remove gummiboot testsPaul Eggleton1
These tests were _deleting_ meta-intel if it happened to appear under COREBASE, which could have been catastrophic if there was any work in progress in that directory. It turns out we don't even need meta-intel, but we do need a machine that's set up appropriately (e.g. genericx86-64). Tests that involve layers outside of OE-Core don't really belong in OE-Core, and genericx86-64 is in meta-yocto-bsp; however we will soon have the capability to have selftest tests in other layers, so remove this here so we can add a fixed version in meta-yocto-bsp after that happens. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: fix RPM4 testPaul Eggleton1
* Use our new runqemu function * Don't hard-code the RPM4 version * Double-check the native version is RPM4 * Check that an rpm 4.x package is in the image manifest (this isn't strictly necessary, but "belt-and-braces" and it serves as an example of how to do that) * Check that the database is working on the target * Ensure the image actually has openssh in it so we can connect to it Initial runqemu adaptation by Richard Purdie <richard.purdie@linuxfoundation.org> Part of the fix for [YOCTO #7994]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: fix SSH without password testsRichard Purdie1
* We need to set EXTRA_IMAGE_FEATURES outright or existing values will affect the test * For test case 1107 we need "empty-root-password" to match the behaviour described in the test case * For test case 1115 we shouldn't be able to connect as root with the features we are setting Test cases 1107 and 1115 have been updated in Testopia to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: Use QemuTarget codeRichard Purdie1
Create a runqemu function which uses the QemuTarget() code from oeqa.targetcontrol to setup the QEMU instance, with all of the added robustness that that gives us. To do this, a datastore is needed for the recipe in question (core-image-minimal) so we do the work needed to set this up. We then use this runqemu function within the imagefeatures tests instead of a hand-rolled implementation. We can then use SSHControl to run the SSH tests rather than rolling our own code to do that as an added bonus. Fixed and extended by Paul Eggleton <paul.eggleton@linux.intel.com>. Part of the fix for [YOCTO #7994]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>