summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2017-01-09wic: Remove gummiboot testAlejandro Hernandez1
Weve now migrated to systemd-boot, the gummiboot test on wic is no longer necessary Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09gummiboot: Remove/change gummiboot references with systemd-bootAlejandro Hernandez1
After systemd-boot was introduced, its been tested for a while with no major issues being found until now, this patch completely replaces all gummiboot instances with systemd-boot ones, taking the next step into cleaning up systemd-boot/gummiboot. [YOCTO #10332] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie1
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22selftest/wic: extending test coverage for WIC script optionsJair Gonzalez1
The previous WIC script selftest didn't cover all of its command line options. Some option variants were included in existing test cases and the following tests were added to complete covering them: 1552 Test wic --version 1553 Test wic help create 1554 Test wic help list 1555 Test wic list images 1556 Test wic list source-plugins 1557 Test wic listed images help 1558 Test debug 1563 Test skip build check 1564 Test build rootfs 1559 Test image vars directory selection 1562 Test alternate output directory Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22selftest/wic: reorganizing test methods by functionalityJair Gonzalez1
Part of the test methods were rearranged to group them by functionality and identify more easily opportunities to extend coverage. Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22selftest/wic: code cleanupJair Gonzalez1
The following changes were added to improve the code readability: - Removed redundant backlashes between brackets - Aligned continuation lines according to PEP8 style - Refactored command execution in the next methods for enhanced legibility: - test_build_image_name(self) - test_gpt_image(self) - test_qemux86_directdisk(self) - test_rootfs_indirect_recipes(self) - test_iso_image(self) - test_mkgummidisk(self) - test_mkefidisk(self) - test_directdisk_bootloader_config(self) - test_qemu(self) - test_bmap(self) - test_systemd_bootdisk(self) - test_sdimage_bootpart(self) Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22selftest/wic: adding Testopia ID numbers to test cases missing itJair Gonzalez1
The following test cases were assigned an ID number on Testopia: 1496 Test generation of .bmap file 1560 Test creation of systemd-bootdisk image 1561 Test creation of sdimage-bootpart image Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22oeqa/utils/commands.py: allow use of binaries from native sysrootMaciej Borzecki1
Tests may need to run a native tool that is not available on the host filesystem, but can be built using one of the *-native recipes. In such case, the tool will be available in native sysroot, and running in from that location will require adjustments to PATH. runCmd() can now take a path to native sysroot as one of its arguments and setup PATH accordingly. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-20Revert "selftest/wic: extending test coverage for WIC script options"Richard Purdie1
This reverts commit 68cb3180c1b0dcee50812b21f98850d188d8621b as this wasn't ready for merge and there are new better versions.
2016-12-19oe-selftest: import git module only when neededEd Bartosh1
git module is not included into standard Python library and therefore causes import errors on the systems where PythonGit is not installed. As git module only used in the code implementing --repository functionality it's better to import git only in the scope that requires it. [YOCTO #10821] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-19selftest/wic: extending test coverage for WIC script optionsJair Gonzalez1
The previous WIC script selftest didn't cover all of its command line options. The following test cases were added to complete covering them: 1552 Test wic --version 1553 Test wic help create 1554 Test wic help list 1555 Test wic list images 1556 Test wic list source-plugins 1557 Test wic listed images help 1558 Test wic debug, skip-build-check and build_rootfs 1559 Test image vars directory selection 1562 Test alternate output directory In addition, the following test cases were assigned an ID number on Testopia: 1560 Test creation of systemd-bootdisk image 1561 Test creation of sdimage-bootpart image Finally, part of the test methods were rearranged to group them by functionality, and some cleanup was made to improve the code's compliance with PEP8 style guide. Fixes [YOCTO 10594] (From OE-Core rev: 3d2ac67765020885a0996ebdd97a576ba37dbec0) Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16meta: remove True option to getVar callsJoshua Lock20
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16postinst: Add a test case to verify postinst scripts behaviorJose Perez Carranza1
Add test case that verify behavior of postinst scripts at roofts time and when is delayed to the first boot directly on the target. Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: Test needed to verify postinst orderFrancisco Pedraza1
It verifies the following: 1. Compile a minimal image. 2. The compiled image will add the layer with the recipe postinst, previously created at: "meta-selftest/recipes-test" 3. Run QEMU. 4. Validate the task execution order. [YOCTO #5319] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16oeqa: move lib/oe tests to oe-selftestRoss Burton6
These tests don't get ran often (as demonstrated by the fact that some were not ported to Python 3), so move them to oeqa/selftest so they get executed frequently and can be extended easily. [ YOCTO #7376 ] Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: buildoptions: skip read-only-image test depending on distroLeonardo Sandoval1
Poky-tiny cannot build core-image-sato, so skip test (read-only-image) in this case. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: bblayers: remove linux kernel checks for show-recipes checkLeonardo Sandoval1
Preferred kernel recipes depends on the distro, so remove the kernel checks to avoid failures on non-poky distros and make the test distro agnostic. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: bbtests: use minimal image so all distros can execute itLeonardo Sandoval1
poky-tiny distro cannot build full-cmdline image, so use an image (core-image-minimal) that can be built in all distros. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: base: new object member to store the DISTRO valueLeonardo Sandoval2
Instead of quering it multiple times, query once and use it on test method skip checks. Also, rename current distro sstate object member to a more meaninful name. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oeqa/utils/commands.py: Fix get_bb_vars() when called without argumentsMariano Lopez1
Commit 9d55e9d489cd78be592fb9b4d6484f9060c62fdd broke calling get_bb_vars() when called without arguments. This fix this issue. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14oe-selftest: add basic tinfoil testsPaul Eggleton1
Add some tests to verify that the new tinfoil API is operating correctly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oe-selftest: devtool: improve test_devtool_modify slightlyPaul Eggleton1
* Check that man .in file actually gets modified, since sed -i doesn't fail if it it doesn't * Use a variable for man file path Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oe-selftest: use tinfoil.parse_recipe()Paul Eggleton1
Use tinfoil.parse_recipe() in order to allow oe-selftest to be used in memres mode. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oe-selftest: make tinfoil quiet when using to start QEMUPaul Eggleton1
We don't need to see the parsing/cache loading message in the oe-selftest output, so use the newly added quiet option to disable it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oe-pkgdata-util: Make read-value handle override variablesOla x Nilsson1
Some variables in pkgdata files have a package-name override. When the bare variable can not be found, try with the override-variant. PKGSIZE is one such variable, and already had special code to handle this. Test included. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13parselogs.py: Don't clog QA with Joule errorsCalifornia Sullivan1
The Joule is very new hardware and there is ongoing kernel and firmware work to fix these issues, which will be available in future kernel and firmware releases. In the meantime, don't clog QA reports. [YOCTO #10611] Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13recipetool: selftest: Add test for recipetool plugin loadingOla x Nilsson1
Test that recipetool plugins are loaded in a well defined order. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13devtool: selftest: add test for devtool plugin loadingOla x Nilsson1
Test that devtool plugins are loaded in a well defined order. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oeqa/utils/metadata.py: Add metadata libraryMariano Lopez1
Adds functions to get metadata from the host running the tests. [YOCTO #9954] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oeqa/utils/commands.py: Make a copy of variables in get_bb_varsMariano Lopez1
The function get_bb_vars will remove items for the list passed as the function argument, this will leave the caller with an empty list and the function never says it will consume the items. This hasn't been found before because only get_bb_var uses this function. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-12targetloader.py: drop test for ClassTypeJuro Bystricky1
ClassType was removed from python3. The code testing for ClassType kept throwing AttributeError exceptions: module 'types' has no attribute 'ClassType' The exceptions prevented loading of any dynamically resolved target controllers. Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07selftest: wic: fix test_qemuEd Bartosh1
Setting WKS_FILE variable in qemux86-64 made wic test to use wrong wks file to produce an image and resulted in test_qemu failure. Used conditional assignment in qemux86-64 and explicitly set WKS_FILE in wic testing suite to make the suite to use wic-image-minimal.wsk. This should fix test_qemu failure. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-07parselogs: Whitelist GPT warnings as the device is fully functionalJair Gonzalez1
The warning occurs when the GPT image is not the same size than the media into which it's being flashed, causing the backup GPT table not being at the end of the disk. However, this is expected as the image is created before having the information about the destination media. The error is harmless, so it will be whitelisted. Fixes [YOCTO 10481]. Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-07parselogs.py: Whitelist iwlwifi firmware load error messagesCalifornia Sullivan1
The iwlwifi module of any given kernel has a minimum and maximum supported firmware version. The kernel begins by attempting to load the maximum version, and decrements until it is successful. The 4.8 kernel's maximum supported firmware version is 24, but thus far only 22 has been released, meaning we get errors for 24 and 23. Filter out iwlwifi firmware load error messages, as they are not necessarily indicative of real problems. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-30runtime: Update test cases numbers for runtime testsJose Perez Carranza4
Update test case numbers on runtime tests to do match with templates defined on Testopia for 2.3 release Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-30oe-selftest: toaster Remove redundant Toaster testMichael Wood1
This test has been ported to be run as part of Toaster's own tests. Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23parselogs: Whitelist NUC6 firmware load error message on genericx86-64California Sullivan1
This was already whitelisted, but the 4.8 kernel changed the error message, causing it to get caught by parselogs again. Fixes [YOCTO #10494]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23devtool: modify: support recipes with only local files as sourcePaul Eggleton1
The hello-mod recipe is unusual in that it has only local files in SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify on it, devtool puts all of those files in an "oe-local-files" subdirectory of the source tree, which is not ${S} (or ${B}) any more and thus building the recipe afterwards fails. It's a bit of a hack, but symlink the files in oe-local-files into the source tree (and commit the symlinks with an ignored commit so that the repo is clean) to work around the problem. We only do this at time of extraction, so any files added to or removed from oe-local-files after that won't be handled, but I think there's a limit to how far we should go to support these kinds of recipes - ultimately they are anomalies. I initially tried a hacky workaround where I set effectively set B = "${WORKDIR}" and that allowed it to build, but other things such as the LIC_FILES_CHKSUM checks still broke because they expected to find files in ${S}. Another hack where I set the sourcetree to point to the oe-local-files subdirectory works for hello-mod but not for makedevs since whilst that is similar, unlike hello-mod it does in fact have files in the source tree (since it has a patch that adds COPYING) and thus the same issue occurred. Also tweak one of the tests that tries devtool modify / update-recipe on the makedevs recipe to try building it since that would have caught this issue. Fixes [YOCTO #10616]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23oe-selftest: devtool: test that updating a file with subdir= worksPaul Eggleton1
If you have a file:// entry in SRC_URI with a subdir= parameter that makes it extract into the source tree, then when you update that file in oe-local-files and run devtool update-recipe then you want the original file to be updated. This was made to work by OE-Core commit 9069fef5dad5a873c8a8f720f7bcbc7625556309 together with 31f1bbad248c36a8c86dde4ff57ce42efc664082, however until now there was no oe-selftest test to verify it. Note that in order to succeed this test also requires the fix "lib/oe/recipeutils: ignore archives by default in get_recipe_local_files()" since the test recipe uses a local tarball. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23devtool: update-recipe: fix handling of compressed local patchesPaul Eggleton1
It is possible to use gzip or bzip2 to compress patches and still refer to them in compressed form in the SRC_URI value within a recipe. If you run "devtool modify" on such a recipe, make changes to the commit for the patch and then run devtool update-recipe, we need to correctly associate the commit back to the compressed patch file and re-compress the patch, neither of which we were doing previously. Additionally, add an oe-selftest test to ensure this doesn't regress in future. Fixes [YOCTO #8278]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23oe-selftest: devtool: test update-recipe with only local filesPaul Eggleton1
Add a test to ensure devtool update-recipe works properly on recipes that contain only local files (since the other tests we have didn't test that). Relates to [YOCTO #10563]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-07Revert "oeqa/selftest/kernel.py: Add new file destined for kernel related tests"Ross Burton1
breaking on selftest NOTE: recipe core-image-minimal-1.0-r0: task do_rootfs: Started ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:license_create_manifest(d) 0003: File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 48, function: license_create_manifest 0044: pkg_dic = {} 0045: for pkg in sorted(image_list_installed_packages(d)): 0046: pkg_info = os.path.join(d.getVar('PKGDATA_DIR', True), 0047: 'runtime-reverse', pkg) *** 0048: pkg_name = os.path.basename(os.readlink(pkg_info)) 0049: 0050: pkg_dic[pkg_name] = oe.packagedata.read_pkgdatafile(pkg_info) 0051: if not "LICENSE" in pkg_dic[pkg_name].keys(): 0052: pkg_lic_name = "LICENSE_" + pkg_name Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/sysroots/qemux86-64/pkgdata/runtime-reverse/kernel-4.8.3-yocto-standard' This reverts commit c3d2df883a9d6d5036277114339673656d89a728.
2016-11-07devtool: add "rename" subcommandPaul Eggleton1
When you run devtool add on a source tree we attempt to figure out the correct name and version for the recipe. However, despite our best efforts, sometimes the name and/or version we come up with isn't correct, and the only way to remedy that up until now was to reset the recipe, delete the source tree and start again, specifying the name this time. To avoid this slightly painful procedure, add a "rename" subcommand that lets you rename the recipe and/or change the version. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-07oe-selftest: devtool: fix error message in _test_recipe_contents()Paul Eggleton1
If a variable is being set in the recipe when we've explicitly passed None as the value to _test_recipe_contents() indicating that it shouldn't be set at all, then we should be printing out the variable name in the assertion message but it seems like I forgot to do a substitution. Also include the value for informational purposes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-07recipetool: create: separate LICENSE items with & by defaultPaul Eggleton1
recipetool sets the LICENSE value based on licenses detected from the source tree. If there are multiple licenses then they were being separated by spaces, but this isn't actually legal formatting and if you're using "devtool add" you get a warning printed when devtool parses the recipe internally. Earlier I had made a conscious decision to do it this way since it's up to the user to figure out whether the multiple licenses should all apply (in which case they'd be separated with &) or if there is a choice of license (in which case | is the correct separator). However, I've come to the conclusion that we can just default to & and then the ugly warning goes away, and it's the safest alternative of the two (and most likely to be correct, since it's more common to have a codebase which is made up of code with different licenses, i.e. all of them apply to the combined work). I've tweaked the comment that we add to the recipe to explicitly state that we've used & and that the user needs to change that if that's not accurate. Fixes [YOCTO #10413]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-01oeqa/selftest/kernel.py: Add new file destined for kernel related testsCostin Constantin1
[YP#7202]: Test for linux-dummy The new kernel.py file is intended for kernel related test cases. The test for linux-dummy will ensure it is in good shape and can be used as a kernel replacement at build time. To do this, the test will first clean sstate for linux-dummy target, ensuring no file is present in the stamps directory. After, core-image-minimal is built, ensuring linux-dummy can be used as a kernel substitute. Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-15Remove RM_OLD_IMAGE, it's no longer usefulJoshua Lock1
Since the move to put image deployment under sstate control in d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically removed before a new image is deployed (the default behaviour of the sstate logic). RM_OLD_IMAGE is therefore no longer required to provide this behaviour, remove the variable and its users. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-07oeqa/sshcontrol: Handle interrupted system call errorRichard Purdie1
Deal with an interrupted system call gracefully: | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/sshcontrol.py", line 55, in _run | if select.select([self.process.stdout], [], [], 5)[0] != []: | InterruptedError: [Errno 4] Interrupted system call Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07wic: selftest: add test for sdimage-bootpartEd Bartosh1
Test creation of sdimage-bootpart image Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07wic: selftest: add test for systemd-bootdiskEd Bartosh1
Test creation of systemd-bootdisk image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>