summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2016-01-19recipetool: create: add basic support for extracting dependencies from cmakePaul Eggleton1
Add support for extracting dependencies from CMakeLists.txt. There's still a bunch of things missing that are outside the scope of OE-Core and we still lack a proper extension mechanism, but this is a good start. This also adds an oe-selftest test to exercise the new code a bit. Implements [YOCTO #7635]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19recipetool: create: improve extraction of pkg-config / lib depsPaul Eggleton1
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict and thus we were skipping some macros. * Add support for PKG_CHECK_EXISTS * Avoid duplicates in warning on missing pkg-config dependencies * Ignore dependency on musl (since this may come up if it's the selected C library) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19selftest/buildhistory: Improve test to remove sources of errorRichard Purdie1
When we add buildhistory, we expect do_rootfs to rerun, but depending on IMAGE_FSTYPES, the number of tasks which would execute after do_rootfs varies (e.g. live would add do_bootimg and we recently added do_image). Therefore limit the test to -c rootfs and then we're clear that only one task should re-run. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19image: Fix wic environment issuesRichard Purdie1
The wic environment function needs to run after the rootfs size is setup. We move this code to a specific task, and depend on that task from the wic images and other places its needed. This fixes: ====================================================================== FAIL: test_image_env (oeqa.selftest.wic.Wic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env self.assertTrue(var in content, "%s is not in .env file" % var) AssertionError: False is not true : ROOTFS_SIZE is not in .env file Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19insane: Start to clean up do_configure_qa codeRichard Purdie1
This cleans up the do_configure_qa code so that the exit status from package_qa_handle_error is handled correctly. It also converts package_qa_check_license to use our standard QA check configuration interface through package_qa_handle_error rather than a mix of return values, bb.fatal and bb.build.FuncFailed. Due to the merging of multiple lines into one message, we need to tweak a QA test to account for this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11oeqa/runtime/logrotate: fix hardcoded root directoryYi Zhao1
Use $HOME instead of /home/root in case user changes the default root directory by ROOT_HOME Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-11oeqa/runtime/smart: fix hardcoded root directoryYi Zhao1
Use $HOME instead of /home/root in case user changes the default root directory by ROOT_HOME Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07x11vnc: remove all references to moved packageIoan-Adrian Ratiu1
Together with the move to meta-oe, all references to x11vnc should be removed from oe-core. There are three of these: a distro alias, a packagegroup rdepends and a runtime test. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07oeqa/selftest/recipetool: update for libjpeg-turbo migrationRoss Burton1
The recipe creation test of libmatchbox now depends on libjpeg-turbo, not jpeg. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07scripts/oe-selftest: Allow to run tests on random/all MACHINEsDaniel Istrate1
Add an option for random MACHINE into oe-selftest: --machine [random/all] 1. random: will set a random MACHINE for each test 2. all: will run tests for all machines Custom machine sets only weak default values (??=) for MACHINE in machine.inc. This let test cases that require a specific MACHINE to be able to override it, using (?= or =). e.g.: oe-selftest --run-tests signing --machine random --> will run all tests switching MACHINE randomly for each test oe-selftest --run-tests signing --machine all --> for each machine will run all tests oe-selftest --run-all-tests --machine random Also update oeqa/selftest/base.py to accomodate this feature. Fix for [YOCTO #5880]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: Added testcase decorators for 2 testsDaniel Istrate2
1391 for test_devtool_add_fetch_simple from devtool 1392 for test_recipetool_create_simple from recipetool Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: moved tc test_buildhistory_does_not_change_signaturesDaniel Istrate2
Moved test_buildhistory_does_not_change_signatures from buildhistory/BuildhistoryBase to buildoptions/BuildhistoryTests. The test being in the base class was causing it to run multiple times. Fix for [YOCTO #8867] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-28oeqa/selftest: support sets in devtool comparisonsRoss Burton2
The devtool and recipetool tests do literal string comparisons, but for some fields the ordering could be irrelevant and potentially non-deterministic. For example, the recipetool_create_simple test started failing with: AssertionError: 'GPLv2 Unknown' != 'Unknown GPLv2' : values for LICENSE do not match The ordering of the LICENSE field isn't relevant. So, if the expected value is a set(), split the string into a set too and compare those. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-27selftest: Added MACHINE = "qemux86" to tests that use runqemuDaniel Istrate2
It makes sense for tests that use runqemu to have MACHINE set as qemu. This also avoid issues when running oe-selftest with --arch random/all option. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22oe-selftest: devtool: add more explicit check for ls outputPaul Eggleton1
test_devtool_deploy_target is failing on the Yocto Project autobuilder apparently when it attempts to cut out some fields from the list. It doesn't fail here and I can't see what the problem lines are, so add a check for lines with too few fields so we can get a look at them next time it fails. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22oe-selftest: add tests for simple devtool add / recipetool create URL casePaul Eggleton2
Add an oe-selftest test case for the newly supported syntax with only the remote URL specified (auto-detecting name and version). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: basic extraction of name/version from filenamePaul Eggleton1
Often the filename (e.g. source tarball) contains the name and version of the software it contains. (This isn't intended to be exhaustive, just to catch the common case.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: support extracting name and version from build scriptsPaul Eggleton2
Some build systems (notably autotools) support declaring the name and version of the program being built; since we need those for the recipe we can attempt to extract them. It's a little fuzzy as they are often omitted or may not be appropriately formatted for our purposes, but it does work on a reasonable number of software packages to be useful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22selftest/buildhistory.py: Test buildhistory does not change sigsDaniel Istrate1
[YOCTO #5953] Add a test to ensure buildhistory does not change signatures. Also removed unused imports. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-16oeqa/selftest: Added testcase decorators for 2 testcasesDaniel Istrate2
bblayers: test_bitbakelayers_showrecipes 1384 wic: test_directdisk_bootloader_config 1385 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-12oeqa/systemd: get runtest target boot time and logBenjamin Esquivel1
oeqa.runtime.systemd.SystemdJournalTests.test_systemd_boot_time this test will query the target boot time from journactl and will print it to the output, if the time is obtained, the test passes it then compares the boot time against the default systemd's timeout TimeoutStartSec and if the boot time is greater than the it will print it too this test prints the startup time in the test log like: ... test_systemd_boot_time (oeqa.runtime.systemd.SystemdJournalTests) ... Startup finished in 6.922s (kernel) + 52.089s (userspace) = 59.011s. Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-12oeqa/systemd: journalctl helper functionBenjamin Esquivel1
a function to request for the journalctl output to the current target system with l_match_units support Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09selftest/wic.py: Add test for custom bootloader configMariano Lopez1
This change just add anoher test to the wic module. It will try to create a image with a custom bootloader configuration. This test will use the example image directdisk-bootloader-config to test the configfile option for the bootloaders. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01oeqa/sshcontrol: don't source profileRoss Burton1
Instead of sourcing /etc/profile to get $PATH including /usr/sbin, just assign to PATH in the ssh invocation. The remote /etc/profile may not actually be manipulating PATH as we expect, and there may be other commands which can interfere with the tests (such as resize emitting a series of control characters on connection). Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01oeqa/runtime/multilib: refactor ELF class extractionRoss Burton1
Instead of duplicating the same code over and over, split it out to a separate function. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: disable creating workspace for extract and search subcommandsPaul Eggleton1
For subcommands that don't actually involve the workspace, don't auto-create the workspace. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01oeqa/qemurunner: pass nographic to runqemu if DISPLAY isn't setRoss Burton1
Not everyone wants to run the tests with a qemu that has a graphical output, so allow display to be None and pass nographic to runqemu in that case. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/sstatetests: prettier output for allarch testRoss Burton1
Instead of creating two lists of full paths and comparing them which in failure produces a list of every stamp file (so all tasks, twice), reduce the filename down to a recipe/task->hash dictionary and compare those, meaning unittest shows the differences in the dictionaries. In the future get_files() should be generalised so all tests in this class can use it, and find a pair of hashes that don't match and run diffsigs on them. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/signing: Added new test for signing sstate.Daniel Istrate1
[YOCTO #8182] Optional signing sstate archives and signature verification [YOCTO #8559] Signing sstate archives with custom dir for gpg keys Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/signing: New test for Signing packages in the package feeds.Daniel Istrate1
[YOCTO # 8134] This test verifies features introduced in bug 8134. It requires as resources the files from meta-selftest/files/signing: For 'gpg --gen-key' the used input was: key: RSA key-size: 2048 key-valid: 0 realname: testuser email: testuser@email.com comment: nocomment passphrase: test123 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oe-selftest: add test for bitbake-layers show-recipesPaul Eggleton1
Add a test for bitbake-layers show-recipes including the recently added -i option. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/layerappend: fix test if build directory is not inside COREBASEPaul Eggleton1
Fix test_layer_appends to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/devtool: fix test if build directory is not inside COREBASEPaul Eggleton1
Fix test_devtool_update_recipe_git to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16qemurunner: Remove the timeout in run_serialMariano Lopez1
Sometmes when there is high load in the server the commands executed in the target take a lot of time to complete and this lead to incorrect dump files or empty files. This is caused because run_serial has a timeout of five seconds when running the commands in the target. This change removes the timeout and just keep reading the socket until it finds the prompt from the target or when the socket is not ready to be read. [YOCTO #8510] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16qemurunner: Enable timestamps on kernel boot-upLeonardo Sandoval1
Having the timestamps in the log is useful when debugging those cases when the booting time takes longer than expected. printk.times=1 enables stamping into every printk Kernel call. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/runtime/multilib: run the arch tests on connmand not connman-appletRoss Burton1
Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/runtime: remove dmesg testRoss Burton1
This has been obsoleted by parselogs. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-27oeqa/utils/decorators: fix missing keyword arguments on decoratorsPaul Eggleton1
We need to handle keyword arguments here or sending a keyword argument to a decorated function that accepts keyword arguments will trigger an error. (This showed up when testcase decorators were added to the recipetool.RecipetoolTests.test_recipetool_appendsrcfiles_basic_subdir test). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-24oeqa/selftest: Added testcase decorators.Daniel Istrate4
Added testcase decorators for testopia integration. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24qemurunner: Show the output of runqemu scriptMariano Lopez1
This change will show runqemu output in the logs when running the runtime tests. This is helpful for debugging testimage errors. [YOCTO #8550] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: improve config writing and cleanupRoss Burton3
The selftest.inc configuration file is deleted in both tearDown() and setUp() so there's no need to use addCleanup() to remove statements from it. Use write_config instead of append_config if the intention is to start from an empty config file, for clarity. Finally remove some misleading comments that claim that append_config() writes to local.conf when it doesn't. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: remove numbers from test namesRoss Burton1
There isn't any required ordering of tests, and the numbers meant that the tests didn't appear in oe-selftest --list-classes. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: clean up selftest.inc in teardownRoss Burton1
Test cases may want to do call bitbake in setUpClass() but at that point the previous selftest.inc is still present which could change the build configuration and result in any built artifacts being removed in the next bitbake invocation as part of the sysroot clean up. Resolve this by cleaning selftest.inc in the tearDown, the clean in setUp should be considered a safety net. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: fix cleaningRoss Burton1
In some situations the native tools built in setUpClass() are wiped from the sysroot by the time the tests are executed, likely due to the cleanup performed in the base setUp() method. Avoid this by doing all of the preparatory building in setUpLocal. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: corrected testcase decorator for test18_iso_imageDaniel Istrate1
Changed testcase decorator for TC test18_iso_image from 1264 to 1346. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: verify that devtool can use plugins in other layersRoss Burton1
In selftest we want to exercise devtool's ability to use plugins from other layers, so add a basic command to reverse a string to meta-selftest and a test in oe-selftest to call it. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/buildoptions: Use the correct script for cleaning the workdirLeonardo Sandoval1
The script 'cleanupworkdir' does not exit, so changing it to 'cleanup-workdir'. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: Updated bitbake TCsDaniel Istrate1
- Added new TC test_force_task_1 (1354); check that do_package_write_rpm() re-executes upon changes in package image. - Updated TC test_force_task_2 (163); changed test recipe to zlib and added do_package() to the task execution list. - Removed unnecessary imports. Fix for bug [YOCTO #5875]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: clean up local DL_DIR/SSTATE_DIR safelyRoss Burton1
Now that the selftests clean selftest.inc in the tearDown using bitbake in a function passed to addCleanup (which happens after teardown) will use the default configuration and not the customised one. As the intention was to clean away the temporary DL_DIR and SSTATE_DIR, simple use track_for_cleanup to delete the entire directory instead. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/utils/ftools: From functions that expect data, check if NoneLeonardo Sandoval1
ftools functions that expect data may get 'None'; this patch does this check and return immediately if this is the case. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>