summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2015-10-03oe-selftest: wic: fix LocalSetupEd Bartosh1
Appended required features to build config every time LocalSetup is run to ensure that core-image-minimal is not rebuilt due to configuration changes. This should fix test22_mkgummidisk and test23_mkefidisk testcases as they depend on specific features set in LocalSetup. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/runexported: Replaced optionparser with argparse.Lucian Musat1
Also added the default json file name as default for the first positional argument. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01testimage: Added IO commands to dumpsMariano Lopez1
This change just add two more commands to the host dumps to get more information related to the IO. [YOCTO #8412] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01Revert "oeqa/runtime: Added one runtime testcase in connman."Cristian Iorga1
connman daemon forks during normal usage, so this test is no longer valid and generates sometimes false positives. This reverts commit 7d01c595c96eb3d67b90caf71050c4e9345257de. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/wic: Use SetupLocal instead of SetupRichard Purdie1
If we replace the code Setup method, we don't get the cleanup it performs and this just resulted in failures on the autobuilder due to a stale config fragment. Setup will call SetupLocal so this should be a safe and easy fix to resolve the auobuilder failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add test that MACHINE doesn't change target sigsRichard Purdie1
When we change between two machines with the same tune, we shouldn't see rebuilds of binaries. This adds a test for this using the qemux86copy machine. We also extend the test to cover multilibs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add check for same sigs for SDKMACHINERichard Purdie1
Extend one of the sstate tests to also ensure that changing SDKMACHINE doesn't change the target task checksums. Also fix a typo and improve debugging by turning the diff filtering off in all cases (if the test fails, we want to full list). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/utils/decorators: Append the testname without the full pathLeonardo Sandoval1
When getting the failures/errors/skipped lists, include the unit test without the full path. This issue was found on this scenario | test_1_logrotate_setup (oeqa.runtime.logrotate.LogrotateTest) ... FAIL | test_2_logrotate (oeqa.runtime.logrotate.LogrotateTest) ... ok Where test_1_logrotate failed and test_2_logrotate should not have run because @skipUnlessPassed("test_1_logrotate_setup") def test_2_logrotate(self): Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oetest: Change logic of a failed testMariano Lopez1
Currently the logic to check if a test failed was to check for an exception in the thread, but some decorators used in the syslog runtime test would generate and handle exceptions; this will mess with the current check logic and will dump the host and the target as if the test failed. This patch changes the check logic to verify if the test that just happend is in the failure test list and dump the host and target accordingly. [YOCTO #8406] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/runexported: Fix a problem with ssh_target_log existing in folder.Lucian Musat1
When copying the exported tests to a remote machine ssh_target_log can be transformed from softlink to file which will throw an error when trying to run again. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29qemurunner: Sanitize output from qemu and qemu pidMariano Lopez1
Currently the output from qemu could contain control or Unicode characters; having such characters in the log will cause an internal server error when sending the report to error reporting web. Control characters can be found in the command line used to run quemu too. This change sanitize the output from qemu and the command line used to run qemu, this way the logs doesn't contain control or Unicode characters and this will solve the issue of error reporting web and generate better logs. The only Unicode character found in the qemu output is the copyright symbol, it really doesn't interfer removing Unicode characters with debugging. [YOCTO #8225] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/decorators: Fixed a problem with decorator logs link.Lucian Musat1
When not doing actual tests with testimage, like for example exporting tests, the link to the log file was still created although the actual log file was not existent. Fixed it by moving the link creation part into the run() method. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/selftest/wic: Added testcase decorator to all testcases + fixed minor ↵Daniel Istrate1
typos. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: better support for local source filesMarkus Lehtonen1
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking repo statusMarkus Lehtonen1
New method for checking the status of the working tree of a repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking srctree repoMarkus Lehtonen1
Removes some code duplication. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add setup() methodMarkus Lehtonen1
In order to remove some code duplication. The method currently only does checking of workspace dir. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-28oeqa/sdk/gcc: Fix makefile testRichard Purdie2
The makefile test was hardcoding a test for gcc. In the SDK context, we need to use $(CC) to use the cross compiler. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28oeqa/parselogs: Updated log parser whitelist.Lucian Musat1
Added some fixes for meta-intel boards and moved some errors from specific images to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28oeqa/decorators: Add timestamp to decorator logs.Lucian Musat1
To avoid logs being overwriten when running the automated tests multiple times, log files include timestamps in their names and a link is created to point to the latest one. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28qemurunner: Handle qemu start failure correctlyRichard Purdie1
If qemu didn't start correctly, we may not have registered the child signal. This results in a nasty traceback which confuses the underlying issue. Cleanup this code and make the handler cleanup conditional. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-26oeqa/oetest: Fix SDK command executionRichard Purdie1
The SDK environment wasn't being sourced when running these test commands, meaning in some cases the cross compiler was being tested, not the SDK tools. This is clearly not the intent so fix this by ensuring the SDK environment is present. This fixes test failures in multilib SDKs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/selftest/archiver: Test that archiver filters on recipe nameDaniel Istrate1
[YOCTO #6929] this test validates the feature introduced in bug 6929 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/utils/dump: Add default commands and directoryMariano Lopez1
Currently if qemu fails when running a selftest and tries to run some commands on the host it will fail because some variables required by the Dumper class do not exist because testimage was not included. This change adds a default parent directory to save the dumps for the host or target. Also adds default commands to run if no commands were provided to the class. With these changes the previous errors using selftest don't show anymore. [YOCTO #8306] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/runexported: Removed DEPLOY_DIR as mandatory.Lucian Musat1
We don't need DEPLOY_DIR for every runtime test so there is no need for it to be mandatory. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/oetest: Remove bb as requirement for oetest.Lucian Musat1
In order for the test export runner to work oetest needs to be separated from bitbake environment. There is no need to use bb import here so we can use a logger instead. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/runtime/parselogs.py: Fix dmesg log retrieve in satoAníbal Limón1
Sato uses busybox that fails to write log using echo "" because dmesg output can contain special characters. [YOCTO #8377] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread startAníbal Limón1
The Starting logging thread message is also executed on run() inside LoggingThread class. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22oeqa/utils/qemurunner.py: Fix HIGH CPU usage on LoggingThreadAníbal Limón1
LoggingThread is used for receive console output from QEMU over TCP, so add filter to only wake poll on read events, also change the event mask variable name to be more descriptive. This fixes HIGH CPU consume caused by wake on ready to write events. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton1
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22oeqa/selftest: buildoptions.py Removed unused importsDaniel Istrate1
Removed unused imports: unittest, logging, pexpect Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-21oeqa/testimage: Enhance -v switch in testimageLucian Musat1
When testimage is run with -v switch now individual test progress can be seen directly in bitbake console. [YOCTO #6841] Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-16oeqa/sstatetests: Add test for nativesdk stamp invariance with MACHINERichard Purdie1
nativesdk-glbic should not rebuild when you change MACHINE but it was. We've fixed that, now add tests to ensure this doesn't happen again. Rather than add yet another stamps test, extend one of the existing ones to cover this instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-15oeqa/decorators: Added decorator to restart the DUT in case of test hang.Lucian Musat2
Once the DUT is hanged during testing, currently all the following test cases have to wait for default timeout to exit. Using this decorator the user can choose a timeout at case by case basis and what happens when the timeout is reached by overwriting the self.target.restart method. [YOCTO #7853] Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12oeqa/targetbuild: Ensure we run gnu-configize so config files are up to dateRichard Purdie1
On aarch64 we need to do this as the versions in the upstream source don't recognise the target triplet causing SDK test failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12oeqa: Test failure/cleanup improvementsRichard Purdie1
Currently, if qemu segfaults, the tests merrily continue trying to execute which takes time for them to timeout and is a bit silly. Worse, no logs about the segfault are shown to the user, its silent! This patch tries to unravel the tangled web of issues and ensures that we: * install a SIGCHLD handler which tells the user qemu exited * check if qemu is running, if it isn't fail the test outright * don't leave processes behind in sshcontrol which would hold bitbake.lock and block shutdown (From OE-Core rev: 77e9363feba53b72429154be5713c46b007ae0a4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-11oeqa/runtime/date: stop systemd-timesyncd during testRoss Burton1
There's a race between systemd-timesyncd manipulating the system time (with NTP lookups) and the test case's time manipulation. Prevent this by stopping systemd-timesyncd for the duration of the test case. Thanks to Khem Raj for root-causing this. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-09oeqa/sstatettests: Improve output of assertRichard Purdie1
Currently if this fails you see: "AssertionError: 1 != 0" which is useless. Add the output of the failed command so we can stand some chance of debugging what went wrong. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: update-recipe: better 'auto' modeMarkus Lehtonen1
Enhance the logic behind the 'auto' mode a bit by only updating the SRCREV if the changes are already found upstream. The logic is simple: update SRCREV only if the current local HEAD commit is found in the remote branch (i.e. 'origin/<branch_name>'). Otherwise resort to patching. This affects a couple of the oe-selftest tests so update those as well. [YOCTO #7907] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: add upgrade featureLeonardo Sandoval1
Upgrades a recipe to a particular version and downloads the source code into a folder. User can avoid patching the source code. These are the general steps of the upgrade function: - Extract current recipe source code into srctree and create a branch - Extract upgrade recipe source code into srctree and rebase with previous branch. In case the rebase is not correctly applied, source code will not be deleted, so user correct the patches - Creates the new recipe under the workspace [YOCTO #7642] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09oeqa/parselogs: Updated log error whitelist.Cristina Agurida1
[YOCTO #7898] [YOCTO #7761] [YOCTO #7804] Signed-off-by: Cristina Agurida <cristina-danielax.agurida@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09qemurunner: Fix killpg pid usageRichard Purdie1
killpg should be being called with a pgid, not a pid. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09qemurunner: Ensure runqemu doesn't survive SIGKILLRichard Purdie1
Currently, we see runqemu and qemu-system-* processes left behind when bitbake is killed by buildbot. This is due to the use of setpgrp() in the runqemu subprocess call. We need the setpgrp call so that all runqemu processes can easily be killed (by killing their process group). This presents a problem if this controlling process itself is killed however since those processes don't notice the death of the parent and merrily continue on. Rather than hack runqemu to deal with this, we add something to qemurunner, at least for now to resolve the issue. Basically we fork off another process which holds an open pipe to the parent and also is setpgrp. If/when the pipe sees EOF from the parent dieing, it kills the process group. This is like pctrl's PDEATHSIG but for a process group rather than a single process. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-07qemurunner: Drop error to a warning to improve user feedbackRichard Purdie1
If a task shows an error, the full log is surpressed since bitbake assumes the user has been shown what is wrong. In this code path that isn't the case and its much more helpful to show the user the full error. Therefore show a warning instead to aid usability. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-07testimage: handle SIGTERM to conclude runqemuMariano Lopez1
In the current state if a SIGTERM is sent to the testimage worker, the worker will exit but runqemu and qemu won't exit and the processes need to be killed manually to free the bitbake lock. This allows to catch the SIGTERM signal in testimage, this way it is possible to stop runqemu and qemu and allow to free the bitbake lock. Also this allows to skip the rest of the tests when running the tests in qemu or real hardware. This also solves minimal breaks in the setup of the runtime test when checking if qemu is alive. [YOCTO #8239] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-06oeqa/decorators: Fixed a problem with tests having the same names.Lucian Musat1
When two or more tests had the same name but different classes then the decorator log whould have the output all wrong. This was because a comparison which was made only between method names but now it compares classes too. [YOCTO #8029] Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-05oeqa: Test failure/cleanup improvementsRichard Purdie4
Currently, if qemu segfaults, the tests merrily continue trying to execute which takes time for them to timeout and is a bit silly. Worse, no logs about the segfault are shown to the user, its silent! This patch tries to unravel the tangled web of issues and ensures that we: * install a SIGCHLD handler which tells the user qemu exited * check if qemu is running, if it isn't fail the test outright * don't leave processes behind in sshcontrol which would hold bitbake.lock and block shutdown Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03testimage: filter proper test cases by tagszjh2
If a test case is decorate by oeqa.utils.decorators.tag, this case will by add a tag, testrunner will filter these tags by TEST_SUITES_TAGS [YOCTO #7849] Signed-off-by: zjh <junhuix.zhang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02qemurunner: Handle lack of data on run serial gracefullyMariano Lopez2
This changes the behavior when data was not received over the serial console when a command is run. With this the socket is no longer closed but it throws and exception that can handled in upper layers. With this the test can continue without throwing errors for not having the socket anymore. [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02wic: add test cases for 3 imagesEd Bartosh1
Added test cases to build qemux86_directdisk, mkgummidisk and mkefdisk images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>