summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa/runtime
AgeCommit message (Collapse)AuthorFiles
2015-07-24oeqa/runtime/parselogs: don't use uname to get MACHINERoss Burton1
Instead of running "uname -n" on the target (which actually returns the hostname) to get something approximating the hostname (or not, if that has been overriden), simply fetch MACHINE from the datastore. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/runtime/parselogs: add more x86 log exceptionsRoss Burton1
Move "fail to add MMCONFIG..." to qemux86_common as we started to also see it on x86-64 with Linux 4.1. Add "can't claim BAR..." to qemux86_common. Generalise "Failed to access perfctr MSR" as the MSR number can change. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/syslog: Removed a pointless testcase and added skip for another.Lucian Musat1
The testcase syslog --help was rather useless and also causing problems on images where syslog was non-busybox, like LSB images so I removed it. Added a skip condition for TC 202 for the same reason. If syslog is non-busybox then the test skips. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added decorators for the remaining auto tests.Lucian Musat7
This helps for the automatic completion of the results in testopia. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added a new automated rpm test.Daniel Istrate1
testcase 195: Check rpm install/removal log file size Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added one runtime testcase in connman.Daniel Istrate1
(testcase 223) Check that only one connmand runs in background. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemumips64 runtime errorRichard Purdie1
Similarly to qemumips, ignore these errors upon bootup so that we have a good QA baseline for new errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemuarm64 runtime errorsRichard Purdie1
The qemuarm64 machine doesn't have graphics so whitelist the X server failures to start. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Fixed a problem in parselogsLucian Musat1
When the test passed it returned an error because grep did not return any error and the variable that held the results was being referenced but not assigned any value. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23meta: Add explict getVar param for (non) expansionRichard Purdie2
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11oeqa/parselogs: Added some more errors to the whitelist.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-11oeqa/parselogs: The logs are now copied and parsed locally.Lucian Musat1
This is to fix a problem with reaching the limit of the whitelist size. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-04-09oeqa/runtime: Boot test for poky-tiny.Lucian Musat1
Bug 6705. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/smart: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/rpm: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ping: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/parselogs: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ldd: Added new decorators for existing automated tests.Lucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/systemd.py: skip instead of failing without avahiPatrick Ohly1
The SystemdServiceTests assume that avahi-daemon is installed, which is not necessarily the case depending on the image being tested. Better check this dependency before starting the tests and skip them if the service is not installed. This has to be done for each test instead of for the entire module, because other tests in the module can run without avavi. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21oeqa/runtime: Added a new auto rpm testLucian Musat1
The test tries to query the rpm list with a non-root user Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20oeqa/runtime: Fixed a problem in logparser where it failed to whitelist some ↵Lucian Musat1
common errors. And some minor tweaks like moving some errors from qemu to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-16lib/oeqa/runtime/systemd: Add a basic runtime test for the journalRandy Witt1
The test_systemd_journal() test will fail if journalctl exits with a non-zero exit status. The exit status is non-zero even if there are no journal files, which is what drove the creation of this test. [Yocto #7388] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-12oeqa/parselogs: Skip hda opcode errorsRichard Purdie1
These occur when running images under virtualisation on machines with high load which sometimes trigger timeouts in the kernel DMA code. They're harmless to ignore in these cases. Adding this since "failing" the build due to this is more annoying thank useful. [YOCTO #7387] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-21oeqa/parselogs: Add exception for new mips error messageRichard Purdie1
The 3.19 kernel introduces this error, ignore it for now. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-19oeqa/ptest: Fixed complementary package install detection and added ↵Lucian Musat1
ptest-runner exit code check Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-15oeqa/ptest: Removed buildhistory as requirement for ptestLucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oeqa/smart: Added some new test casesLucian Musat1
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oe-pkgdata-util: improve command-line usagePaul Eggleton1
* Use argparse instead of optparse for standardised help output, options and a much cleaner code structure * Look up pkgdata directory automatically so the user doesn't have to specify it * Use standard logging NOTE: this does mean a slight change in syntax - if you do want to specify the pkgdata directory (usually only necessary if you're calling it from within the build process) you need to use the parameter -p (or --pkgdata-dir) and specify this before the command, not after it. Examples: oe-pkgdata-util find-path /sbin/mke2fs oe-pkgdata-util lookup-recipe libelf1 oe-pkgdata-util read-value PKGSIZE libc6 oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-01-07oeqa/parselogs: Added a check in case the folder location does not contain ↵Lucian Musat1
any log files Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31oeqa/parselogs: Add harmless failure warning introduced by new qemu version ↵Richard Purdie1
on qemuppc Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25oeqa/runtime: Added some new filters for the log parser Including custom ↵Lucian Musat1
filters for edgerouter, minnow and jasperforest. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-10-02oeqa/dmesg: Whitelist usbhid failuresRichard Purdie1
We fixed these in parselog but not here. This test really can just be deleted now really. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02oeqa/parselogs: Ignore qemu usbhid errorsRichard Purdie1
These are harmless from the USB pointer device we install, ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10oeqa/runtime: Automatic test for ptestLucian Musat1
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder. No cleanup is done for packages after the test is finished. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-08-29oeqa: xorg log test is being replaced by parselogs so removeRichard Purdie1
Now we have the parselogs test, this one can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/pasrselogs: Improve the machine/string whitelistRichard Purdie1
Currently the whitelist is imcomplete, inaccurate and suffers duplication. These changes: * Add common groups of errors * Change to make the default whitelist clear * Correctly (or at least better) escape the regexp expressions * Add in missing machines to allow builds on the autobuilder to suceed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/runtime: Automatic test for parsing the logs on a machine and search ↵Lucian Musat1
for certain error keywords. This adds a common new qa test for general processing of log files. One significant improvement is machine dependent ignore filters. This can be used to replace several weaker individual QA tests that are currently used. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11oeqa/runtime: add new cpp test and fileCorneliu Stoicescu2
This tests out the limits header which we've noticed does have problems in some SDK builds. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25lib/oeqa: Fix accidental revert of codeRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/rutime: Added testcase decorators for automated runtime tests. Also ↵Lucian Musat26
added LogResults decorator for oeTest class in oetest.py Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/runtime: Added skipModule import for test modules that use it.Lucian Musat8
The modules that use skipModule should import it themselves and not rely on somebody else to import it. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqa: fix return status in pam.py to match shadow-4.2.1Chen Qi1
The return statuses of commands like `su --help' or `passwd --help' in shadow 4.2.1 version are different from those in shadow 4.1.4.3 version. Now that we've upgraded shadow to 4.2.1, we need to fix these statuses in the pam.py to make things work as expected. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-31lib/oeqa: add a test target controller for EFI targetsStefan Stanacar1
The purpose of this module is to deploy a test image on a EFI-enabled hardware and run our runtime tests. A bit of background: - testimage.bbclass uses the concept of TEST_TARGET which is a class name that is responsible for target deploying. A layer can provide it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget (ssh into an already up and running machine and run tests), the default one being qemu. - basically testimage does something like: target.deploy() try: target.start() runTests() finally: target.stop() This module assumes a running EFI machine with gummiboot as bootloader and core-image-testmaster installed (or similar). Also your hardware under test has to be in a DHCP-enabled network that gives it the same IP for each reboot. One time setup (master image): - build core-image-testmaster with EFI_PROVIDER = "gummiboot" - install the image on the target Test image setup: - build your test image, e.g core-image-sato as you usually do, but with these in local.conf: IMAGE_FSTYPES += "tar.gz" - Now run the tests: INHERIT += "testimage" TEST_TARGET = "GummibootTarget" TEST_TARGET_IP = "192.168.2.3" bitbake core-image-sato -c testimage Other notes: - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling. The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like TEST_POWERCONTROL_CMD="powercontrol.exp test 10.11.12.1 nuc1" in local.conf. Basically my expect script does: 'ssh test@10.11.12.1 "pyctl nuc1 <arg>" and runs a python script there that controls power for a label called nuc1'. The reason why my expect script has to ssh into another machine is because of network topology, and that machine is the one actually connected to the test rack and the power strip. That's why TEST_POWERCONTROL_CMD and _ARGS need to be customized for one's setup, the only requirement being that it accepts: on/off/cycle as the last argument. - if no command is defined it would use classic reboot. This is fine as long as the machine actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where some manual interaction is okay from time to time. [YOCTO #5614] Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02oeqa/runtime/dmesg: Ignore DMA timeout errorsRichard Purdie1
We see these from qemu images on servers under load. They're not interesting and clutter the build output so lets ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-12oeqa/runtime: smart: don't run in auto mode when rpm is not the pmStefan Stanacar1
TEST_SUITES = "auto" would run smart tests even for non-rpm images, which is pointless. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11oeqa/utils: targetbuild: take proxy into accountStefan Stanacar3
A previous commit broke downloads when proxies are involved, let's fix it. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-04oeqa/runtime/systemd: remove race in settle()Ross Burton1
The settle() function had a race where services could still be activating at two minutes but then when the final log is output, they've activated. Remove this race and generally clean up the code. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-02oeqa/runtime: syslog: update --help testStefan Stanacar1
busybox 1.22 now returns exitcode 0 instead of 1 for --help options, so this test needs to be updated when busybox gets upgraded to 1.22. https://bugs.busybox.net/show_bug.cgi?id=5612 http://git.busybox.net/busybox/commit/?id=efd0698f74caab0a0c8a51228b923ee142e8e278 Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-01-29oeqa/runtime: smart: fix adding channelsStefan Stanacar1
Without this it won't add core2-64 and similar channels, as the directory name in deploy_dir/rpm uses _ not - as the package arch. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29oeqa/runtime: drop restartTarget methodStefan Stanacar3
Restart is a property of a specific target, not of a test class, should a test really need to restart the target the direct method should be called. Also some tests used this to enforce more ram, which makes sense only for qemu targets only (and the inital reason this was needed isn't valid anymore, qemu machines had the default ram size bumped a while ago). Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-01-15 06:23:19 +0000