summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa/runtime
AgeCommit message (Collapse)AuthorFiles
2017-02-23oeqa/runtime/context.py: Fix use of getTarget() with testexportMariano Lopez1
The idea on getTarget is to use kwargs to send custom variables to different targets, instead of this, a new variable was added (just used for custom targets) and this broke testexport. So in order to fix it, just add the custom variable to kwargs. This fixes the use of getTarget() in testexport class that was introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15testimage: Implement custom target loadingJuro Bystricky1
This patch implements custom target loading for testimage, currently missing due to major changes to the test framework. Custom targets can be defined in various meta-layers, so we need an extra path information in order to find them. Any other additional info is retrieved as usual via the variables TEST_TARGET and TEST_SUITES Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05oeqa/runtime: Improve failure log outputRichard Purdie4
Printing a message which says "configure failed" without the log output is effectively useless. If a command fails, print the output by default and simplify the calling code which makes debugging any of these failures much easier. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31oeqa/runtime/cases: Rename syslog module to oe_syslogAníbal Limón1
Debian based distros has a builtin syslog module so when try to load tests using unittest it references the builtin module instead of runtime/cases. [YOCTO #10964] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31runtime/cases: Fix case numbers, missing cases and unused classesMariano Lopez3
gcc: Removed unneded lines. multilib: fixed case number. syslog: added a missing test. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSESMariano Lopez1
smart test requires to build the image using rpm packages, this check was included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is not true when building packages for rpm and deb/ipk. So this would check IMAGE_PKGTYPE instead. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31runtime/cases/connman.py: Stop using oeRuntimeTestMariano Lopez1
oeRuntimeTest class is not used anymore as part of runtime migration, this particular case was missed, so fix it. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23oeqa/runtime/context.py: Add defaults for runtime contextMariano Lopez1
This adds default values to OERuntimeTestContextExecutor class in order to make easier the execution of exported test that were generated with testexport class. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23testimage.bbclass: Add package install featureMariano Lopez2
This allows to use the package install feature with the new OEQA framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23runtime/cases/smart.py: Migrate smart testsMariano Lopez1
This migrates the smart test from the old framework to the new one. This has its own commit because smart test was using bb and oe libraries that are available when exporting the test cases to run in a different host. Because of the removal of bb and oe libraries index and packages feeds creation will be managed in testimage bbclass. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/cases: Migrate runtime tests.Mariano Lopez28
This migrates current runtime test suite to be used with the new framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/context.py: Add logger to getTargetMariano Lopez1
Current targets (ssh and qemu) require a logger in their constructors, so in order to get a new target we need to provide the logger. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/context.py: Prepare for qemuAníbal Limón1
This commit will prepare context to use qemu and not just simpleremote. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor classAníbal Limón1
To avoid getVar calls inside a utils module, also moves get_host_dumper import inside testexport isn't needed. [YOCTO #10231] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/context: Move helper functions for process args to executorAníbal Limón1
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/context: Add runtime option group and options for target type ↵Aníbal Limón1
and server ip. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters ↵Mariano Lopez1
per line Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/files: Move runtime files from old directoryMariano Lopez4
As part of the refactor we require to move the files used in runtime testing to the new directory. This also adds the path to the runtime test context. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime: Add OEHasPackage decoratorMariano Lopez1
This new decorator will be used to skip the test if the image under test doesn't have the required packages installed. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime: Add case, context and loader classes for runtime testingMariano Lopez3
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader to be used for runtime testing. As expected there are some changes in runtime context: - Adds the target to be used for runtime testing, the default is a SSH connection to the device under test running a OE image. - Runtime context requires image manifest because several tests are skipped if a package is missing or installed. - Several tests require the output of the ps command and it changes its output and arguments if busybox o procps is installed, so the case must use the correct ps command. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime: Move to runtime_casesAníbal Limón31
The new oeqa core framework will modify the structure of the runtime folder the new runtime folder will have python code inside to support runtime test cases. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa: Fix files handling on runtime tests.Aníbal Limón3
Common files was move to oeqa/files from oeqa/runtime/files because the same files are used across Runtime,SDK,eSDK tests. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime: Fix TargetBuildProject instancesAníbal Limón3
TargetBuildProject was refactored to avoid bitbake dependency so the instance don't allow to pass data store anymore. classes/testimage: Export proxies before run tests The TargetBuildProject based tests download archives from network. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/utils: {Target,SDK,}BuildProject remove dependency of bbAníbal Limón1
Don't use bitbake references inside utils modules, in order todo that changes getVar calls for arguments in the __init__ method like dl_dir for all the classes and testlogdir, builddatetime in SDKBUildProject. Also don't export proxies inside _download_archive method, a good practice is to setup the proxies at init of the process instead of do it in this helper module. [YOCTO #10231] [YOCTO #10599] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/utils: Move targetbuild to buildproject moduleAníbal Limón2
The new buildproject module will contain only BuildProject class a helper class for build source code. The remaining classes TargetBuildProject and SDKBuildProject was move to runtime and sdk respectively. [YOCTO #10599] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk moduleAníbal Limón1
It doesn't make sense to have files related to sdk module into runtime module. [YOCTO #10599] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa: Move common files to oeqa/files instead of runtime onlyAníbal Limón4
Those files are used by runtime and sdk test cases, so move to base directory of oeqa module. [YOCTO #10599] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-16runtime: Add cleanup for logrotate testsJose Perez Carranza1
Delete logrotate dir to avoid errors when test are executed more than 1 time on the same target. Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie1
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16meta: remove True option to getVar callsJoshua Lock9
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13parselogs.py: Don't clog QA with Joule errorsCalifornia Sullivan1
The Joule is very new hardware and there is ongoing kernel and firmware work to fix these issues, which will be available in future kernel and firmware releases. In the meantime, don't clog QA reports. [YOCTO #10611] Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-07parselogs: Whitelist GPT warnings as the device is fully functionalJair Gonzalez1
The warning occurs when the GPT image is not the same size than the media into which it's being flashed, causing the backup GPT table not being at the end of the disk. However, this is expected as the image is created before having the information about the destination media. The error is harmless, so it will be whitelisted. Fixes [YOCTO 10481]. Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-07parselogs.py: Whitelist iwlwifi firmware load error messagesCalifornia Sullivan1
The iwlwifi module of any given kernel has a minimum and maximum supported firmware version. The kernel begins by attempting to load the maximum version, and decrements until it is successful. The 4.8 kernel's maximum supported firmware version is 24, but thus far only 22 has been released, meaning we get errors for 24 and 23. Filter out iwlwifi firmware load error messages, as they are not necessarily indicative of real problems. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-30runtime: Update test cases numbers for runtime testsJose Perez Carranza4
Update test case numbers on runtime tests to do match with templates defined on Testopia for 2.3 release Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23parselogs: Whitelist NUC6 firmware load error message on genericx86-64California Sullivan1
This was already whitelisted, but the 4.8 kernel changed the error message, causing it to get caught by parselogs again. Fixes [YOCTO #10494]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-05parselogs.py: Add disabling eDP error to x86_common whitelistCalifornia Sullivan1
The NUC6 firmware tells the kernel to try and initialize an embedded DisplayPort it does not have, causing this warning. Its harmless, so just whitelist it. Fixes [YOCTO #9434]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-29parselogs: Update uvesafb errors in qemu whitelistJianxun Zhang1
This change only whitelists the timeout message in infinite wait case in uvesafb driver. With the latest timeout patch introducing infinite wait in uvesafb driver, we whitelist the timeout message since it works as a warning for issues related to timeout not to be fixed in build servers. We remove other errors for bug-discovering purposes in some cases where these lines are still worthy to be caught (not whitelisted): The removed errors show up again in the infinite wait case. It indicates a different root cause or the timeout patch doesn't work correctly. Timeout happens when developers explicitly set a non-negative timeout of a limited period to wait for task completion in uvesafb driver. Timeout or/and errors occur when kernel doesn't have the latest timeout patch in driver. Note: The latest timeout patch is tracked by: a2966330bcd29e99c0403235edb29433b0c53d56 [YOCTO #8245] Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28parselogs.py: Add amd_nb error to x86_common whitelistCalifornia Sullivan1
This has always silently failed on hardware without AMD Northbridge, and a recent kernel patch made it not silent. It would be ideal to only whitelist the error for genericx86 MACHINEs and disable the CONFIG option that enables it in intel-* MACHINEs, but in order to disable this configuration option we would have to enable EXPERT and DEBUG_KERNEL, which we don't want. Instead just whitelist it on all x86 MACHINEs. Fixes [YOCTO #10261]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20oeqa/runtime/parselogs.py: Add ignore of tsc calibration fail in x86Aníbal Limón1
We are experimenting failures to calibrate CPU's using TSC in x86 VM's due to usage of nested KVM [1], this is a known issue [2][3] in virtualization environments, for detail explnation see [4]. Also we already have an ignore for 'TSC Fast calibration fail'. [1] http://errors.yoctoproject.org/Errors/Details/83684/ [2] https://bugzilla.redhat.com/show_bug.cgi?id=814231 [3] https://lists.nongnu.org/archive/html/qemu-devel/2010-09/msg01547.html [4] https://www.kernel.org/doc/Documentation/virtual/kvm/timekeeping.txt Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-14Revert "oeqa.runtime.smart: work around smart race issues"Richard Purdie1
We have a proper fix now so we can remove the workaround. This reverts commit 4d268abc2fc892c5d34449f78c8e9f2b1a9d6bac.
2016-09-13oeqa: Remove linux user utilized for rpm test.Edwin Plauchu1
When trying to re-test smart rpm tests. A fail arises due to a linux user previously created upon the image. We've added a few lines to delete such user and his home dir when finishing test. [YOCTO #9204] Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-09oeqa/parselogs: Don't use cwd for file transfersRichard Purdie1
If you run: MACHINE=A bitbake <image> -c testimage MACHINE=B bitbake <image> -c testimage and A has errors in parselogs, machine B can pick these up and cause immense confusion. This is because the test transfers the log files to cwd which is usually TOPDIR. This is clearly bad and this patch uses a subdir of WORKDIR to ensure machines don't contaminate each other. Also ensure any previous logs are cleaned up from any existing transfer directory. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09oeqa.runtime.smart: work around smart race issuesMarkus Lehtonen1
Yucku hack around test failures which ultimately are caused by a race in smartpm itself. Issuing smartpm commands in quick succession causes races in package cache of smartpm on some systems. This patch mitigates the problem by sleeping for 1 second after each smartpm command that modifies the system. [YOCTO #10244] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09oeqa/runtime/smart: Prune feeds to save memoryRichard Purdie1
Full package feed indexes overload a 256MB image so reduce the number of rpms the feed. Filter to p* since we use the psplash packages and this leaves some allarch and machine arch packages too. [YOCTO #8771] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09parselogs: Whitelist qemux86 error message with qemu 2.7.0Robert Yang1
qemu 2.7.0 introduces kernel errors: [ 2.310768] pci 0000:00:00.0: [11ab:4620] type 00 class 0x060000 [ 2.311338] pci 0000:00:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size) [ 2.311604] pci 0000:00:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size) [ 2.311835] pci 0000:00:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size) [ 2.312063] pci 0000:00:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size) [ 2.312323] pci 0000:00:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size) [ 2.314320] pci 0000:00:0a.0: [8086:7110] type 00 class 0x060100 [ 2.315363] pci 0000:00:0a.1: [8086:7111] type 00 class 0x010180 Whitelist this for now since this is preferable to the random failures we're seeing from qemuppc with 2.6.0. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09parselogs: Whitelist qemuppc error message with qemu 2.7.0Richard Purdie1
qemu 2.7.0 introduces kernel errors: [0.474981] pci 0000:00:0f.0: reg 0x10: [io 0x0400-0x041f] [0.483796] pci 0000:00:0f.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff82800000) [0.484204] pci 0000:00:0f.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.488077] pci 0000:00:0f.0: reg 0x30: [mem 0x83000000-0x8303ffff pref] [0.488903] pci 0000:00:10.0: [1af4:1005] type 00 class 0x00ff00 [0.490485] pci 0000:00:10.0: reg 0x10: [io 0x0480-0x049f] [0.496512] pci 0000:00:10.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff83800000) [0.496783] pci 0000:00:10.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.500345] pci 0000:00:11.0: [1af4:1001] type 00 class 0x010000 [0.501790] pci 0000:00:11.0: reg 0x10: [io 0x0500-0x053f] [0.507362] pci 0000:00:11.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff84000000) [0.507677] pci 0000:00:11.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.513905] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00 [0.516493] PCI 0000:00 Cannot reserve Legacy IO [io 0x0000-0x0fff] [0.517512] pci 0000:00:0f.0: BAR 4: assigned [mem 0x80800000-0x80ffffff 64bit pref] [0.518877] pci 0000:00:10.0: BAR 4: assigned [mem 0x82800000-0x82ffffff 64bit pref] [0.519890] pci 0000:00:11.0: BAR 4: assigned [mem 0x83800000-0x83ffffff 64bit pref] Whitelist this for now since this is preferable to the random failures we're seeing from qemuppc with 2.6.0. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05oeqa/parselogs: Add qemuarm64 warning from 4.8 kernel to whitelistRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05oeqa/parselogs/qemuarm: Whitelist amba and jitter for 4.8+ kernelsBruce Ashfield1
With the update to the 4.8 kernel the versatile platform (and hence qemuarm) has switched to a device tree boot. We are using an ummodified mainline kernel versatilepb device tree, which includes definitions of multiple amba devices. These devices are not present in the qemu system emulation, hence throw warnings during boot. These warnings are not unique to oe-core, and rather than carry kernel patches to the device tree (for now), we whitelist the known warnings so qa testing will pass. We also can't turn amba off completely, since it is providing valid devices (like the serial port) and AMBA is force selected by other kconfig values. We also have a jitterentropy warning that shows up on some hosts. This warning is harmless, and like amba we can't turn it off in a fragment since it is force selected by crypto (and we'd rather not turn all crypto off). So we add it to the whitelist while investigations continue into what is needed in the host to support this fully. Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25oeqa/runtime/rpm: use su instead of sudoRoss Burton1
This test works fine with su, which is more likely to be installed in images than sudo. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-25parselogs: Ignore uvesafb timeoutsRichard Purdie1
We're periodically seeing uvesafb timeouts on the autobuilder. Whitelist these errors as there is little it seems we can do about them and we therefore choose to ignore them rather than fail the builds. [YOCTO #8245] There is a better solution proposed in the bug with a -1 timeout however this avoids failed builds until such times as that is implemented. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-09 19:21:17 +0000