summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2015-02-14oe-pkgdata-util: add some QA testsPaul Eggleton1
Test each of the subcommands that this utility provides. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-02-14oe-pkgdata-util: improve command-line usagePaul Eggleton1
* Use argparse instead of optparse for standardised help output, options and a much cleaner code structure * Look up pkgdata directory automatically so the user doesn't have to specify it * Use standard logging NOTE: this does mean a slight change in syntax - if you do want to specify the pkgdata directory (usually only necessary if you're calling it from within the build process) you need to use the parameter -p (or --pkgdata-dir) and specify this before the command, not after it. Examples: oe-pkgdata-util find-path /sbin/mke2fs oe-pkgdata-util lookup-recipe libelf1 oe-pkgdata-util read-value PKGSIZE libc6 oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-01-23oeqa/utils/decorators: Try and improve ugly _ErrorHandler tracebacksRichard Purdie1
Currently, if one module is skipped, any other module calling skipModule causes tracebacks about _ErrorHandler not having a _testMethodName method. This reworks the code in a way to avoid some of the problems by using the id() method of the objects. It also maps to the correct name format rather than "setupModule" or just skiping the item entirely. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-01-07oeqa/parselogs: Added a check in case the folder location does not contain ↵Lucian Musat1
any log files Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31oeqa/selftest: fix test_incremental_image_generation for changes in log outputYi Zhao1
test_incremental_image_generation case failed because the log output chanaged: FAIL: test_incremental_image_generation (oeqa.selftest.buildoptions.ImageOptionsTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/buildarea3/yzhao1/poky-build/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f return func(*args) File "/buildarea3/yzhao1/poky-build/meta/lib/oeqa/selftest/buildoptions.py", line 25, in test_incremental_image_generation self.assertEqual(0, res.status, msg="No match for openssh-sshd in log.do_rootfs") AssertionError: 0 != 1 : No match for openssh-sshd in log.do_rootfs ---------------------------------------------------------------------- Using re search instead grep Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31oeqa/parselogs: Add harmless failure warning introduced by new qemu version ↵Richard Purdie1
on qemuppc Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-25oeqa/selftest: fix test_force_task so it doesn't taint the entire buildPaul Eggleton1
Using -C sets a taint on the specified task, and m4-native is in the dependency chain for just about everything, which means that everything gets forced to rebuild. We don't need the influence of this test to extend outside of the test itself, so ensure that the taint gets cleared at the end of the test. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/selftest: populate pkgdata/shlibs in test_recipetool_create_gitPaul Eggleton1
Build some recipes so that we have the needed data in the sysroot for recipetool to refer to when scanning for dependencies. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/selftest: improve failure messages for devtool testsPaul Eggleton1
assertTrue prints "False is not True" if it fails, which is pretty much useless. Use a more appropriate assertion test where practical and add a message where it isn't. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/selftest: skip test_incremental_image_generation if not using rpmPaul Eggleton1
We have just implemented incremental ipk image generation, but at the moment this test doesn't support that, so skip it if not using rpm. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/selftest: fix test_event_handler for changes in bitbake outputPaul Eggleton1
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/utils: fix testcase decorator to allow calling tests individuallyPaul Eggleton1
Without this, running tests individually failed with a traceback: $ oe-selftest --run-test buildoptions.ImageOptionsTests.test_incremental_image_generation 2014-12-23 14:40:37,636 - selftest - INFO - Checking that everything is in order before running the tests 2014-12-23 14:40:38,408 - selftest - INFO - Running bitbake -p 2014-12-23 14:40:40,235 - selftest - INFO - Loading tests from: oeqa.selftest.buildoptions.ImageOptionsTests.test_incremental_image_generation Traceback (most recent call last): File "/home/user/poky/scripts/oe-selftest", line 179, in <module> ret = main() File "/home/user/poky/scripts/oe-selftest", line 164, in main suite.addTests(loader.loadTestsFromName(test)) File "/usr/lib64/python2.7/unittest/loader.py", line 91, in loadTestsFromName module = __import__('.'.join(parts_copy)) File "/home/user/poky/meta/lib/oeqa/selftest/buildoptions.py", line 12, in <module> class ImageOptionsTests(oeSelfTest): File "/home/user/poky/meta/lib/oeqa/selftest/buildoptions.py", line 14, in ImageOptionsTests @testcase(761) NameError: global name 'func' is not defined Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-25oeqa/utils: replace some tabs with spacesPaul Eggleton2
Python code should use spaces only for indentation. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-12-21devtool: add QA testsPaul Eggleton1
Add some QA tests for devtool (and recipetool). These aren't comprehensive but at least they are a start, and have already helped me catch and fix a number of regressions. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21oeqa/utils: make get_bb_var() more reliablePaul Eggleton1
* Enable querying exported variables * Use strip() to remove quotes so any internal quotes are not disturbed Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25oeqa/runtime: Added some new filters for the log parser Including custom ↵Lucian Musat1
filters for edgerouter, minnow and jasperforest. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-10-02oeqa/dmesg: Whitelist usbhid failuresRichard Purdie1
We fixed these in parselog but not here. This test really can just be deleted now really. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02oeqa/parselogs: Ignore qemu usbhid errorsRichard Purdie1
These are harmless from the USB pointer device we install, ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-29oeqa/utils: Added filter to LogResults decorator to enforce custom log level.Lucian Musat1
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-22oeqa/selftest: Added decorators to buildoptions.pyLucian Musat1
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
2014-09-10oeqa/runtime: Automatic test for ptestLucian Musat1
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder. No cleanup is done for packages after the test is finished. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-10oeqa/utils/logparser.py: results based log parser utilityLucian Musat1
A module for parsing results based logs like ptest, compliance and performance. Supports breaking the logs into multiple sections and also provides a result object to use the parser with. The parser is initialized with the regex required to identify results and section statements in the target log file. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-10oeqa: Added package installer to oetest to aid in future automatic install ↵Lucian Musat1
of packages Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-01oeqa: sstatetests should now look for glibc-initialKhem Raj1
Signed-off-by: Khem Raj <raj.khem@gmail.com>
2014-08-29oeqa: xorg log test is being replaced by parselogs so removeRichard Purdie1
Now we have the parselogs test, this one can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/pasrselogs: Improve the machine/string whitelistRichard Purdie1
Currently the whitelist is imcomplete, inaccurate and suffers duplication. These changes: * Add common groups of errors * Change to make the default whitelist clear * Correctly (or at least better) escape the regexp expressions * Add in missing machines to allow builds on the autobuilder to suceed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/runtime: Automatic test for parsing the logs on a machine and search ↵Lucian Musat1
for certain error keywords. This adds a common new qa test for general processing of log files. One significant improvement is machine dependent ignore filters. This can be used to replace several weaker individual QA tests that are currently used. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-27oeqa/sdk/buildsudoku.py: add setUpModule method to run only when gtk+ in ↵Corneliu Stoicescu1
installed. Adding setUpModule in order to skip the module when gtk+ is not installed in the toolchain. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-27oeqa/oetest.py: enable sdk tests to use hasFeature and hasPackage methods.Corneliu Stoicescu1
In order to use the hasFeature and hasPackage methods, we need to make oeSDKTest extend oeTest and also set the test context (tc) attribute in the oeTest class when loading the tests. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23oeqa/utils/httpserver: Reset SIGTERM handlerRichard Purdie1
With bitbake-worker installing a SIGTERM handler, we now need to reset the one here to ensure that when this process shuts down, it doesn't take the rest of the task with it. This does appear to be the only place in OE that we have this problem. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/sdk/: add sdk tests for sudoku, iptables and cvsCorneliu Stoicescu4
Add test modules for sdk tests. NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/utils/targetbuild.py: add support for sdk testsCorneliu Stoicescu1
- Create new abstract class BuildProject that provides basic functionality for a project/package building class * contains abstract method _run() that needs to be implemented by all extending classes. - The old TargetBuildProject class now extends the abstract BuildProjct class - Introducing new SDKBuildProject that extends the abstract BuildProjct class NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/oetest.py: enable sdk testingCorneliu Stoicescu1
- add support for sdk tests in the loadTests and runTests methods - add new oeSDKTest test object NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11oeqa/runtime: add new cpp test and fileCorneliu Stoicescu2
This tests out the limits header which we've noticed does have problems in some SDK builds. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11lib/oeqa/selftest: Don't match log level in outputTyler Hall1
To facilitate changing the log level of the "Fetcher failure" message, search only for the message without the "Error:" prefix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02oeqa/utils/decorators.py: add import osCorneliu Stoicescu1
An 'import os' was omitted here while testing the previous decorators using runtime tests that import the os module before this one. Unfortunately oe-selftest fails because of this missing import. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-07-29selftest/buildoptions.py: fix QA_WARN test and add more output when failingCorneliu Stoicescu1
The -ccleansstate should be done before building the package for the second time. Also printing the command output when failing. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25lib/oeqa: Fix accidental revert of codeRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/rutime: Added testcase decorators for automated runtime tests. Also ↵Lucian Musat27
added LogResults decorator for oeTest class in oetest.py Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/runtime: Added skipModule import for test modules that use it.Lucian Musat8
The modules that use skipModule should import it themselves and not rely on somebody else to import it. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa: Refactor test skipping decorators to use the unittest result objectLucian Musat2
In order to make the test skipping decorators independent of the oeTest object we rely on the unittest result object to construct skip, fail and error lists used by these decorators. Created a new object getResults that analyses upper frames and retrieves the unittest result object instance, then return a list of failed, skipped and error tests. Also removed the oetest import from decorators.py because it was no longer required. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqs/selftest: Added test case decorators for toasterLucian Musat1
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqa: fix return status in pam.py to match shadow-4.2.1Chen Qi1
The return statuses of commands like `su --help' or `passwd --help' in shadow 4.2.1 version are different from those in shadow 4.1.4.3 version. Now that we've upgraded shadow to 4.2.1, we need to fix these statuses in the pam.py to make things work as expected. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10oeqs/selftest: Added test case decorators for the rest of selftest testcases ↵Lucian Musat4
except toaster Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-07-03Added testcase decorator to use in logging. Added class decorator LogResults ↵Lucian Musat3
that outputs test results in separate log file. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-30oeqa/selftest/_toaster.py: fix 'IndentationError' issuesIonut Chisanovici1
There were some indentation errors when the _toaster.py was merged into master. This patch fix those errors. Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-30oeqa/selftest/toaster.py: renamed to _toaster.pyIonut Chisanovici1
In order not to run the toaster selftests by default, we renamed to _toaster.py Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-24oeqa/targetcontrol.py: Separate the matching of supported image fstypes from ↵Corneliu Stoicescu1
the resulting value check. Because we used a bb.fatal call inside the get_image_fstype classmethod, this caused problems when accessed without instantiating the object with a valid bb environment. Separating the matching part of the classmethod(that is usable by outside scripts) from the check of the resulting value. The matching is done within a new classmethod and the latter keeps the old method name and internal functionality, this way we don't have to change any other target controllers code. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13controllers/masterimage.py: Make testimage kernel naming pattern universalCorneliu Stoicescu1
YB: #6254 The QemuTargetControl class does not specify any kernel naming but the runqemu script it uses uses the " KERNEL_IMAGETYPE + MACHINE + '.bin' " naming scheme. Modifying the other major target controller class, MasterImageHardwareTarget, to use the same kernel naming scheme. This is usefull also to outside scripts that want to anticipate the kernel file name for all target controllers. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13oeqa/controllers/testtargetloader.py: add 'import os'Corneliu Stoicescu1
YB: #6254 This module uses os but relies on other modules to import it. Adding 'import os' in order to be self-sustained. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2025-01-19 20:23:27 +0000