summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa/core
AgeCommit message (Collapse)AuthorFiles
2017-04-12oeqa/core/runner: Append PID in the test result folder nameLeonardo Sandoval1
It was found a case (see the bugzilla entry) where two runners were running at the same second, creating identical test result folders, so one of them (the second runner) was not able to create the folder because the other has already created it, raising the following exception (many text was removed from log) NOTE: Executing RunQueue Tasks NOTE: Running task 1 of 2 (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/recipes-core/images/core-image-minimal.bb:do_testsdkext) NOTE: Running task 2 of 2 (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/recipes-sato/images/core-image-sato.bb:do_testsdkext) NOTE: recipe core-image-sato-1.0-r0: task do_testsdkext: Started NOTE: recipe core-image-minimal-1.0-r0: task do_testsdkext: Started . . The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_testsdkext(d) 0003: File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/classes/testsdk.bbclass', lineno: 188, function: do_testsdkext 0184: 0185:testsdkext_main[vardepsexclude] =+ "BB_ORIGENV" 0186: 0187:python do_testsdkext() { *** 0188: testsdkext_main(d) 0189:} 0190:addtask testsdkext 0191:do_testsdkext[nostamp] = "1" 0192: File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/classes/testsdk.bbclass', lineno: 171, function: testsdkext_main 0167: except Exception as e: 0168: import traceback 0169: bb.fatal("Loading tests failed:\n%s" % traceback.format_exc()) 0170: *** 0171: result = tc.runTests() . . File: '/usr/lib64/python3.5/os.py', lineno: 241, function: makedirs 0237: cdir = bytes(curdir, 'ASCII') 0238: if tail == cdir: # xxx/newdir/. exists if xxx/newdir exists 0239: return 0240: try: *** 0241: mkdir(name, mode) 0242: except OSError: 0243: # Cannot rely on checking for EEXIST, since the operating system 0244: # could give priority to other errors like EACCES or EROFS 0245: if not exist_ok or not path.isdir(name): Exception: FileExistsError: [Errno 17] File exists: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/build/TestResults_20170409130114' [YOCTO #11318] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28oeqa/core/loader.py: Do not import underscore modules by defaultMariano Lopez1
Underscore modules are meant to be run only when manually added to the test suite, so far another mechanisms are in place to make this happen with runtime, sdk, and esdk (mostly in test* bbclasses). This will add such functionality in the core framework so other specific frameworks can take use this without adding something else. [YOCTO #10980] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oeqa/core/loader.py: Avoid importing tests with built-ins nameMariano Lopez1
If importing a test with the same name as a built-in module, it will silently import the built-in and check for tests in built-in module. This happened with syslog module in debian based machines, so add a raise to avoid this behavior. [YOCTO #10978] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oeqa/core/loader.py: Give meaningful error when failed to load classesMariano Lopez1
With this we get the class that is actually having the problem, not just a TypeError with an unknown class causing the error. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23oeqa/runtime/context.py: Fix use of getTarget() with testexportMariano Lopez1
The idea on getTarget is to use kwargs to send custom variables to different targets, instead of this, a new variable was added (just used for custom targets) and this broke testexport. So in order to fix it, just add the custom variable to kwargs. This fixes the use of getTarget() in testexport class that was introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15oeqa/loader.py: Avoid error in TypeError exceptionJuro Bystricky1
Without this an exception would occur Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15qemu.py: add 'elf' as supported fstypeJuro Bystricky1
Add 'elf' as also supported by OEQemuTarget. Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15oeqa/core/runner.py: OEStreamLogger fix loggingAníbal Limón1
Change conditional to avoid losing stack traces when log into the logger. The logic only needs to don't log empty lines. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31oeqa/core/context.py: Add validation for run-tests optionAníbal Limón1
The run-tests option is optional so if isn't specified set to None instead of crash on split(). Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31meta: remove remaining True option to getVar callsMing Liu1
This is a complementary fix to commit 7c552996: [ meta: remove True option to getVar calls ] it intended to remove all True option to getVar calls, but there are still some remaining. Search made with the following regex: getVar ?\((.*), True\) Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31testimage.bbclass: Allow to run tests on autobuilder's imagesMariano Lopez1
With the change to the new framework data store dependecy was removed, instead a new file is generated and used in testimage. When testing builds from the autobuilders the test data values are from the autobuilder, including the paths. Some tests require paths to current environment in order to run, this commit will update such paths and fix the error of running images donwloaded from autobuilders. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23testimage.bbclass: Add package install featureMariano Lopez1
This allows to use the package install feature with the new OEQA framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/utils/test.py: Add functions to get module pathMariano Lopez1
This will add functions to get module file path from a test case or a complete suite. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23core/target/qemu.py Adds qemu targetMariano Lopez1
This adds qemu target to be used in testimage. It uses the current QemuRunner class in order to boot and control qemu. [YOCTO #10231] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/context: Add option to select tests to runMariano Lopez1
This add the option to select what tests to run in the <module>[.<class>[.<test>]] format. Currently it just support modules Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVarMariano Lopez1
skipIfNotDataVar will skip a test if a variable doesn't have certain value. skipIfNotInDataVar will skip a test if a value is not in a certain variable. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/decorator/data.py: Add skipIfNotFeature decoratorMariano Lopez1
This adds a new decorator to check if image under tests has certain DISTRO_FEATURE or IMAGE_FEATURE. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/target Add OESSHTarget to sent commands to targets using SSHMariano Lopez2
With this commit now it is possible to add targets with SSH for testing. Most of it was imported for existing code, with improvements in log handling. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core: Add READMEAníbal Limón1
The README has an introduction and explains how to run the test suite and creates a new Test component. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/cases: Add example test casesAníbal Limón3
Serves as an first input of how to the OEQA framework works. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/context: Add support of OETestContextExecutorAníbal Limón1
The OETestContextExecutor class supports to use oe-test for run core test component also is a base class for the other test components (runtime, sdk, selftest). Te principal functionality is to support cmdline parsing and execution of OETestContext, the test components could extend the common options to provide specific ones. The common options between test components are test data file, output log and test cases path's to scan. Also it initializes the logger to be passed to the whole OEQA framework. [YOCTO #10230] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core: Add tests for the OEQA frameworkMariano Lopez13
This test suite covers the current functionality for the OEQA framework. For run certain test suite, $ cd meta/lib/oeqa/core/tests $ ./test_data.py Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVarAníbal Limón1
The OETestDataDepends decorator skips a test case if a variable isn't into test data (d). The skipIfDataVar decorator skips a test case if a variable has certain value. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core/decorator: Add support for OETimeout decoratorMariano Lopez1
The OETimeout provides support for specify certain timeout in seconds for a test case, if the timeout is reach the SIGALRM is sent and an exception is raised to notify the timeout. [YOCTO #10235] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/core/decorator: Add support for OETestID and OETestTagMariano Lopez2
These two decorators stores certain TAG or ID for the test case also provides support for filtering in loading step. [YOCTO #10236] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/core/decorator: Add support for OETestDependsAníbal Limón1
The OETestDepends decorator could be used over test cases to define some dependency between them. At loading time sorting the tests to grauntee that a test case executes before also raise an exception if found a circular dependency between test cases. At before test case run reviews if the dependency if meet, in the case of don't it skips the test case run. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core: Add loader, context and decorator modulesAníbal Limón3
loader: Implements OETestLoader handling OETestDecorator and filtering support when load tests. The OETestLoader is responsible to set custom methods, attrs of the OEQA frameowork. [YOCTO #10231] [YOCTO #10317] [YOCTO #10353] decorator: Add base class OETestDecorator to provide a common way to define decorators to be used over OETestCase's, every decorator has a method to be called when loading tests and before test execution starts. Special decorators could be implemented for filter tests on loading phase. context: Provides HIGH level API for loadTests and runTests of certain test component (i.e. runtime, sdk, selftest). [YOCTO #10230] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/core: Add utils module for OEQA frameworkMariano Lopez4
misc: Functions for transform object to other types. path: Functions for path handling. test: Functions for operations related to test cases and suites. [YOCTO #10232] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/core: Add base OEQA frameworkAníbal Limón4
case: Defines OETestCase base class that provides custom methods/attrs defined by the framework. Every OETestCase instance contains a reference to the test data (d), the test context (tc) and the logger. Also implements _oe{SetUp,TearDown}Class for make special handling of OEQA decorators and validations. runner: Defines OETestRunner/OETestResult with support for RAW and XML result logs. exception: Custom exceptions related to the OEQA framework based on class OEQAException. [YOCTO #10230] [YOCTO #10233] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-08 18:34:23 +0000