| Age | Commit message (Collapse) | Author | Files |
|---|
|
Instead of patching and sedding the makefile, just install test/example and
execute it in run-ptest. example is the bulk of the test suite, as minimal as
it is.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Drop remove.ldconfig.call.patch, as it's easier to just set LDCONFIG=true.
Pass uname=GNU via the documented configure option instead of undocumented
environment variable.
Rename zlib-1.2.11/ to just zlib/ as we don't ship multiple versions.
Send ldflags-tests.patch upstream and update Upstream-Status.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python
that was the case with Phytec's BSP, the fix was to replace the do_patch rule with :
do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py
Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Ubuntu's /bin/sh symlinks to /bin/dash by default, so
subprocess.check_call(oe-init-build-env, cwd=builddir) would be failed since
pwd is builddir, and there is no $builddir/oe-init-build-env, this would
lead to other confusing errors, check and error it out earlier to make it
easier to locate the problem.
We don't meet the problem when manually run ". oe-init-build-env" is because
Ubuntu's default login shell is bash, but subprocess.check_call() doesn't
respect to login shell, so the error only happens in situations like
subprocess.check_call().
And also print errors to stderr as oe-buildenv-internal does.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
import time
time.tzset()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'time' has no attribute 'tzset'
enable tzset in both python versions
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Fix up for Thud context ie python3_3.5.6]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
The 2019a release of the tz code and data is available. It reflects the following changes, which were either circulated on the tz mailing list or are relatively minor technical or administrative changes:
Briefly:
Palestine "springs forward" on 2019-03-30 instead of 2019-03-23.
Metlakatla "fell back" to rejoin Alaska Time on 2019-01-20 at 02:00.
Changes to past and future timestamps
Palestine will not start DST until 2019-03-30, instead of 2019-03-23 as
previously predicted. Adjust our prediction by guessing that spring
transitions will be between 24 and 30 March, which matches recent practice
since 2016. (Thanks to Even Scharning and Tim Parenti.)
Metlakatla ended its observance of Pacific standard time,
rejoining Alaska Time, on 2019-01-20 at 02:00. (Thanks to Ryan
Stanley and Tim Parenti.)
Changes to past timestamps
Israel observed DST in 1980 (08-02/09-13) and 1984 (05-05/08-25).
(Thanks to Alois Treindl and Isaac Starkman.)
Changes to time zone abbreviations
Etc/UCT is now a backward-compatibility link to Etc/UTC, instead
of being a separate zone that generates the abbreviation "UCT",
which nowadays is typically a typo. (Problem reported by Isiah
Meadows.)
Changes to code
zic now has an -r option to limit the time range of output data.
For example, 'zic -r @1000000000' limits the output data to
timestamps starting 1000000000 seconds after the Epoch.
This helps shrink output size and can be useful for applications
not needing the full timestamp history, such as TZDIST truncation;
see Internet RFC 8536 section 5.1. (Inspired by a feature request
from Christopher Wong, helped along by bug reports from Wong and
from Tim Parenti.)
Changes to documentation
Mention Internet RFC 8536 (February 2019), which documents TZif.
tz-link.html now cites tzdata-meta
<https://tzdata-meta.timtimeonline.com/>.
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
move tzcode and tzdata-native into new dir
Updates are now limited to the common include file.
[v2]
Fix UMMARY typ0 in timezone.inc
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[packages stayed the same, align for future updates]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
License-Update: copyright years updated
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Whether the AFALG engine (use of hardware crypto via AF_ALG) is enable or
disable depends on whether the host kernel is 4.1 or above, which has no bearing
on whether the target system supports it.
Remove the complicated logic and simply enable/disable as requested.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
The old bsd cryptodev engine was removed in
https://github.com/openssl/openssl/pull/3699
and the new one added in:
https://github.com/openssl/openssl/pull/3744
It can be enabled by configuring with "enable-devcryptoeng".
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
It is only needed by 95-test_external_pyca_data which is
actually skipped on the target.
[YOCTO #13204]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
openssl-ptest was recording now results, despite most tests passing. Fix
so that the successes/skips/failures are reported correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
These patches were applied, hoping that they will eventually be accepted
upstream but they have been rejected, I think its best that they are
dropped so we can avoid novel unintended behaviours that no other
distros will be seeing
(From OE-Core rev:54550aa42378ce4b215bccbfd95e5e650b0d2efa)
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Sometimes we meet the following failure for the test_lookup_recipe
test case.
AssertionError: 'zlib\nbusybox is in the RPROVIDES of target-sdk-provid[32 chars]ummy' != 'zlib\nbusybox'
zlib
+ busybox- busybox is in the RPROVIDES of target-sdk-provides-dummy:
- target-sdk-provides-dummy
This is because target-sdk-provides-dummy rprovides busybox.
So clean things up to avoid failure.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Simplify and removed unnecessary codes.
Refactor to allow pythonic loop.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Currently the manual execution display step by sorting
the step as string, where steps were not being sorted
correctly when there are more than 9 steps.
Fixed the step sorting by sorting step as integer.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Current manualexecution required pressing enter button to show each step
information, where this was wasting execution time. Enable display
full steps without needing to any press enter button.
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Current input checking does not match the standard input practiced
by QA team. Change the input checking to match the standard
input practiced by the QA team.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Rather than simply discarding the ptest data, change the code to discard
it when writing out the new testresult files, or optionally either preserve
it, or write it as seperate discrete logs.
This means the autobuilder should start writing out individual ptest log
files as well as allowing ueers to extract these manually.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Store operations using a single file as a source weren't working as the os.walk
command didn't like being given a single file. Fix the store operation to
work for single files.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
All test id (eg. @alias) inside manual testcase file shall follow the same
test id naming convention from oeqa automated tests (eg. selftest,
runtime, sdk, etc), where the test id consists of
<test_module>.<test_suite>.<test_function>. Furthermore, there shall be
only 1 unique test_module per each manual testcases file, where
test_module match the file name itself.
This file was using test_module name that does not match the file name
itself. Fixed test_module name as well as the test_suite name.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
As part of the solution to replace Testopia, manual test cases
need to be migrated to OEQA. These manual test case json files
will serve two use cases.
Use case#1: as input to the future commandline-based manual test
runner script, where this script will display actions and expected
result information in user friendly text, then it will capture
user input for test result and log, finally it will write test
result and log into existing standardize test result json format
from OEQA framework for automated tests.
Use case#2: QA will open and read these json file manually for
planning manual test execution. Any reader interested in
understanding manual test cases will open and read these files.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
All test id (eg. @alias) inside manual testcase file shall follow the same test id
naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where
the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore,
there shall be only 1 unique test_module per each manual testcases file.
This file was using more than 1 unique test_module for testcases. Furthermore,
some of the testcases were defined using different test_suite where it was not needed.
This patch fix the manual testcases file to have only 1 unique test_module
as well as test_suite to simplify test id naming.
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Two changes made in oeqa/manual/compliance-test.json:
1. All test id (eg. @alias) inside manual testcase file shall follow the same test id
naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where
the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore,
there shall be only 1 unique test_module per each manual testcases file.
This file was using more than 1 unique test_module for testcases. Furthermore, some of
the testcases were defined using different test_suite where it was not needed. This patch
fix the manual testcases file to have only 1 unique test_module as well as test_suite to
simplify test id naming.
2. Added 2 missing test cases for Beaglebone Stress Test.
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Two changes made in oeqa/manual/bsp-hw.json:
1. All test id (eg. @alias) inside manual testcase file shall follow the same test id naming
convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where the
test id consists of <test_module>.<test_suite>.<test_function>. Furthermore,
there shall be only 1 unique test_module per each manual testcases file.
This file was using more than 1 unique test_module for testcases.
Furthermore, some of the testcases were defined using different test_suite where it was not
needed. This patch fix the manual testcases file to have only 1 unique
test_module as well as test_suite to simplify test id naming.
2. As per review by Intel and Windriver team, 7 testcases were found obsolete. Removed 7 testcases.
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
QA team found that 10 manual bsp test cases from Testopia for
Beaglebone, EdgeRouter, and MPC need to be up streamed
[YOCTO #12650]
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Manual test step for SDK is updated. Previously toolchain was "poky-glibc-x86_64-core-image-sato-sdk<type-arch>-toolchain-<release-version>.sh"
But toochain for core-image-sato-sdk is not available in releases after 2.1, hence changed it to "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh
Other less significant update is to add exepcted result in intermediate test steps.
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
"poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh"
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Added json file for compliance test cases to the manual runtime tests
Signed-off-by: Sudhir Sharma <sudhirx.sharma@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
This reverts commit a384248938ea9db096866bf4ec8678d35ca62a12.
This package update slipped in doing the maint process. Removing it.
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This supports glibc 2.29 which is appearing in distros like Ubuntu 19.04
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid uninative checksum warnings when building on aarch64 hardware.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst its tempting to call bitbake_main(), the script also directly executes
bitbake. By doing both this creates all kinds of races. Therefore
change to use subprocess execution in all cases.
(From OE-Core rev: 0507ec4c6069490c9a503e9e2b65f0e7b1962d34)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The parent code currently assumed that any parent branch is locally
checked out which may not be the case.
Use the local branch by default but fall back to the origin. This
also means removing the later saftey check as the branch may not exist
locally.
This fixes the autobuilder resulttool test pushing code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We found that manualexecution does not capture test suite values
correctly if there are more than one test suite in test cases.
After verification has made we found out we should retrieved
full test cases value <test_module.test_suite.test_case> from
oeqa/manual/ json file rather than split it them into new
variables test_suite and test_cases.
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable roll-up all test results belong to a commit
and to provide a roll-up report.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To enable store for testresults.json file from manualexecution,
add layers metadata to configuration and add "manual" map to
resultutils.store_map.
To enable regression for manual, add "manual" map to
resultutils.regression_map. Also added compulsory configurations
('MACHINE', 'IMAGE_BASENAME') to manualexecution.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sorted regression results to provide friendly viewing of report.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
ptest suites with no results don't show up on the reports even though we have
a duration for them. Fix this so the fact they report no tests is visible.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently some older results files cause the code to give tracebacks.
Handle these missing sections more cleanly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently we cant store results if the results files span multiple
different build revisons. Remove this limitation by iterating.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is a combined patch of the various tweaks and improvements I
made to resulttool:
* Avoid subprocess.run() as its a python 3.6 feature and we
have autobuilder workers with 3.5.
* Avoid python keywords as variable names
* Simplify dict accesses using .get()
* Rename resultsutils -> resultutils to match the resultstool ->
resulttool rename
* Formalised the handling of "file_name" to "TESTSERIES" which the code
will now add into the json configuration data if its not present, based
on the directory name.
* When we don't have failed test cases, print something saying so
instead of an empty table
* Tweak the table headers in the report to be more readable (reference
"Test Series" instead if file_id and ID instead of results_id)
* Improve/simplify the max string length handling
* Merge the counts and percentage data into one table in the report
since printing two reports of the same data confuses the user
* Removed the confusing header in the regression report
* Show matches, then regressions, then unmatched runs in the regression
report, also remove chatting unneeded output
* Try harder to "pair" up matching configurations to reduce noise in
the regressions report
* Abstracted the "mapping" table concept used to pairing in the
regression code to general code in resultutils
* Created multiple mappings for results analysis, results storage and
'flattening' results data in a merge
* Simplify the merge command to take a source and a destination,
letting the destination be a directory or a file, removing the need for
an output directory parameter
* Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression
mappings
* Have the store command place the testresults files in a layout from
the mapping, making commits into the git repo for results storage more
useful for simple comparison purposes
* Set the oe-git-archive tag format appropriately for oeqa results
storage (and simplify the commit messages closer to their defaults)
* Fix oe-git-archive to use the commit/branch data from the results file
* Cleaned up the command option help to match other changes
* Follow the model of git branch/tag processing used by oe-build-perf-report
and use that to read the data using git show to avoid branch change
* Add ptest summary to the report command
* Update the tests to match the above changes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution <manualjsonfile>
By default testresults.json store in <build_dir>/tmp/log/manual/
[YOCTO #12651]
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
