summaryrefslogtreecommitdiff
path: root/meta/lib/oeqa
AgeCommit message (Collapse)AuthorFiles
2018-06-04oeqa/core/target/ssh.py: increase maximum read bytes from 1024 to 4096Chen Qi1
When running testimage task for core-image-sato-sdk, the following error appeared. UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 at position 0: invalid start byte Checking the codes, I found it's caused by setting a 1024 limit for the read method of the StreamReader object. Comments from the manual: """ The chars argument indicates the number of decoded code points or bytes to return. The read() method will never return more data than requested, but it might return less, if there is not enough available. """ When running `systemctl status --full' on target, this error occurs. This patch increase the bytes limit to 4096 to fix the error. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logicChen Qi1
The test_file_connman should be executed only when 'lib32-connman' is installed and 'connman' is not installed. When lib32-connman and connman are both installed, the /usr/sbin/connmand could be from connman or lib32-connman, depending on the installation order. What we want to check is the connmand command from lib32-connman, so we need to make sure that connman is not there to cause chaos. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/runtime/cases/multilib.py: skip if needed packages are not availableChen Qi1
1) The test cases use 'readelf' command to do the check. This command is from binutils. So skip the test if the needed binutils package is not installed. The related error message in log.do_testimage is like below. Output: sh: readelf: not found 2) The test case tests /lib/libc.so.6 from lib32-libc6. So skip the test if lib32-libc6 is not installed. The related error message in log.do_testimage is like below. Output: readelf: Error: 'lib/libc.so.6': No such file Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/runtime/cases/rpm.py: skip if rpm not availableChen Qi1
This test case should only run when rpm package is installed. So skip it if rpm package is not installed. This fixes: RESULTS - rpm.RpmBasicTest.test_rpm_help - Testcase 1059: FAILED Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/core/decorator/data.py: fix skipIfNotInDataVarChen Qi1
The var might not be set, resulting in unexpected error. RESULTS - multilib.MultilibTest.test_check_multilib_libc - Testcase 1593: ERROR The above error is due to MULTILIBS being not set, which is the default for OE. This patch fixes this problem. Also, the debugging message in skipIfNotInDataVar is currently confusing. Instead of DEBUG: Checking if 'MULTILIBS' value is in 'multilib:lib32' to run the test it should be DEBUG: Checking if 'MULTILIBS' value contains 'multilib:lib32' to run the test This patch also fixes it. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/core/decorator/__init__.py: use 'cls' instead of 'obj'Chen Qi1
Use 'cls' instead of 'obj' to better reflect that registerDecorator actually serves as a class decorator. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/core/decorator/__init__.py: set metaclass to ABCMetaChen Qi1
OETestFilter is a subclass of OETestDecorator. It wants to make use of @abstractmethod decorator. But such decorator requires metaclass to be ABCMeta to have effect. So add it now to achieve the designed behaviour. Comments from python's manual: """ Using this decorator requires that the class's metaclass is ABCMeta or is derived from it. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/core/loader.py: support the 'auto' keywordChen Qi1
In previous OEQA, having 'auto' in TEST_SUITES results in executing as many test cases as possible. This behaviour is broken for now. From the codes in core/loader.py, I can see that it tries to use another keyword 'all'. But in fact, it does not work. I've checked the current manual. The manual says using 'auto'. Below is the current information in manual. """ Alternatively, you can provide the "auto" option to have all applicable tests run against the image. TEST_SUITES_append = " auto" """ So we should restore this behaviour. This patch does so. Also, output warning message is some module is named as 'auto', as this is a reserved keyword. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-29oeqa/sdk/cases/buildgalculator.py: skip if gettext not availableChen Qi1
We need to skip this testcase when gettext is not available. Otherwise, we will have the following error at configure. error: possibly undefined macro: AM_NLS Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-05-14runtime/dnf: Add new dnf test casesJose Perez Carranza1
Add test cases to test “exclude” and “installroot“ options, also modify the logic of filtering packages on the feed to have all the packages needed by the tests. [YOCTO #10744] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-11oeqa/selftest/case: fix typoHongxu Jia1
s/meta-sefltest/meta-selftest/g Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-05-04oeqa/runtime/cases/python: use python 3 rather than python 2Alexander Kanavin1
For example, core-image-sato skipped the test alltogether, as it no longer pulls in Python 2.x at all. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04btrfs-tools: update to 4.15.1Alexander Kanavin1
Drop upstreamed 0001-Fix-build-with-musl-missing-header-include-for-dev_t.patch Add ftw-subdir-walk.patch as it resolves the RECIPE_NO_UPDATE_REASON. Add --disable-zstd as libzstd isn't provided in oe-core. Fix wic testcase, as the minimal fs size is now bigger. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04logrotate: update to 3.14.0Yi Zhao1
Since the wtmp and btmp definitions had been moved from logrotate.conf to logrotate.d in this release, we also need to install them to /etc/logrotate.d/. Also update oeqa runtime logrotate test case. Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-20oeqa/selftest/runqemu: qemu nfs testing not ready for deploymentRichard Purdie1
This test shouldn't have merged yet since we don't run portmap/rpcbind on the autobuilder infrastructure and the test therefore cannot succeed. We need to document this, set it up, then enable the test. The test itself is fine and good to have so its left in the code but disabled for now. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-19oe-selftest: runqemu: add tests for qemu boot and shutdownYeoh Ee Peng1
QA team were testing qemu boot image and shutdown on each qemu architecture manually. Add automated test to test qemu boot on ext4 and nfs, finally check that it can shutdown properly. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-04-18oeqa/runtime/stap.py: add runtime test for systemtapVictor Kamensky2
Add runtime test for stap to test basic SystemTap operations: can compile very basic module and run on target device. Note we disable (-DSTP_NO_VERREL_CHECK) SystemTap additional kernel release check since during OE testing mismatching kernel-devsrc and kernels are used. Signed-off-by: Victor Kamensky <kamensky@cisco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-04-07devtool: Ensure added layer sets LAYERSERIES_COMPATRichard Purdie1
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool needs to set this to avoid warnings which break various tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-07devtool/oeqa: Ensure added layers set LAYERSERIES_COMPATRichard Purdie1
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool/oeqa needs to set this to avoid warnings which break various tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30make-mod-scripts/kernel-devsrc: Fix objtool issuesRichard Purdie1
Kernels which use tools/objtool can now fail when building external modules due to objtool being missing, the generated files can also cause problems for kernel-devsrc. Ensure objtool is generated in make-mod-scripts by also calling "make prepare". For devsrc, delete the generated binaries since they'd be native binaries and unsuitable for the target. The oeqa kernel module tests also need to have the additional "make prepare" step added. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-16buildperf: measure the size of core-image-sato rootfsRoss Burton1
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-12oe-selftest: add a test for failing package post-installation scriptletsAlexander Kanavin1
The test runs a scriptlet that has an intentionally failing command in the middle and checks for two things: 1) that bitbake does warn the user about the failure 2) that scriptlet execution stops at that point. The test is run for all three package types: rpm, deb, ipk. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-06selftests: Add test case for booting a generic EFI boot partition imageCalifornia Sullivan1
Simple test case that adds 'efi' to MACHINE_FEATURES, sets WKS_FILE to "efi-bootdisk.wks.in", installed required boot items, and attempts to boot the wic image. Quick check to make sure that the feature actually works. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04oeqa/runtime/opkg.py: add runtime test for opkgYeoh Ee Peng1
Add runtime test for opkg to test that it can install ipk package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-03-04oeqa/runtime/apt.py: add runtime test for aptYeoh Ee Peng1
Add runtime test for apt to test that it can install deb package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-27package_manager: Filter to only rpms we depend uponRichard Purdie1
Currently do_rootfs gets to see all rpms in the deploy directory. This filters that view to only rpms which the image recipe has actual depends upon which potentially removes some sources of confusion in the image construction. This makes builds more reproducibile and also fixes contamination issues where dnf picks up packages it shouldn't be able to 'see'. [YOCTO #12039] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-24sstatetests: pass the right files when comparing sigfilesRoss Burton1
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: add a test for recipes without maintainersAlexander Kanavin1
'bitbake -c checkpkg world' is moved to class initializer to avoid it being run twice in a row. The no-maintainers test checks only oe-core recipes, as other layers may be be configured, and assigning maintainership to specific people via maintainers.inc is known to be used only in oe-core. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: fix tests to reflect man->man-db transitionAlexander Kanavin1
Rather than apply a patch twice, an incorrect patch is applied (as the new man-db recipe does not have any patches yet). Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-20oeqa/buildperf/base: Fix fetchall reference to use runall bitbake optionRichard Purdie1
The fetchall task was removed, use its replacement bitbake option. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-09oe-selftest: meta_ide: add tests for meta-ide-supportYeoh Ee Peng1
QA team were testing meta-ide-support manually. Add automated tests to test that bibtake meta-ide-support will create the toolchain and environment setup script. Also test that after using environment setup script, one can compile c program and build cpio project. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29oeqa/runtime: add gobject-introspection testRoss Burton1
(From OE-Core rev: 497a95b576e19e39e20ac280d0db24f51b7c9679) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-23imagefeatures: disable f2fs from test_image_features by defaultSaul Wold1
Since the primary f2fs utilities are provided by the meta-openembedded meta-filesystems layer, we disable the testing of that functionality here. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-14selftest/cases/devtool.py: fix workspace layer checkingRobert Yang1
Fixed: $ oe-selftest -r devtool.DevtoolTests.test_create_workspace [snip] 2017-12-20 18:28:59,404 - oe-selftest - INFO - Traceback (most recent call last): File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/workspace2/lyang1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 177, in test_create_workspace self.assertTrue('/workspace' not in result.output, 'This test cannot be run with a workspace layer in bblayers.conf') AssertionError: False is not true : This test cannot be run with a workspace layer in bblayers.conf [snip] $ bitbake-layers show-layers NOTE: Starting bitbake server... layer path priority ========================================================================== meta /workspace2/lyang1/poky/meta 5 meta-poky /workspace2/lyang1/poky/meta-poky 5 meta-yocto-bsp /workspace2/lyang1/poky/meta-yocto-bsp 5 meta-selftest /workspace2/lyang1/poky/meta-selftest 5 There is no workspace layer, but I'm in /workspace2, this patch can fix the problem. [YOCTO #12442] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-06bitbake.conf: Set AUTOREV to have a vardepvalueRichard Purdie1
If you have a recipe which does not include SRCPV in PV but does set SRCREV = "${AUTOREV}" and you run do_fetch, then change the repo to a new commit then run do_unpack, do_unpack will fail since the new commit doesn't exist in the repo that was fetched. The problem is the revision chosen is not represented in the do_fetch task hash. It if were, the fetch would rerun first and the commit would be present. It works when PV includes SRCPV since that does contain the chosen commit from the AUTOREV. The solution is to include the SRCPV value into the representation of AUTOREV used for checksum calculation purposes. Add a selftest for this issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06oeqa/runtime/buildcpio: Use our own mirror for sourceRichard Purdie1
We see occasional network glitches which break this test. Use our own mirror (which has a .gz instead of .bz2) to avoid the errors, we're not trying to test network connectivity. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: fail when ptests fail on targetAlexander Kanavin1
That's the whole point isn't it? Previously this testcase succeeded even if some of the underlying on-target tests failed; the only way to find out if anything was wrong was to manually inspect the logs. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run ↵Alexander Kanavin1
only when ptest-runner is availalble; Previously the test would execute only when all available ptests for packages in the image were installed; some of those tests may be broken, never finish, take a very long time or simply irrelevant to the user who wants to check ptests of only a few specific packages, and does so by listing them explicitly via IMAGE_INSTALL_append or similar. Presence of ptest-runner means there is at least one ptest package installed as they pull it in via a class dependency; ptest-runner is not generally installed otherwise. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-04core/loader.py: fix regex to include all available test casesLeonardo Sandoval1
Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all. [YOCTO #12385] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-02oeqa/qemu: remove elf image typeSaul Wold1
Since we are depercating the ELF image type, we should not test it here either. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-02oe-selftest: devtool: switch away from mkelfimagePaul Eggleton1
mkelfimage is about to be removed, so test_devtool_modify_git needs to use a different recipe. psplash is a reasonable choice given it uses a git repository (and probably will forever), and doesn't have too many dependencies, so change the test to use that recipe instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-21rpm: update to 4.14.0Alexander Kanavin1
Previously oe-core had a development snapshot of rpm, it's better to update to something more stable. Removed patches: 0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch (upstream is using pkg-config) 0001-When-nice-value-cannot-be-reset-issue-a-notice-inste.patch (functionality has been moved to a plugin, we disable plugins for rpm-native) 0012-Use-conditional-to-access-_docdir-in-macros.in.patch (merged upstream) Changed patches: 0001-Fix-build-with-musl-C-library.patch (one previous musl issue has been resolved upstream; another has been added) Rest of the patches are trivial rebases. Update the signing oe-selftest so that the reference output matches the upstream changes. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-21oeqa/selftest/cases/wic: update OETestID decorator for test_qemuJoshua Lock1
Test case ID 1422 is not correct for this issue, the correct test case ID for test_qemu is 1424. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-20oeqa/selftest/buildoptions: Add test for fortran compiler buildingRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-20oeqa/selftest/archiver: Add test for srpm archiver mode dependenciesRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-18meta-selftest: wic: Add test for --use-uuid / --fsuuidTom Rini1
- Mount a '/media' partition to wic-image-minimal.wks with a known UUID. - In test_qemu, sort our output from checking the output of 'mount' as it may not be stable. Also, do not check the exit code as passing any output to cut ensures a 0 exit code. - Check for a 'UUID=' line in /etc/fstab with out expected output. Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-18targetcontrol.py: use oe.types.boolean for QEMU_USE_KVMRobert Yang1
So that both QEMU_USE_KVM = "True" and "1" will work. [YOCTO #12343] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10oe-selftest: devtool: fix test for changes in lzo recipePaul Eggleton1
acinclude.m4 is about to be removed from the lzo recipe which breaks test_devtool_update_recipe_local_files_2. Create a synthetic recipe in meta-selftest with some local files and use that instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10utils: qemurunner.py: cleanup grammar and ensure consistencyMark Asselstine1
Minor grammar correction along with making the term 'login banner' consistent throughout to make searching logs easier. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10utils: qemurunner.py: Log both 'failed to reach login banner" reasonsMark Asselstine1
The current logging always assumes the boot timeout has expired yet there is a second reason we might have ended up in a position where no login banner was found, that being a socket disconnect. Add logging for the disconnect case and make the timeout expiration conditional on the timeout being exhausted. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>