| Age | Commit message (Collapse) | Author | Files |
|---|
|
We now have a function in cooker itself that can do this lookup;
additionally, the rewritten tinfoil's cooker adapter has its own
implementation that can work remotely, so if we use it then this
function can work in that scenario as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
through BitBake
If you printed a warning through bb.warn() / bbwarn or an error through
bb.error() / bberror, this was also being picked up by our log_check
mechanism that was designed to pick up warnings and errors printed by
other programs used during do_rootfs. This meant you saw not only the
warning or error itself, you saw it a second time through log_check,
which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the
logger and add a handler that we can use to find out if any warning or
error we find in the logs is one we should ignore as it has already been
printed.
Fixes [YOCTO #8223].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some variables in pkgdata files have a package-name override. When
the bare variable can not be found, try with the override-variant.
PKGSIZE is one such variable, and already had special code to handle this.
Test included.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The Joule is very new hardware and there is ongoing kernel and firmware
work to fix these issues, which will be available in future kernel and
firmware releases. In the meantime, don't clog QA reports.
[YOCTO #10611]
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some virtualized environments like Linux-VServer do not have the
entries under /proc that the new system usage sampling expected,
leading to an exception when trying to open the files.
Now the presence of these files is checked once before enabling the
corresponding data collection. When a file is missing, the
corresponding log file is not written either and pybootchart will not
draw the chart that normally displays the data.
Errors while reading or writing of data samples is intentionally still
a fatal error, because that points towards a bigger problem that
should not be ignored.
Reported-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Test that recipetool plugins are loaded in a well defined order.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Test that devtool plugins are loaded in a well defined order.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Adds functions to get metadata from the host running the tests.
[YOCTO #9954]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The function get_bb_vars will remove items for the list passed
as the function argument, this will leave the caller with an
empty list and the function never says it will consume the items.
This hasn't been found before because only get_bb_var uses this
function.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
ClassType was removed from python3.
The code testing for ClassType kept throwing AttributeError exceptions:
module 'types' has no attribute 'ClassType'
The exceptions prevented loading of any dynamically resolved target
controllers.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using subprocess call and check_output, it is better to use arrays
rather than strings when possible to avoid whitespace and quoting
problems.
[ YOCTO #9342 ]
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Setting WKS_FILE variable in qemux86-64 made wic test to
use wrong wks file to produce an image and resulted in
test_qemu failure.
Used conditional assignment in qemux86-64 and explicitly
set WKS_FILE in wic testing suite to make the suite to use
wic-image-minimal.wsk. This should fix test_qemu failure.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.
The disk monitor log added another 16KB in that example build. The
overall buildstat was 20MB, so the overhead for monitoring system
utilization is small enough that it can be enabled by default.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Hooks into the new monitordisk.py event and records the used space for
each volume. That is probably the only relevant value when it comes to
visualizing the build and recording more would only increase disk
usage.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
/proc/[diskstats|meminfo|stat] get sampled and written to the same
proc_<filename>.log files as during normal bootchat logging. This will
allow rendering the CPU, disk and memory usage charts.
Right now sampling happens once a second, triggered by the heartbeat
event.That produces quite a bit of data for long builds, which will be
addressed in a separate commit by storing the data in a more compact
form.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The warning occurs when the GPT image is not the same size than the
media into which it's being flashed, causing the backup GPT table
not being at the end of the disk. However, this is expected as the
image is created before having the information about the destination
media. The error is harmless, so it will be whitelisted.
Fixes [YOCTO 10481].
Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The iwlwifi module of any given kernel has a minimum and maximum
supported firmware version. The kernel begins by attempting to load the
maximum version, and decrements until it is successful. The 4.8 kernel's
maximum supported firmware version is 24, but thus far only 22 has been
released, meaning we get errors for 24 and 23.
Filter out iwlwifi firmware load error messages, as they are not
necessarily indicative of real problems.
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Based on run() in bitbake/lib/bb/process.py, ExecutionError() expects strings
not bytes. Passing bytes results in a "TypeError: Can't convert 'bytes' object
to str implicitly" exception.
Fixes Bug 10729
Signed-off-by: Martin Vuille <jpmv27@yahoo.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Update test case numbers on runtime tests to do match
with templates defined on Testopia for 2.3 release
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This test has been ported to be run as part of Toaster's own tests.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This was already whitelisted, but the 4.8 kernel changed the error
message, causing it to get caught by parselogs again.
Fixes [YOCTO #10494].
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some rolling release distros, such as Arch Linux, don't include a
VERSION_ID field in their os-release file.
Change release_dict_osr() to better handle this optional field
being absent.
Further improve the resilience of the release_dict_*() methods by
always returning a dict and using dict.get() in distro_identifier()
to supply a default, empty string, value when then key is missing.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.
I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.
Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.
Fixes [YOCTO #10616].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When attempting to run devshell, if no terminal is available, the
error being thrown was not very specific. This adds a list of
commands that failed, informing the user of what they can install to
fix the error.
[ YOCTO #10472]
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you have a file:// entry in SRC_URI with a subdir= parameter that
makes it extract into the source tree, then when you update that file in
oe-local-files and run devtool update-recipe then you want the original
file to be updated. This was made to work by OE-Core commit
9069fef5dad5a873c8a8f720f7bcbc7625556309 together with
31f1bbad248c36a8c86dde4ff57ce42efc664082, however until now there was no
oe-selftest test to verify it.
Note that in order to succeed this test also requires the fix
"lib/oe/recipeutils: ignore archives by default in
get_recipe_local_files()" since the test recipe uses a local tarball.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
By default, have get_recipe_local_files() not return any archive
files. This prevents a local tarball from being erroneously removed
from SRC_URI if you run "devtool modify" on a recipe followed by
"devtool update-recipe". It doesn't actually help you to directly
update the contents of such tarballs, but at least now it won't break
the recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.
Additionally, add an oe-selftest test to ensure this doesn't regress in
future.
Fixes [YOCTO #8278].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If a patch applied by a recipe has no header and we turn the recipe's
source into a git tree (when PATCHTOOL = "git" or when using devtool
extract / modify / upgrade), the commit message ends up consisting only
of the original filename marker ("%% original patch: filename.patch").
When we come to do turn the commits back into a set of patches in
extractPatches(), this first line ends up in the "Subject: " part of
the file, but we were ignoring it because the line didn't start with the
marker text. The end result was we weren't able to get the original
patch name. Strip off any "Subject [PATCH x/y]" part before looking for
the marker text to fix.
This caused "devtool modify openssl" followed by "devtool update-recipe
openssl" (without any changes in-between) to remove version-script.patch
because that patch has no header and we weren't able to determine the
original filename.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Add a test to ensure devtool update-recipe works properly on recipes
that contain only local files (since the other tests we have didn't test
that).
Relates to [YOCTO #10563].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The debian policy manual and MaintainerScripts wiki page states that the
postinst script is supposed to be called with the `configure` argument
at first install, likewise the preinst script is supposed to be called
with the `install` argument on first install.
https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
https://wiki.debian.org/MaintainerScripts
Signed-off-by: Linus Wallgren <linus.wallgren@scypho.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The LSB Distributor ID and os-release NAME differ for most of the
distributions tested by the Yocto Project (CentOS, Debian, Fedora,
openSUSE and Ubuntu) however for all but openSUSE the os-release ID
matches the LSB Distributor ID when both are lowered before
comparison.
Therefore, in order to improve the consistency of identification of
a distribution, switch to using the os-release ID and converting
the ID value to lowercase.
Table showing comparison of LSB Distributor ID to os-release fields NAME
and ID for current Yocto Project supported host distributions:
Distribution | Version | Distributor ID | NAME | ID |
-------------------------------------------------------------------------
CentOS | 7 | CentOS | CentOS Linux | centos |
Debian | 8 | Debian | Debian GNU/Linux | debian |
Fedora | 23 | Fedora | Fedora | fedora |
Fedora | 24 | Fedora | Fedora | fedora |
openSUSE | 13.2 | openSUSE project | openSUSE | opensuse |
openSUSE | 42.1 | SUSE LINUX | openSUSE Leap | opensuse |
Ubuntu | 14.04 | Ubuntu | Ubuntu | ubuntu |
Ubuntu | 16.04 | Ubuntu | Ubuntu | ubuntu |
[YOCTO #10591]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
os-release(5) is an increasingly standard source of operating system
identification and more likely to be present on modern OS deployments, i.e.
many container variants of common distros include os-release and not the
lsb_release tool.
Therefore we should favour parsing /etc/os-release in distro_identifier(),
try lsb_release when that fails and finally fall back on various distro
specific sources of OS identification.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Rather than have the distro_identifier method look for different keys in
the dict depending on the source ensure that each function for retrieving
release data uses the same key names in the returned dict.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The ELF parser was assuming that the segment tables are in the first 4kb of the
binary. Whilst this generally appears to be the case, there have been instances
where the segment table is elsewhere (offset 2MB, in this sample I have). Solve
this problem by mmap()ing the file instead.
Also clean up the code a little whilst chasing the problem.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
There's no need to import glob inside copyhardlinktree() as it's
already imported for the entire path module.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Instead of checking against a file that represents a distribution that hasn't
existed for years, fetch package names for Clear Linux instead.
[ YOCTO #10601 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
breaking on selftest
NOTE: recipe core-image-minimal-1.0-r0: task do_rootfs: Started
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:license_create_manifest(d)
0003:
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 48, function: license_create_manifest
0044: pkg_dic = {}
0045: for pkg in sorted(image_list_installed_packages(d)):
0046: pkg_info = os.path.join(d.getVar('PKGDATA_DIR', True),
0047: 'runtime-reverse', pkg)
*** 0048: pkg_name = os.path.basename(os.readlink(pkg_info))
0049:
0050: pkg_dic[pkg_name] = oe.packagedata.read_pkgdatafile(pkg_info)
0051: if not "LICENSE" in pkg_dic[pkg_name].keys():
0052: pkg_lic_name = "LICENSE_" + pkg_name
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/sysroots/qemux86-64/pkgdata/runtime-reverse/kernel-4.8.3-yocto-standard'
This reverts commit c3d2df883a9d6d5036277114339673656d89a728.
|
|
When you run devtool add on a source tree we attempt to figure out the
correct name and version for the recipe. However, despite our best
efforts, sometimes the name and/or version we come up with isn't
correct, and the only way to remedy that up until now was to reset the
recipe, delete the source tree and start again, specifying the name this
time. To avoid this slightly painful procedure, add a "rename"
subcommand that lets you rename the recipe and/or change the version.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If a variable is being set in the recipe when we've explicitly passed
None as the value to _test_recipe_contents() indicating that it
shouldn't be set at all, then we should be printing out the variable
name in the assertion message but it seems like I forgot to do a
substitution. Also include the value for informational purposes.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
recipetool sets the LICENSE value based on licenses detected from the
source tree. If there are multiple licenses then they were being
separated by spaces, but this isn't actually legal formatting and if
you're using "devtool add" you get a warning printed when devtool
parses the recipe internally.
Earlier I had made a conscious decision to do it this way since it's up
to the user to figure out whether the multiple licenses should all apply
(in which case they'd be separated with &) or if there is a choice of
license (in which case | is the correct separator). However, I've come
to the conclusion that we can just default to & and then the ugly
warning goes away, and it's the safest alternative of the two (and most
likely to be correct, since it's more common to have a codebase which is
made up of code with different licenses, i.e. all of them apply to the
combined work).
I've tweaked the comment that we add to the recipe to explicitly state
that we've used & and that the user needs to change that if that's not
accurate.
Fixes [YOCTO #10413].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This library suffered as part of the Python 2 to Python 3 migration and stopped
working entirely.
Fix all the migration problems such as files being treated as strings but opened
in binary mode, insufficient use of with on files, and so on.
Rewrite large amounts to be Pythonic instead of C-in-Python.
Update OpenSuse and Fedora URLs.
Fedora now splits the archive alphabetically so handle that.
[ YOCTO #10562 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you use devtool update-recipe with the --append option, and a "local"
(in oe-local-files) has been modified we copy it into the specified
destination layer. With the way the devtool update-recipe code works now
the source is always a temp directory, and printing paths from within
that is just confusing, so if the path starts with the temp directory
then just print the file name alone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Do not use --force-depends when trying to remove all dependent packages,
as it removes only the selected package and not the dependent packages.
Signed-off-by: Samuli Piippo <samuli.piippo@qt.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When there is a relative symlink in the layer, for example:
symA -> ../out/of/layer/file
symA will be invalid fater copied, it would be invalid from build time
if it points to a relative path, and would be invalid after extracted
the sdk if it points to a absolute py. Dereference symlink when copy
will fix the problem.
Use tar rather than shutil.copytree() to copy is because:
1) shutil.copytree(symlinks=Fasle) has bugs when dereference symlinks:
https://bugs.python.org/issue21697
And Ubunutu 1404 doesn't upgrade python3 to fix the problem.
2) shutil.copytree(symlinks=False) raises errors when there is a invalid
symlink, and tar just prints a warning, tar is preferred here since
the real world is unpredicatable
3) tar is faster than shutil.copytree() as said by oe.path.copytree()
So use tar to copy.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
[YP#7202]: Test for linux-dummy
The new kernel.py file is intended for kernel related test cases.
The test for linux-dummy will ensure it is in good shape and can
be used as a kernel replacement at build time. To do this, the
test will first clean sstate for linux-dummy target, ensuring no
file is present in the stamps directory. After, core-image-minimal
is built, ensuring linux-dummy can be used as a kernel substitute.
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Konsole has dropped support for the nofork flag. It has been replaced with the seperate flag.
Signed-off-by: Michael Davis <michael.davis@essvote.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since the move to put image deployment under sstate control in
d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically
removed before a new image is deployed (the default behaviour of the
sstate logic).
RM_OLD_IMAGE is therefore no longer required to provide this
behaviour, remove the variable and its users.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Add a function (and test suite) to turn the ELF machine field (e_machine) into a
string, so we can tell the user "x86-64" instead of 0x3E.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Deal with an interrupted system call gracefully:
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/sshcontrol.py", line 55, in _run
| if select.select([self.process.stdout], [], [], 5)[0] != []:
| InterruptedError: [Errno 4] Interrupted system call
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
