| Age | Commit message (Collapse) | Author | Files |
|---|
|
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In commit 20ea6d274bb0a9a5addb111f32793de49b907865, debugfs generation
for images using opkg, which included openssl was fixed.
However, that broke the generation of the opkg-based images, that lacks
openssl. The error is a python stack trace, showing that shutil.copytree
tries to copy a non-existing directory.
This relates to [YOCTO #9040].
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use a common _log_check_common() function (based on the old
_log_check_warn() function) to implement the logic for both
_log_check_warn() and _log_check_error().
The main benefit of this is that now all error messages will be
reported again, not just the first one found. Additionally the output
will now look the same for both error and warning messages.
This removes the context for the error messages. However, since there
was no indication in the output that some of the lines were context,
they were more confusing than helping.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For each warning found in the log, _log_check_warn() would output a
line stating that it had found a warning, then the actual warning and
finally an empty line. This is quite excessive when there are many
warnings in the log.
With this change the output is instead a line stating how many
warnings were found, followed by the warnings. This makes the output
much more compact and actually much more readable.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This will make _log_check_warn() exclude the same lines as
_log_check_error() does.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before there were three different ways to exclude a line from being
searched for error messages in _log_check_error(). Now there is only
one: an array of regular expressions. This should make it easy to add
more excludes if nedded.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In commit 0387d095 lines with "NOTE:" in them were excluded from the
log check for warnings. However, those lines were only there in the
first place since the regular expression that is used to find warning
messages explicitly included those lines...
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The fact that this function was overridden in the RpmRootfs class
seems to have led to a number of misstakes when changes have been made
to the base function in the Rootfs class. E.g., this change will
properly solve ticket 7789, which was supposedly solved in 38871dc0,
but that change had no effect in practice as the log_check_regex that
was modified for RpmRootfs class was not used by the RpmRootfs version
of _log_check_error()...
The only thing _log_check_error() in RpmRootfs did that the base
function in Rootfs did not do was to skip lines in the log that begin
with a + sign. This has now been moved to the base function instead.
[YOCTO #7789]
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The reverted commit added a warning with the log line that triggered
the log check for error messages before the warning that states that
an error has been found in the log. However, the error line is output
by the call to bb.fatal() that follows immediately after the original
warning, which makes it redundant. Additionaly, having two warnings
contradicts the intent of commit 8dfdd329 where the log warnings were
tidied up.
This reverts commit f9cf31525fc885e1a0f65bd55654631257f87078.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When enabling extra DEBUGFS image generation with opkg, the following error is
seen when openssl-cnf is included in the image.
Collected errors:
* file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/oe-build/build-ccu/tmp/work/ccu-oe-linux-gnueabi/ccu-image/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.
Lots of similar issues was fixed by an earlier commit in oe-core,
5084ed9401250ed269a49d27b303806ab173c5d5, but openssl-cnf is outside of that fix.
Followup to [YOCTO #9490]
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory,
the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
[YOCTO #9490]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, ROOTFS_POSTINSTALL_COMMAND was run only after the opkg rootfs
install post processing phase. This patch makes it generic so it is run fter
any package manager specific rootfs post processing, but before _run_intercepts().
[YOCTO #9248]
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove opkg package feed lists after generating the rootfs. The
lists have been generated by the local feed but are no longer
necessary after image generation. This still leaves the package
management fully usable (and hence is different from omitting
package-management from IMAGE_FEATURES).
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].
This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.
[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.
It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.
This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.
Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.
[YOCTO #8235]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The method _uninstall_unneeded uses the file installed_pkgs.txt,
this file is left after the build and can cause confusion. This
changes allow to get the installed packages using functions of
rootfs instead of the installed_pkgs.txt file. With this change
now is possible to remove the file without breaking anything.
[YOCTO #8444]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some intercepts may want to access files in the native sysroot that are not on
$PATH (such as something in $libexecdir) but any use of STAGING_DIR_NATIVE in
the postinst that calls the intercept will be "baked" into the package, so if
sstate is reused it will use paths that may not exist.
Solve this by exporting the location of the native sysroot in the environment so
the postinst and intercept can use an environment variable instead of a bitbake
variable.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since the log_check_regex can potentially be false positive it
makes sense to print the whole line where error is found.
This way user will be able to see the error and understand
if it's valid or not.
[YOCTO: #7789]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
rootfs.py: added a test in DpkgOpkgRootfs which skips the
get package post installs step (get_pkgs_postinsts), when
PACKAGE_INSTALL is empty.
[YOCTO #7664]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
update-rc.d is now needed by systemd to interact with sysv
scripts, so if we have a mixed system, then we can not uninstall
update-rc.d as it is going to break systemd functionality, we check for
sysvinit and sytemd in distro and also ensure that its not building an
image with readonly rootfs
Signed-off-by: Khem Raj <raj.khem@gmail.com>
|
|
This reverts commit acf90b6c299afe9e9c8fa33c3c6992bfcf40fbbf.
(Reverted to apply the better v2 of the patch)
|
|
* without this the output wasn't shown anywhere even when the bb.warn
says:
"See log for details!"
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
update-rc.d is now needed by systemd to interact with sysv
scripts, so if we have a mixed system, then we can not uninstall
update-rc.d as it is going to break systemd functionality
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* useful when we need to overlay/extend intercept scripts from oe-core
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.
[YOCTO #7664]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The rpm didn't run RPM_PREPROCESS_COMMANDS or RPM_POSTPROCESS_COMMANDS,
the similar to deb, this patch fix the problem.
And fix a typo:
DEB_POSTPROCESS_COMMAND -> DEB_POSTPROCESS_COMMANDS
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a check for kernel modules so we don't un-necessarily run the depmods, this
will also handle the case with linux-dummy does not place the kernel-abiversion
since it also does not have kernel modules.
[YOCTO #7884]
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
postinsts scripts.
The old code don't take into account package dependencies causing
undefined execution order in postinsts scripts, in order to fix:
Add DpkgOpkgRootfs class for store common operations in DpkgRootfs
and OpkgRootfs.
Add _get_delayed_postinsts_common method that process Depends from
status file in dpkg/opkg and resolve dependency order causing an
execption if found circular dependencies.
[YOCTO #5318]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* bb.warn() should only be called once per warning - UIs such as Toaster
assume that this is the case, so adjust the output accordingly. (It's
tricky here because we have to include "log_check" on every line or
we'll end up looping forever as the log checking code's own messages
retrigger the log check, sigh...)
* Iterating over a file already splits by line, there's no need to do it
manually.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The companion debug filesystem contains only the package database and the
complementary *-dbg packages for the main filesystem component. This is
useful in a production environment to produce a companion filesystem capable
of remote system debugging, without requiring corresponding debug symbols or
source code on the device.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
If dbg pkgs have already been installed to the rootfs image,
the installation to companion debug filesystem will fail,
because both of image creation make use of the same pm
database.
In this situation, try to copy installed dbg files from rootfs
image to companion debug filesystem.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Acked-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The rootfs logfile check errors out if it finds an "exit 1" in
a log. But consider the shell idiom:
if ! $command; then exit 1; fi
and consider a postinstall script with a "set -x" for debugging
purposes (to get log output in /var/log/postinstall.log.)
Solution: Ignore lines prefixed with a +, because those show shell
code even if a specific fragment won't be executed.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Current regex can cause false negatives if paths in bitbake log files
contain "ERR" or "Fail". do_rootfs fails with return code 1 in
this case.
Improved regexp is based on error messages produced by rpm.
Those are found by analyzing rpm source code.
[YOCTO #7789]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This brings back commit d41097a970e9bfa553cd4bc3c9fad4b9073d7bd5,
which got lost during transition from image.bbclass to rootfs.py.
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Extract the common codes of log checking and add the ability of log
checking for deb and ipk package backend.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch involves two changes.
1. Extend the regular expression to also catch '^WARNING:' in _log_check_warn.
Warnings from bb.note or bbnote begin with 'WARNING:'. So if we decide to
catch warnings at rootfs time, we should not ignore those produced by
the build system itself.
2. Delay _log_check in rootfs process so that more warnings are likely to be
catched. Note that we should at least delay the _log_check after the
execution of ROOTFS_POSTPROCESS_COMMANDS, because we want to catch warnings
there.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch creates a variable for the install_dir name so it can be
something other than /install, also by doing this we can correctly
clean up the empty directories (/install/tmp) during the clean-up
phase. The new default is /oe_install so as to not conflict with other
possible packages that might use /install to place files.
[YOCTO #7353]
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The following commit caused log checking for warnings not working for RPM
rootfs.
rootfs.py: ignore "NOTE:" when catching warnings
The problem is that checking for warnings is always skipped because the
following statement is always true.
if 'log_check' or 'NOTE:' in line:
This patch fixes the above problem so that warning checking in RPM rootfs
can work again.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The script looks for any warn|Warn pattern in the log. However,
it may happen that the warn|Warn pattern appears as part of the name
of the objects described in the NOTE lines of the log. Thus, to avoid
false positives, ignore the line if it contains the word "NOTE:"
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This allows an image to skip the creation of kernel depmod
data. It is useful for creating an image that will run as a
container image inside a host with no knowledge of the parent's
kernel.
Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since the rewrite of the image construction code in python a few
releases ago, we remove a couple of packages from the image as one of
the final steps when constructing the image (notably update-rc.d and
run-postinsts). However, because of the order of operations, these
packages are still listed both in the buildhistory
installed_package*.txt files and in the manifest file created next to
the image, which is wrong.
There were two possible solutions to this: (1) change the order such
that the uninstallation occurs before calling ROOTFS_POSTPROCESS_COMMAND
or (2) add another hook variable in such that we can have the
package list collection code run at the right time. Because it's
currently possible (but very much not recommended) to install additional
packages within ROOTFS_POSTPROCESS_COMMAND, which may have postinstall
scripts and thus require the packages we would otherwise uninstall if we
were to take option 1, option 2 is really the least likely to cause
problems. Therefore, add ROOTFS_POSTUNINSTALL_COMMAND and make the image
and buildhistory classes use it.
Fixes [YOCTO #6479].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Rename it to _uninstall_unneeded.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Using populate_sysroot for this data was a nice idea but flawed as it
doesn't work in multilib builds. Instead we can use PKGDATA_DIR since
this is consistent over multilib builds. It also turns out to be
slightly neater code too. Hopefully this resolves the problem once and
for all.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the rpm package backend enabled, running:
bitbake <image>
bitbake virtual/kernel -c clean
bitbake <image> -c rootfs -f
results in an image with incorrect kernel module dependency information.
The problem is that the System.map and kernel-abiversion files are needed
for depmod and after the recent kernel changes, these are no longer in
sstate.
Its reasonable to require the kernel to unpack/build if you're
about to build a module against it. It is not reasonable to require this
just to build a rootfs.
Therefore stash the needed files specifically for depmod.
Also fix some STAGING_KERNEL_DIR references which were incorrect, found
whilst sorting through his change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the changes to kernel layout, we need to refer to the correct
location of the kernel abiversion file.
Thanks to Saul, Randy, Darren and Bruce to figuring out the issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The fix filters out irrelevant messages, and makes the catching more
accurate, the inner warn message in do_rootfs usually comes from the
output of complementary install, and pattern the format to catch it.
Here is the example of irrelevant messages:
...
|WARNING: log_check: There is a warn message in the logfile
|WARNING: log_check: Matched keyword: [warn]
|WARNING: log_check: `tmp/deploy/rpm/core2_64/pam-plugin-warn
-1.1.6-r5.0.core2_64.rpm' -> `tmp/work/intel_x86_64-wrs-linux/
wrlinux-image-installer/1.0-r0/rootfs/Packages.intel/./core2_64/
pam-plugin-warn-1.1.6-r5.0.core2_64.rpm'
...
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Package managements (smart/apt-get/opkg-cl) generate some warn messages
to stdout, and we need to catch them and output by bb.warn.
Here is an example, while invoking smart to attempt install doc packages,
if install failed, it generates warn message to stdout.
...
|warning: Can't install util-linux-doc-2.24.2-r1@i586: Can't
install util-linux-doc-2.24.2-r1@i586: no package provides info
...
The fix catches it and outputs:
...
|WARNING: log_check: There is a warn message in the logfile
|WARNING: log_check: Matched keyword: [warn]
|WARNING: log_check: warning: Can't install util-linux-doc-2.24.2-r1@
i586: Can't install util-linux-doc-2.24.2-r1@i586: no package provides
info
...
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
|
|
Use RPM_PREFER_ELF_ARCH to instead of RPM_PREFER_COLOR as rpm upstream
suggested, and use "4" to instead of "3" since it is a bit mask (not
enumeration), so we need "4" here.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
