| Age | Commit message (Collapse) | Author | Files |
|---|
|
Ensure we capture stderr, always write the output to the log, and don't be so
emotional in the warning message.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
* when OPKGLIBDIR doesn't have the default /var/lib value it will
silently fail to copy package database from normal rootfs to debugfs
rootfs and then when trying to install *-dbg complimentary packages
it won't install anything, because installed_pkgs.txt file generated
from debugfs is empty
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To properly look at this patch, you probably need a side-by-side diff viewing tool.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
|
|
No need to store the configuration as class members,
just pass it directly into the method.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
|
|
It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file in the deploy directory warrants the complexity it brings elsewhere.
Let's just remove it entirely.
In particular, if two do_image_complete tasks run in parallel they risk
both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time.
Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file. In theory this should be safe because "cp -alf" will just cause one
to overwrite the other. Unfortunately, coreutils cp also has a race[1]
which means that if one copy creates the file at just the wrong point the
other will fail with:
cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to
+‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists
[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you use subprocess.check_output() the traceback will contain the output
when the command fails which is very useful for debugging. There is no
good reason not to use this everywhere.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When removing unneeded packages from a (read-only) rootfs
during rootfs creation, alternative symlinks from those
packages may or may not be removed.
The reason is as follows:
update-alternatives(-native) is used during package
installation as part of the image creation. It uses
a database which contains entries for all the
alternative symlinks possible, and the -native version
uses the target's database by means of $OPKG_OFFLINE_ROOT,
i.e. the rootfs we're in the process of creating.
Once the rootfs has been created, OE removes certain
packages because we have a read-only rootfs - in
particular ROOTFS_RO_UNNEEDED which includes
VIRTUAL-RUNTIME_update-alternatives, i.e. the
update-alternatives. Recently, a change was made in
OE, where uninstallation of update-alternatives from the
rootfs causes removal of its database, too, to save space
(700KiB (uncompressed) in a busybox system)
b24a63d71b517af701dfedbc7f7b541d25af708f
http://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb?id=b24a63d71b517af701dfedbc7f7b541d25af708f
Following from that, if update-alternatives is removed
from the target file system, update-alternatives-native
has no database anymore, meaning it can't manage any of
the alternative symlinks anymore.
Because the order of packages to uninstall is
non-deterministic, and update-alternatives could well
be removed before any packages that use the mechanism
provided, sometimes the extra symlinks are removed,
sometimes not.
By sorting the list of packages to be removed such that
update-alternatives is removed last, we can ensure that
that tings work reliably. (Certainly opkg seems to
uninstall packages in the order given on the command
line.)
[YOCTO #10916]
Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The /etc and /usr/lib/ssl directories were only put into the opkg-generated
debugfs because of a bug in opkg which means that a conffile has to exist if
we're running 'opkg status'. This is now fixed, so the workaround can be
reverted.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
through BitBake
If you printed a warning through bb.warn() / bbwarn or an error through
bb.error() / bberror, this was also being picked up by our log_check
mechanism that was designed to pick up warnings and errors printed by
other programs used during do_rootfs. This meant you saw not only the
warning or error itself, you saw it a second time through log_check,
which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the
logger and add a handler that we can use to find out if any warning or
error we find in the logs is one we should ignore as it has already been
printed.
Fixes [YOCTO #8223].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Changed deployment directory from DEPLOY_DIR_IMAGE to
IMGDEPLOYDIR to make sstate machinery to do final deployment and
generate manifest.
Renamed variable deploy_dir to deploy_dir_image in selftest code
to avoid confusion with DEPLOYDIR variable.
Updated the code of rootfs.py:Rootfs class to use IMGDEPLOYDIR variable
as it's now used as a new deployment destination.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Current functionality allows for the removal of certain packages
based on the read-only image feature. This patch extends this
functionality by adding the FORCE_RO_REMOVE variable, which will
remove these packages regardless of any image features.
[ YOCTO #9491 ]
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Use the new task progress functionality to report progress during
do_rootfs. This is a little coarse and ideally we would have some
progress within the installation section, but it's better than
nothing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In commit 20ea6d274bb0a9a5addb111f32793de49b907865, debugfs generation
for images using opkg, which included openssl was fixed.
However, that broke the generation of the opkg-based images, that lacks
openssl. The error is a python stack trace, showing that shutil.copytree
tries to copy a non-existing directory.
This relates to [YOCTO #9040].
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use a common _log_check_common() function (based on the old
_log_check_warn() function) to implement the logic for both
_log_check_warn() and _log_check_error().
The main benefit of this is that now all error messages will be
reported again, not just the first one found. Additionally the output
will now look the same for both error and warning messages.
This removes the context for the error messages. However, since there
was no indication in the output that some of the lines were context,
they were more confusing than helping.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For each warning found in the log, _log_check_warn() would output a
line stating that it had found a warning, then the actual warning and
finally an empty line. This is quite excessive when there are many
warnings in the log.
With this change the output is instead a line stating how many
warnings were found, followed by the warnings. This makes the output
much more compact and actually much more readable.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This will make _log_check_warn() exclude the same lines as
_log_check_error() does.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before there were three different ways to exclude a line from being
searched for error messages in _log_check_error(). Now there is only
one: an array of regular expressions. This should make it easy to add
more excludes if nedded.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In commit 0387d095 lines with "NOTE:" in them were excluded from the
log check for warnings. However, those lines were only there in the
first place since the regular expression that is used to find warning
messages explicitly included those lines...
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The fact that this function was overridden in the RpmRootfs class
seems to have led to a number of misstakes when changes have been made
to the base function in the Rootfs class. E.g., this change will
properly solve ticket 7789, which was supposedly solved in 38871dc0,
but that change had no effect in practice as the log_check_regex that
was modified for RpmRootfs class was not used by the RpmRootfs version
of _log_check_error()...
The only thing _log_check_error() in RpmRootfs did that the base
function in Rootfs did not do was to skip lines in the log that begin
with a + sign. This has now been moved to the base function instead.
[YOCTO #7789]
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The reverted commit added a warning with the log line that triggered
the log check for error messages before the warning that states that
an error has been found in the log. However, the error line is output
by the call to bb.fatal() that follows immediately after the original
warning, which makes it redundant. Additionaly, having two warnings
contradicts the intent of commit 8dfdd329 where the log warnings were
tidied up.
This reverts commit f9cf31525fc885e1a0f65bd55654631257f87078.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When enabling extra DEBUGFS image generation with opkg, the following error is
seen when openssl-cnf is included in the image.
Collected errors:
* file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/oe-build/build-ccu/tmp/work/ccu-oe-linux-gnueabi/ccu-image/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.
Lots of similar issues was fixed by an earlier commit in oe-core,
5084ed9401250ed269a49d27b303806ab173c5d5, but openssl-cnf is outside of that fix.
Followup to [YOCTO #9490]
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory,
the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
[YOCTO #9490]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, ROOTFS_POSTINSTALL_COMMAND was run only after the opkg rootfs
install post processing phase. This patch makes it generic so it is run fter
any package manager specific rootfs post processing, but before _run_intercepts().
[YOCTO #9248]
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove opkg package feed lists after generating the rootfs. The
lists have been generated by the local feed but are no longer
necessary after image generation. This still leaves the package
management fully usable (and hence is different from omitting
package-management from IMAGE_FEATURES).
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].
This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.
[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
In the current state some of the base utils (update-rc.d,
base-passwd, shadow, and update-alternatives) are unistalled
when there is no package manager in the image. Checking for
previous commits, the unistall of these utils were to be
done in a read-only filesystem.
It is a valid option to have these utils without a package
manager, and also make sense to remove them when building a
read-only filesystem.
This changes the check logic from having a package mananger
to if is a read-only filesystem to remove the utils.
Another change implemented with this patch is that delayed
post installs now doesn't depend if there is a package manager.
Also it is a valid option to have post install scripts without
package manger.
[YOCTO #8235]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The method _uninstall_unneeded uses the file installed_pkgs.txt,
this file is left after the build and can cause confusion. This
changes allow to get the installed packages using functions of
rootfs instead of the installed_pkgs.txt file. With this change
now is possible to remove the file without breaking anything.
[YOCTO #8444]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some intercepts may want to access files in the native sysroot that are not on
$PATH (such as something in $libexecdir) but any use of STAGING_DIR_NATIVE in
the postinst that calls the intercept will be "baked" into the package, so if
sstate is reused it will use paths that may not exist.
Solve this by exporting the location of the native sysroot in the environment so
the postinst and intercept can use an environment variable instead of a bitbake
variable.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since the log_check_regex can potentially be false positive it
makes sense to print the whole line where error is found.
This way user will be able to see the error and understand
if it's valid or not.
[YOCTO: #7789]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
rootfs.py: added a test in DpkgOpkgRootfs which skips the
get package post installs step (get_pkgs_postinsts), when
PACKAGE_INSTALL is empty.
[YOCTO #7664]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
update-rc.d is now needed by systemd to interact with sysv
scripts, so if we have a mixed system, then we can not uninstall
update-rc.d as it is going to break systemd functionality, we check for
sysvinit and sytemd in distro and also ensure that its not building an
image with readonly rootfs
Signed-off-by: Khem Raj <raj.khem@gmail.com>
|
|
This reverts commit acf90b6c299afe9e9c8fa33c3c6992bfcf40fbbf.
(Reverted to apply the better v2 of the patch)
|
|
* without this the output wasn't shown anywhere even when the bb.warn
says:
"See log for details!"
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
update-rc.d is now needed by systemd to interact with sysv
scripts, so if we have a mixed system, then we can not uninstall
update-rc.d as it is going to break systemd functionality
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* useful when we need to overlay/extend intercept scripts from oe-core
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.
[YOCTO #7664]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The rpm didn't run RPM_PREPROCESS_COMMANDS or RPM_POSTPROCESS_COMMANDS,
the similar to deb, this patch fix the problem.
And fix a typo:
DEB_POSTPROCESS_COMMAND -> DEB_POSTPROCESS_COMMANDS
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a check for kernel modules so we don't un-necessarily run the depmods, this
will also handle the case with linux-dummy does not place the kernel-abiversion
since it also does not have kernel modules.
[YOCTO #7884]
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
postinsts scripts.
The old code don't take into account package dependencies causing
undefined execution order in postinsts scripts, in order to fix:
Add DpkgOpkgRootfs class for store common operations in DpkgRootfs
and OpkgRootfs.
Add _get_delayed_postinsts_common method that process Depends from
status file in dpkg/opkg and resolve dependency order causing an
execption if found circular dependencies.
[YOCTO #5318]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* bb.warn() should only be called once per warning - UIs such as Toaster
assume that this is the case, so adjust the output accordingly. (It's
tricky here because we have to include "log_check" on every line or
we'll end up looping forever as the log checking code's own messages
retrigger the log check, sigh...)
* Iterating over a file already splits by line, there's no need to do it
manually.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The companion debug filesystem contains only the package database and the
complementary *-dbg packages for the main filesystem component. This is
useful in a production environment to produce a companion filesystem capable
of remote system debugging, without requiring corresponding debug symbols or
source code on the device.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
If dbg pkgs have already been installed to the rootfs image,
the installation to companion debug filesystem will fail,
because both of image creation make use of the same pm
database.
In this situation, try to copy installed dbg files from rootfs
image to companion debug filesystem.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Acked-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The rootfs logfile check errors out if it finds an "exit 1" in
a log. But consider the shell idiom:
if ! $command; then exit 1; fi
and consider a postinstall script with a "set -x" for debugging
purposes (to get log output in /var/log/postinstall.log.)
Solution: Ignore lines prefixed with a +, because those show shell
code even if a specific fragment won't be executed.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Current regex can cause false negatives if paths in bitbake log files
contain "ERR" or "Fail". do_rootfs fails with return code 1 in
this case.
Improved regexp is based on error messages produced by rpm.
Those are found by analyzing rpm source code.
[YOCTO #7789]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This brings back commit d41097a970e9bfa553cd4bc3c9fad4b9073d7bd5,
which got lost during transition from image.bbclass to rootfs.py.
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
