| Age | Commit message (Collapse) | Author | Files |
|---|
|
A warning that doesn't say what the PN is doesn't really help.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The parameter "path" would be redefined inside package_qa_walk() which
is useless, so remove it.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed:
If more than one files have the same QA issue, it only prints the last
one, others are overrided, for example:
messages["host-user-contaminated"] = "foo1"
messages["host-user-contaminated"] = "foo2"
Only foo2 will be printed.
Add package_qa_add_message() to hanlde messages, so that all of them
will be printed. The package_qa_add_message() is from RP.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed race issue when:
WARN_QA_append = " buildpaths"
PACKAGE_CLASSES = "package_ipk"
$ bitbake e2fsprogs
[snip]
*** 0595: with open(path) as f:
0596: file_content = f.read()
0597: if tmpdir in file_content:
0598: messages["buildpaths"] = "File %s in package contained reference to tmpdir" % package_qa_clean_path(path,d)
0599:
Exception: IOError: [Errno 2] No such file or directory: '/path/to/tmp/work/core2-64-poky-linux/e2fsprogs/1.42.9-r0/packages-split/e2fsprogs-ptest/CONTROL/control'
The similar to package_deb.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When cleaning old builds from the sysroots, also print the sysroot architecture.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The config.log written by autoconf includes many pages of useless output,
followed by an obfuscated error message, and then more pages of every variable
that's been set. It's only understandable if you're well versed in how autoconf
behaves, and often in simple failure modes doesn't actually make it clear what
the problem was.
Instead of outputting the whole config.log to the console when do_configure()
fails, use bbfatal_log so the human-readable configure output (not the
config.log) is shown to the user, and tell the user where config.log can be
found if they need it.
[ YOCTO #8856 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If there are lots of changes between the previous build and the build about to
start bitbake will potentially print pages of:
DEBUG: Stamp $BUILD/stamps/corei7-64-poky-linux/libdrm/2.4.66-r0 is not reachable, removing related manifests
Instead of spamming the console with this list, write the list of manifests only
to the debug log and simply write a count to the console. This way the user
doesn't get spammed but still knows what is happening if bitbake appears to
stall with heavy I/O.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
do_rootfs task needs to depend on signing-keys:do_export_public_keys.
The rpm signing public key needs to be present in order to prevent a
crash because it is imported into the rootfs rpmdb before rootfs
creation starts.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If we "bitbake X -c packagedata" and the packagedata comes from sstate, we
don't need any of the tasks dependencies. This is similar to the
populate_lic case, we only care about the end result.
Therefore short circuit the dependencies so packagedata doesn't pull in
any other dependencies.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix a regression caused by OE-Core revision
eabeb26335b1a4eb1e68218160dbdbe8fdf36272 where we lost the task
dependency on packages in TOOLCHAIN_HOST_TASK (such as
meta-environment-extsdk-${MACHINE}) which led to errors about missing
packages when building the extensible SDK. This happened because I only
moved half of the dependencies coming from the standard SDK task to
do_sdk_depends.
While I'm at it, tidy up the do_populate_sdk_ext[depends] line.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Shell function names can't contain '-' characters, which means our image
task names also can't. Add some mapping to use '_' instead of the '-' so
images like "rpi-sdimg" work again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that image generation consists of separate tasks, we can improve
the dependencies in these classes. We can also incorporate the tasks
within do_image_complete instead of do_build so we have a specific task
for when the deploy directory is populated with images.
This fixes various expectations about do_image_completed and the way
I'd tried to use it to fix some sanity test problems.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic command can be used externally but for this to work, the wic
environment file needs to be present. Therefore write this out
universally, it runs in parallel with other image construction so
any performance implications are negligible.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, iso and hddimg links don't respect RM_OLD_IMAGE. This
updates them to use the common symlinks code so that they behave
like the rest of the system.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When RM_OLD_IMAGE is set, old manifest files should be removed
along with old image files and our QA tests expect this.
This patch ensures this happens.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similarly to DATETIME, don't expand TMPDIR in image commands.
This ensures some of the stamp comparisons we make in the
QA tests work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_image_* tasks contained the expanded version of DATETIME. Due
to the expansion, we couldn't exclude the value from the task checksum
which meant the task would rerun.
We fix this by deleting the DATETIME value during expansion so we don't
expand any references to at that time. This means the task's hash can be
stable rather than having hardcoded date/time values. It will get expanded
at execution time.
This also fixes errors shown by -S:
NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/locked-sigs.inc
ERROR: Bitbake's cached basehash does not match the one we just generated (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-sato/images/core-image-sato.bb.do_image_tar)!
ERROR: The mismatched hashes were 77872792556367f1dde49a1425caf1a0 and 9bb0aca6286ab7dd22d3c69964beb665
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:
======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.
It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.
Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst in theory return values are nice, handling the complex
aggregation of the return values is ugly. The code already uses QA_SANE
as a marker to signal problems. Whilst that isn't as idealistic, it
makes the code massively cleaner, so rely on this instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move do_kernel_link_vmlinux() from kernel-yocto.bbclass into
kernel.bbclass so that it's available to any kernel recipe.
Note that the task is not enabled by default in kernel-yocto.bbclass,
so don't enable by default in kernel.bbclass either. To enable, see
the example in linux-yocto.inc, ie:
addtask kernel_link_vmlinux after do_compile before do_install
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_kernel_link_vmlinux() task modifies the build directory (not
the source tree) and should not be skipped when externalsrc is being
used.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
SDK artifacts were being dumped alongside the image file dump.
However, the dump was not being triggered correctly, as it
should be attached as a postfunc to do_populate_sdk, rather than
do_rootfs, as do_rootfs doesn't always fire when populating
the SDK directory.
By adding a postfunc to do_populate_sdk, the postfunc is fired
in the correct situations.
Also fixes the directory which is scanned for SDK artifacts, as
this is different from the image file directory.
[YOCTO #7603]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This removes unnecessary spaces inserted before semicolons in the
modified USERADD_PARAM_${PN} and GROUPADD_PARAM_${PN} variables. This
should not affect the handling of the variables as the only one that
actually sees the semicolons is the code in useradd.bbclass that uses
cut to split the variables at them, and any whitespace preceeding or
following the semicolons will be properly ignored.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Read and merge the passwd/group files before parsing the user and
group definitions. This means they will only be read once per
recipe. This solves a problem where if a user was definied in multiple
files, it could generate group definitions for groups that should not
be created. E.g., if the first passwd file read defines a user as:
foobar::1234::::
and the second passwd file defines it as:
foobar:::nogroup:The foobar user:/:/bin/sh
then a foobar group would be created even if the user will use the
nogroup as its primary group.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The original code was near impossible to follow, and missed a couple
of cases. For example, if one added the following line to the passwd
file specified in USERADD_UID_TABLES:
foobar:x:12345:nogroup::/:/bin/sh
and then specified the user as:
USERADD_PARAM_${PN} = "--system foobar"
one would then assume that the foobar user would be created with the
primary group set to nogroup. However, it was not (the primary group
would be foobar), and the only way to get it correct was to explicitly
add --gid nogroup to the USERADD_PARAM_${PN}.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The [<on_true>, <on_false>][not <condition>] construct may solve the
problem of implementing a conditional operator, but it is not very
readable. At least I find this:
uaargs.groupid = field[3] or uaargs.gid or uaargs.groupname
a lot more readable than this:
uaargs.groupid = [uaargs.gid, uaargs.groupname][not uaargs.gid]
uaargs.groupid = [field[3], uaargs.groupid][not field[3]]
Also, the official conditional operator since Python 2.5 (<on_true> if
<condition> else <on_false>) does not evaluate both <on_false> and
<on_true> as [<on_true>, <on_false>][not <condition>] does.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If --no-user-group is specified in USERADD_PARAM_${PN} for a user and
no --gid is specified, then we should not assume that the group name
for the user is the user name.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The useradd options --create-home/--no-create-home and
--user-group/--no-user-group are mutually exclusive and should be
treated as such.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The absolute path (/path/to/configure) caused VPATH in Makefile to be an
absolute path, and then it will be in elf files, use relative path to
run configure can fix the problem.
This will reduce a lot of buildpaths QA issues in a world buld.
[YOCTO #8894]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
kernel ARCH
For a bare-bone kernel recipe which specifies 32 bit x86 target,
a 64 bit .config will be generated from do_configure task when
building 32-bit qemux86, once all of these conditions are true:
* arch of host is x86_64
* kernel source tree used in build has commit ffee0de41 which
actually chooses i386 or x86_64 defconfig by asking host when
ARCH is "x86" (arch/x86/Makefile)
* bare-bone kernel recipe inherits directly from kernel without
other special treatments.
Build will fail because of the mismatched kernel architecture.
The patch sets ARCH i386 or x86_64 explicitly to configure
task to avoid this host contamination. Kernel artifact is also
changed so that it can map i386 and x64 back to arch/x86 when
needed.
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).
The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.
Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.
(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.
The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.
The solution is to check QA_SANE in places we use the handle_error
function.
[YOCTO #8907]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
newline
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES. For instance, when using mingw32 a dependency
error will occur:
NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']
This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext
Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.
Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.
Also bring in some additional mappings to Debian architecture names
that weren't being handled.
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
xz gives *much* faster decompression times for the SDK which in itself
is a good reason to use it. It also gives better compression.
One downside is its slower but we care about the end user case
first, build performance secondary.
It also assumes the SDK user has a tar capable of understanding a xz
compressed file but that should be common enough now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the separation of do_rootfs, some rootfs references need changing
to image_complete.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
