openembedded-core.git/meta/classes/uninative.bbclass, branch thud Mirror of openembedded-core uninative: Add support for aarch64 hosts 2018-09-20T12:41:34+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2018-09-19T12:31:07+00:00 b72e54b9a649faf4eb292e3bc643d5d06d4d4fea Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative: Set the dynamic linker to use at compile time 2018-04-18T17:48:20+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2018-04-18T10:38:06+00:00 35867ee035030ab76fc9ccdb0eb1c3f80126301c Its possible some dynamic runtime library in the dependency chain may come from sstate and link to libraries which need the libc from uninative. If we don't do this and binaries are run at do_install time they would fail to find the symbols from the later libc. Examples: cmake-native do_install: bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-native/usr/lib/libexpat.so.1) dbus-native do_install: tmp/work/x86_64-linux/dbus-native/1.12.2-r0/build/bus/.libs/lt-dbus-daemon: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/x86_64-linux/dbus-native/1.12.2-r0/recipe-sysroot-native/usr/lib/libexpat.so.1) This issue is resolved when the interpreter is changed at sstate unpack time but this isn't soon enough to avoid issues at compile/install time. By specifing which dynamic linker/loader to use at compile time, this race window is removed entirely. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Its possible some dynamic runtime library in the dependency chain may
come from sstate and link to libraries which need the libc from
uninative. If we don't do this and binaries are run at do_install time
they would fail to find the symbols from the later libc. Examples:

cmake-native do_install:
bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

dbus-native do_install:
tmp/work/x86_64-linux/dbus-native/1.12.2-r0/build/bus/.libs/lt-dbus-daemon: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/x86_64-linux/dbus-native/1.12.2-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

This issue is resolved when the interpreter is changed at sstate unpack
time but this isn't soon enough to avoid issues at compile/install time.

By specifing which dynamic linker/loader to use at compile time, this
race window is removed entirely.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds 2018-04-18T17:48:20+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2018-04-17T14:42:31+00:00 d18bf7fa8e80d6cfaf3fdbe1ab06eec84b954432 We have a problem when for example, a glibc 2.27 based system builds some library like libpopt-native and puts it into sstate then it is reused on a pre glibc-2.27 system to build something which depends on popt like rpm-native. This results in an error like: recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27' In the past we've had this problem with new symbols like getrandom and getentropy, here its with a more complex symbol where there is an old version and a newer version. We've looked into various options, basically we cannot link against our uninative libc/ld.so since we don't have the right headers or compiler link libraries. The compiler doesn't allow you to switch in a new set either, even if we did want to ship them. Shipping a complete compiler, dev headers and libs also isn't an option. On the other hand if we follow the ld man page, it does say: """ The reasons for allowing undefined symbol references in shared libraries specified at link time are that: - A shared library specified at link time may not be the same as the one that is available at load time, so the symbol might actually be resolvable at load time. """ which is exactly this case. By the time the binary runs, it will use our uninative loader and libc and the symbol will be available. Therefore we basically have a choice, we get weird intermittent bugs, we drop uninative entirely, or we pass this option. If we pass the option, we can drop the other workarounds too. (From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
We have a problem when for example, a glibc 2.27 based system builds some
library like libpopt-native and puts it into sstate then it is reused
on a pre glibc-2.27 system to build something which depends on popt like
rpm-native. This results in an error like:

recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'

In the past we've had this problem with new symbols like getrandom and
getentropy, here its with a more complex symbol where there is an old
version and a newer version.

We've looked into various options, basically we cannot link against our
uninative libc/ld.so since we don't have the right headers or compiler
link libraries. The compiler doesn't allow you to switch in a new set
either, even if we did want to ship them. Shipping a complete compiler,
dev headers and libs also isn't an option.

On the other hand if we follow the ld man page, it does say:

"""
The reasons for allowing undefined symbol references in shared libraries
specified at link time are that:

- A shared library specified at link time may not be the same as the one
  that is available at load time, so the symbol might actually be
  resolvable at load time.
"""

which is exactly this case. By the time the binary runs, it will use
our uninative loader and libc and the symbol will be available.

Therefore we basically have a choice, we get weird intermittent bugs,
we drop uninative entirely, or we pass this option.

If we pass the option, we can drop the other workarounds too.

(From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative: add variables to the whitelist so that it does not re-triger recipe parsing 2018-04-03T22:53:13+00:00 Cuero Bugot cbugot@sierrawireless.com 2018-03-16T17:31:30+00:00 75bb95ada98ef129d2fa48568f27dddb078c852c When uninative is activated (poky's default) internal datastore variables are modified (NATIVELSBSTRING and SSTATEPOSTUNPACKFUNCS) to enable uninative support. This is happening after parsing is done at the beginning of the build. On the next bitbake call the recipe would be parsed if the two variables above were not added to the parsing whitelist BB_HASHCONFIG_WHITELIST. The fix is to add these two variables to the recipe parsing whitelist BB_HASHCONFIG_WHITELIST, this is done at recipe parsing time, only when uninative.bbclass is used. Signed-off-by: Cuero Bugot <cbugot@sierrawireless.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
When uninative is activated (poky's default) internal datastore variables are modified (NATIVELSBSTRING and SSTATEPOSTUNPACKFUNCS) to enable uninative
support. This is happening after parsing is done at the beginning of the build. On the next bitbake call the recipe would be parsed if the two
variables above were not added to the parsing whitelist BB_HASHCONFIG_WHITELIST.

The fix is to add these two variables to the recipe parsing whitelist BB_HASHCONFIG_WHITELIST, this is done at recipe parsing time, only when
uninative.bbclass is used.

Signed-off-by: Cuero Bugot <cbugot@sierrawireless.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative: Add compatiblity version check 2018-03-15T02:50:10+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2018-03-14T16:52:18+00:00 d6f6101cd0ae92e8ad2dec0bcb6db5044726edf9 If glibc is newer on the host than in uninative, the failure mode is pretty nasty for clusters where the sstate is shared, including the Yocto Project autobuilder. This check aborts the use of uninative in such scenarios where a newer glibc version appears and avoids corruption of sstate caches. We use ldd to check the glibc version since that is included in libc-bin (or equivalent) which locales use so it should always be present. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
If glibc is newer on the host than in uninative, the failure mode is
pretty nasty for clusters where the sstate is shared, including the Yocto
Project autobuilder.

This check aborts the use of uninative in such scenarios where a newer
glibc version appears and avoids corruption of sstate caches.

We use ldd to check the glibc version since that is included in libc-bin
(or equivalent) which locales use so it should always be present.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative.bbclass: Fix broken symlink issue 2017-12-10T22:41:43+00:00 Mark Hatle mark.hatle@windriver.com 2017-12-07T21:20:04+00:00 bfd9664edad7044b5da53fc33b8d0f6508f00950 If two builds are sharing the same DL_DIR, and the uninative file is local to a layer. When the first build gets to uninative it creates the link local to itself, and subsequent users can use the same link. However if that first build then is deleted from the disk, the symlink is no longer valid (broken). We need to update the system to detect this case, and use the model implemented by the bitbke fetch2 code. Look for a broken link, remove it, then try to create the link and ignore an exception if it already exists (since we just unlinked any bad one). Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
If two builds are sharing the same DL_DIR, and the uninative file is local
to a layer.  When the first build gets to uninative it creates the link local
to itself, and subsequent users can use the same link.  However if that first
build then is deleted from the disk, the symlink is no longer valid (broken).

We need to update the system to detect this case, and use the model
implemented by the bitbke fetch2 code.  Look for a broken link, remove it,
then try to create the link and ignore an exception if it already exists
(since we just unlinked any bad one).

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative: Allow a local own-mirror to work with uninative 2017-08-01T10:31:38+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2017-08-01T10:14:14+00:00 3b0309dbb5d52e9d7abf6fd89cc999ac4cc5d2fb The games we play with path manipulation of DL_DIR in uninative mean standard PREMIRRORS don't work and we can't easily put 'chksum' into the url path from a url parameter with the current fetcher url handling to make a generic remapping. We therefore add to PREMIRRORS when we know the chksum to create a premirror mapping which can work. [YOCTO #9888] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
The games we play with path manipulation of DL_DIR in uninative mean standard
PREMIRRORS don't work and we can't easily put 'chksum' into the url path from
a url parameter with the current fetcher url handling to make a generic
remapping. We therefore add to PREMIRRORS when we know the chksum to
create a premirror mapping which can work.

[YOCTO #9888]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
classes/uninative: set SSTATEPOSTUNPACKFUNCS[vardepvalueexclude] properly 2017-04-11T17:09:19+00:00 Paul Eggleton paul.eggleton@linux.intel.com 2017-04-07T04:57:22+00:00 a147838c3dfd4c53084a19b052b8d4e183293412 Append to the value with appendVarFlag() instead of setting it outright, so that we can also append to it in other places. Accordingly, this varflag is pipe-separated (since we want to be able to exclude any string fragment, in this case including the leading space), thus put a leading pipe character to play nicely with any existing value. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Append to the value with appendVarFlag() instead of setting it outright,
so that we can also append to it in other places. Accordingly, this
varflag is pipe-separated (since we want to be able to exclude any
string fragment, in this case including the leading space), thus put a
leading pipe character to play nicely with any existing value.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uninative.bbclass: split long command 2017-03-17T16:52:44+00:00 Leonardo Sandoval leonardo.sandoval.gonzalez@linux.intel.com 2017-03-14T09:37:25+00:00 c93b3b18f8daa8b419cc65b52cb5ceccfb1c142d Single long commands are difficult to read and maintain. Split it to make it more human-friendly. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Single long commands are difficult to read and maintain. Split it to make
it more human-friendly.

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
uninative: remove unneeded call to sparse copy 2017-03-11T16:08:50+00:00 Stephano Cetola stephano.cetola@linux.intel.com 2017-03-10T15:15:52+00:00 a2519a2f96bfea53a527c704e620e3584c97c67d As of this commit: 39f5a05152aa0c3503735e18dd3b4c066b284107 patchelf no longer inflates file sizes. Since the files are no longer inflated by patchelf, we can skip using cp with the --sparse option. More details as to how patchelf has changed are available in that commit log. Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
As of this commit:

39f5a05152aa0c3503735e18dd3b4c066b284107

patchelf no longer inflates file sizes. Since the files are no longer
inflated by patchelf, we can skip using cp with the --sparse option.
More details as to how patchelf has changed are available in that
commit log.

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>