openembedded-core.git/meta/classes/externalsrc.bbclass, branch thud

externalsrc.bbclass: Set BB_DONT_CACHE for non-target recipes

2018-10-08T13:13:48+00:00

BB_DONT_CACHE was not set for non-virtual recipes where PN != BPN, such
as quilt-native.  Recipes that do not set BBCLASSEXTEND should always
have BB_DONT_CACHE set by externalsrc.

Signed-off-by: Ola x Nilsson 
Signed-off-by: Richard Purdie

externalsrc.bbclass: don't configure with --disable-dependency-tracking

2018-05-29T10:25:04+00:00

One of the uses of externalsrc is to enable iterative editing and
rebuilding of source files during development. In such situations,
disabling Automake dependency tracking can lead to sources not being
rebuilt even though files they depend on have been modified.

Signed-off-by: Andre McCurdy 
Signed-off-by: Ross Burton

classes/externalsrc: ensure cleandirs code handles non-absolute paths

2018-04-05T13:48:58+00:00

It's possible that a trailing or extra slash somewhere in the external
source path could result in the directory not being removed from
cleandirs; it's also possible that a cleandirs entry is somewhere
underneath the source tree and that tree should never have parts of it
deleted by the build system. Use oe.path.is_path_parent() (which makes
paths absolute before checking them) to find out if any path in
cleandirs is anywhere underneath the external source path, and drop it
if it is.

Signed-off-by: Paul Eggleton 
Signed-off-by: Richard Purdie

classes/externalsrc: handle if cleandirs contains python expressions

2018-04-05T13:48:58+00:00

Use the existing oe.recipeutils.split_var_value() function to split the
unexpanded value of the cleandirs varflag, in case it contains python
expressions - we don't want to split the expression itself as the chunks
will not expand properly individually and we can miss something that
expands to the source tree (and thus it can get deleted, the avoidance
of which is the whole point of this code).

Signed-off-by: Paul Eggleton 
Signed-off-by: Richard Purdie

externalsrc: do not call make clean for recipes with CLEANBROKEN = "1" set

2018-03-25T08:33:36+00:00

ERROR: distrho-ports-0.0.0+git999-r0 do_buildclean: oe_runmake failed
ERROR: distrho-ports-0.0.0+git999-r0 do_buildclean: Function failed: do_buildclean
ERROR: Logfile of failure stored in: <...>/temp/log.do_buildclean.17285
Log data follows:
| DEBUG: Executing shell function do_buildclean
| NOTE: make clean
| make clean -C libs/drowaudio
| make[1]: Entering directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio'
| make clean -C build-drowaudio
| make[2]: Entering directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio/build-drowaudio'
| make[2]: *** No rule to make target 'clean'.  Stop.
| make[2]: Leaving directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio/build-drowaudio'
| make[1]: *** [Makefile:7: clean] Error 2
| make[1]: Leaving directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio'
| make: *** [Makefile:73: clean] Error 2
| ERROR: oe_runmake failed

Signed-off-by: Andreas Müller 
Signed-off-by: Ross Burton

externalsrc.bbclass: Suppress git errors

2018-02-16T18:05:26+00:00

Suppress any warnings git might generate when searching for a valid git
directory, as there are use cases where the directory is expected to not
exist and the warning is superfluous

Signed-off-by: Joshua Watt 
Signed-off-by: Richard Purdie

externalsrc: fix ExpansionError if the source dir does not exist yet

2017-11-08T19:54:22+00:00

The externalsrc class code assumes that the source directory
(EXTERNALSRC) exists before bitbake is called. Otherwise do_configure
will fail obviously since externalsrc does not fetch anything.

Commit 3ca6085729d9 ("externalsrc: Handle .git not being a directory")
changed this behaviour. Now on a missing EXTERNALSRC directory we get
a bb.data_smart.ExpansionError during _parsing_, way before
do_configure can be run.

This new behaviour creates two problems:

 * First, there error message is very cryptic (and it's hard to
   provide a better message since no task is ever run):

     ERROR: ExpansionError during parsing /<...>/<...>.bb
     Traceback (most recent call last):
     bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception FileNotFoundError: [Errno 2] No such file or directory: '<...>'

 * Second, this prevents creating a class based on externalsrc that
   automatically fetches the code in EXTERNALSRC before do_compile
   runs.

Fix both problems by simply calling git with '-C ${EXTERNALSRC}'
instead of calling git inside the non-existing directory. This changes
from a bb.data_smart.ExpansionError to a
subprocess.CalledProcessError, which is in line with what's actually
going on: git is telling us it can't find the git dir.

Also remove a comment that does not apply anymore.

Signed-off-by: Luca Ceresoli 
Cc: Joshua Watt 
Signed-off-by: Ross Burton

classes/externalsrc: delete symlinks at start of do_buildclean

2017-09-21T08:23:59+00:00

To help users find the work and log directories (especially within the
eSDK) we create symlinks to these from the source tree. However during
do_buildclean we call "make clean", and some project Makefiles which
delete something like "*/*/lib" which will match files underneath the
oe-workdir symlink and fail. do_buildclean is called from do_clean which
is in turn called by devtool reset by default, and thus devtool reset is
blocked. An example of a recipe where this is visible is the openssl-qat
recipe in meta-intel.

In order to fix this, delete the symlinks at the start of do_buildclean
since we shouldn't need them at that point anyway, and thus make clean
won't be able to trip over them.

Fixes [YOCTO #11036].

Signed-off-by: Paul Eggleton 
Signed-off-by: Ross Burton

externalsrc: Handle .git not being a directory

2017-08-15T23:01:42+00:00

Use git rev-parse to determine the location of the .git directory, in
case it is not an immediate child of EXTERNALSRC (e.g. when using
submodules). In the event git can't resolve the .git directory, fall
back to the non-git method for hashing.

Signed-off-by: Joshua Watt 
Signed-off-by: Richard Purdie

externalsrc: place copy of git index into /tmp and do not use copyfile2

2017-07-25T14:56:33+00:00

Using shutil.copy2() to copy .git/index to a temporary file tries to
copy SELinux attributes which might fail for confined users in SELinux
environments.

E.g. our builders are running in docker containers and modification of
sources (inclusive updated of .git/index) is done outside.  Trying to
copy .git/index fails with

| $ python3 -c 'import shutil; shutil.copy2("index", "a")'
| ...
| PermissionError: [Errno 13] Permission denied: 'a'

and an AVC like

| denied  { relabelto } for  pid=18043 comm="python3" name="a" dev="dm-29" ino=1067553 scontext=system_u:system_r:container_t:s0:c39,c558 tcontext=unconfined_u:object_r:build_file_t:s0 tclass=file permissive=0

is created.  This can not be solved by adapting the SELinux policy because
this is a very deep constraint violation:

| constrain file { create relabelfrom relabelto } ((u1 == u2 -Fail-)  or (t1 == can_change_object_identity -Fail-) ); Constraint DENIED
|
| Possible cause is the source user (system_u) and target user (unconfined_u) are different.

I do not see much sense in using 'shutil.copy2()' here; 'shutil.copyfile()'
seems to be a better choice (target file is created in a secure way by
tempfile.NamedTemporaryFile()).

By placing the tempfile into /tmp we avoid potential problems related to
git's 'core.sharedRepository'.  As a (positive) side effect, the source
tree will not be modified anymore (at least by this part of code) which
prevented to mount it read-only from somewhere else.

Signed-off-by: Enrico Scholz 
Signed-off-by: Ross Burton 
Signed-off-by: Richard Purdie