Currently there are multiple issues with useradd:

* If base-passwd rebuilds, it wipes out recipe specific user/group additions
  to sysroots and causes errors
* If recipe A adds a user and recipe B depends on A, it can't see any of the
  users/groups A adds.

This patch changes base-passwd so it always works as a postinst script
within the sysroot and copies in the master files, then runs any
postinst-useradd-* scripts afterwards to add additional user/groups.

The postinst-useradd-* scripts are tweaked so that if /etc/passwd doesn't exist
they just exit, knowning they'll be executed later. We also add a dummy entry to
the dummy passwd file from pseudo so we can avoid this too.

There is a problem where if recipe A adds a user and recipe B depends on A but
doesn't care about users, it may not have a dependency on the useradd/groupadd
tools which would therefore not be available in B's sysroot. We therefore also
tweak postinst-useradd-* scripts so that if the tools aren't present we simply
don't add users. If you need the users, you add a dependency on the tools in the
recipe and they'll be added.

We add postinst-* to SSTATE_SCAN_FILES since almost any postinst script of this
kind is going to need relocation help.

We also ensure that the postinst-useradd script is written into the sstate
object as the current script was only being added in a recipe local way.

Thanks to Peter Kjellerstedt <pkj@axis.com> and Patrick Ohly for some pieces
of this patch.

[Yocto #11124]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Commit 51edde653707e7a3cd2186082458f01f32cd1996 makes a wrong assumption
that SSTATE_MIRRORS have write permissions.

A mirror is by definition outside of it's user control. In my use case
it happens I does not have permissions to update the access time of the
dereferenced symbolic-link file.

Checked if file is writable before changing its atime.

Thanks to Paulo Neves for the patch.

[YOCTO #11307]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

We never need these tasks as dependencies of other sstate tasks since
they're only ever needed to build artefacts so we can always skip them
and save some time/space.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

When you enable the systemd DISTRO_FEATURE, opkg-native contains systemd units
which have a relocation fixme list. When systemd isn't in DISTRO_FEATURES, there
are no fixmes required. Unfortunately as sstate isn't cleaning up its installation
directory before use, if you install the systemd version, then install the
non-systemd version from sstate, it would leave behind the fixme file from the
systemd version and breakage results as it would try and fixup files which don't
exist.

The solution is to ensure the unpack/install directory is clean before use. It
does raise other questions about opkg-native, systemd and DISTRO_FEATURES but there
is an underlying sstate issue here too which would cause missing file failures.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Too many places in dnf/rpm4 stack make that assumption; let's not fight against it.

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>

.siginfo files are not being accessed from local or NFS-mounted
sstate mirrors when sstate package is installed, so their atime
is not updated. If sstate mirror is cleaned based on access time,
they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command
dereferences symlinks pointing to the local mirror and updates
atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file in the deploy directory warrants the complexity it brings elsewhere.
Let's just remove it entirely.

In particular, if two do_image_complete tasks run in parallel they risk
both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time.
Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file. In theory this should be safe because "cp -alf" will just cause one
to overwrite the other. Unfortunately, coreutils cp also has a race[1]
which means that if one copy creates the file at just the wrong point the
other will fail with:

 cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to
+‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

The current relocation code is broken, at least in the native case. Fixing it
would mean trying pass in new data on sstate tasks about the relative positioning
of symlinks compared to the sstate relocation paths. Whilst we could do this,
right now I'm favouring making this an error and fixing the small number of
problematic recipes we have in OE-Core (3).

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

According to profile data, repeated calls to bb.debug and bb.note in
the extend_recipe_sysroot() codepath were accounting for 75% of the time
(1.5s) in calls from tasks like do_image_complete.

This batches up the log messages into one call into the logging system
which gives similar behaviour to disabling the logging but retains the
debug information.

Since setscene_depvalid is also called from bitbake's setscene code,
we have to be a little creative with the function parameters and leave
the other debug output mechanism in place. This should hopefully
speed up recipe specific sysroots.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>