openembedded-core.git/meta/recipes-kernel/kern-tools, branch pyro

kernel-yocto/kern-tools: fix do_validate_branches clean stage

2017-04-28T10:26:05+00:00

It was reported that do_validate_branches was failing with the following
error:

  Log data follows:
  | DEBUG: Executing shell function do_validate_branches
  | HEAD is now at fe0fb8d Merge tag 'v4.10.9' into standard/base
  | mkdir: cannot create directory .: File exists
  |
  | [ERROR] Can't find patch dir at ./patches/standard/base
  | usage: kgit s2q
  | WARNING: exit code 1 from a shell command.
  | ERROR: Function failed: do_validate_branches

This was triggered by the execution of 'kgit-s2q --clean' after forcing
the SRCREV to something other than the tip of the branch. --clean is
being run to remove any sentinel files from previous kernel builds to
ensure that the tree is in a consistent state.

There were two bugs, --clean was being executed and not exiting the
script as it was supposed to. Hence validation for applying patches
was done, and threw the error that eventually makes it to the console.

And the second bug is that since do_validate_branches actually calls
kgit-s2q --clean, the dependency on kern-tools-native needs to be on
that function (versus do_kernel_metadata which runs later).

With the tweaked kern-tool + the dependency fix, we no longer see this
error.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie

kernel-yocto.bbclass: ensure repatching when HEAD is checked out

2017-02-19T14:16:50+00:00

In some cases it is seen that kernel_checkout and validate_branches are
run again in simultaneous builds. During do_patch the kgit-s2q mechanism
looks for a sentinel file inside the .git directory, finds a fence post
and starts picking up patches after that.

This can create trouble as validate_branches checks out the HEAD of the
branch and so the patches should be reapplied rather than skipped due to
finding of the fence post.

We can call kgit-s2q --clean to remove the sentinel file when the
branches are checked out.

Signed-off-by: Awais Belal 
Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie

kern-tools: re-enable scc merge command

2017-02-05T09:20:20+00:00

The ability to merge two branches directly from a .scc file was
dropped during the streamlining of the tools.

As was pointed out by David Vincent , there is
once again a valid use case for this functionality, so we restore the
capability.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Ross Burton

kern-tools: kconf_check: flag CONFIG options with invalid spaces

2017-01-23T12:03:58+00:00

Options with spacs around = signs will be droped/ignored by the
kernel.

The audit phase can detect this and warn the user:

    % kconf_check --report -o .kernel-meta/cfg/ \
      linux-qemux86-standard-build/.config `pwd` $cfgs

     [errors (3): .kernel-meta/cfg/fragment_errors.txt
       There are errors withing the config fragments.

    % cat .kernel-meta/cfg/fragment_errors.txt
       Warning: Ignoring "CONFIG_PARAVIRTT_CLOCK =y" -- spaces around equals are invalid
       Warning: Ignoring "CONFIG_PARAVIRTTT_CLOCK = y" -- spaces around equals are invalid
       Warning: Ignoring "CONFIG_PARAVIRTTTT_CLOCK= y" -- spaces around equals are invalid

Signed-off-by: Bruce Ashfield 
Signed-off-by: Ross Burton

kern-tools: scc: Fix process_file error check

2017-01-23T12:03:58+00:00

    Commit 626ceac135fa66277c2fa53197be33cc9d4d7614 broke the error
    check in process_file by adding in three lines that stomp on $? which
    print the output file when verbose is set.

    Move output file on verbose print to an elif after the error check.

    Signed-off-by: George McCollister 

Signed-off-by: Bruce Ashfield 
Signed-off-by: Ross Burton

kern-tools: ensure that no shared directories are used

2016-12-09T08:49:28+00:00

We need to avoid using shared/common directories for any files that are
part of specific build, since permissions issues in multi user
environments will cause issues.

Integrating the following commit to solve the issue:

   scc: move unused patch queue under output dir

Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie

kern-tools: fix processing for no branch meta-data

2016-12-07T10:45:47+00:00

Lernel meta-data that has patches, but no branches, can trigger an
error due to no branch specific patch queue.

This error then cascades to more issues since the tools are using
a named file in /tmp to store and display error messages to the
user.

We fix both issues though the following kern tools tweaks:

  commit bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c
  Author: Bruce Ashfield 
  Date:   Fri Dec 2 13:09:40 2016 -0500

    scc: use mktemp for consolidated output capture

    To provide useful error messages the tools dump pre-processed
    files and messages to a temporary file. If multiple users are
    doing builds, this means they either race, or can have permissions
    issues.

    By creating the temporary file via mktemp, we avoid both issues.
    (We also make sure to clean these up on exit, or /tmp will get
    polluted quickly).

  commit a287da4bfe0b4acb8f2b0627bd8e7abd1a1dde26
  Author: Bruce Ashfield 
  Date:   Fri Dec 2 13:08:08 2016 -0500

    patch: do not assume a branch specific patch queue is needed

    When processing input files per-branch and global patch queues are
    generated. If the meta-data has not created any branches in the
    repo, no branch specific queue is required.

    The tools assumed that one is always valid, and hence would throw a
    non-zero exit code and stop processing.

    By testing for a named per-branch queue, we avoid this issue.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Ross Burton

kern-tools: error checking and tree generation fixes

2016-11-30T15:47:10+00:00

During processing of the kernel meta data the kern tools were
not properly exiting on syntax errors or invalid commands.

Noticing and debugging these issues wasn't trivial. To make this
easier, we now trap the error and dump the offending meta-data
for the user to see.

There was also an issue with creating branches during tree
generation, which is resolved by always switching to the
active branch.

The following are the commit logs of the changes themselves:

[
  commit b36f6f9a5695f2084b83823393e13ca42284bed9
  Author: Paul Gortmaker 
  Date:   Sat Oct 22 17:23:25 2016 -0400

      kgit-scc: dont mention meta-repo in help ; it doesnt exist

      Signed-off-by: Paul Gortmaker 
      Signed-off-by: Bruce Ashfield 

  commit 08463d684c1952e74c25344cddace4c3f24c739d
  Author: Bruce Ashfield 
  Date:   Mon Oct 31 14:30:12 2016 -0400

      scc: exit on error

      If there is an error in the processing of the input files, scc
      should exit and inform the user.

      scc is executed on a combined/preprocessed file and as a result
      it doesn't have the granularity to see each input file individually.

      Rather than moving preprocessing into scc (from spp), we can trap
      the line number of the error and dump context around the line.
      This gives the user a pointer to the input file and the specific
      line that caused the problem.

      Signed-off-by: Bruce Ashfield 

  commit bf99953e8ac14cee653e559f2d4a6022c847a182
  Author: Bruce Ashfield 
  Date:   Fri Oct 28 21:23:27 2016 -0400

      kgit-meta: always checkout branches on branch commands

      During a tree generation we must always make the branch active when
      we see any kind of branch command. This ensures that any subsequent
      patches are applied in the proper context.

      Previously, only branch creation was changing the active branch, and
      this mean that tree generation was not determinstic and relied
      on the order of processing to generate a correct tree.

      Signed-off-by: Bruce Ashfield 
]

Signed-off-by: Bruce Ashfield 
Signed-off-by: Ross Burton

kern-tools: make patches with shell-unsafe characters valid

2016-10-19T16:07:46+00:00

If a patch contains characters like ; or (), it can cause processing
errors in the patch queue.

We had previously fixed this issue by renaming the patches to not
include invalid characters, but with this change to the kern tools
that ensures patch names are wrapped in quotes, we avoid shell
processing and hence they are valid.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie

kernel-yocto: streamline patch, configuration and audit phases

2016-08-18T08:27:13+00:00

We've been running with a set of kern-tools that were designed to work
with build systems that knew nothing about git, trees, commits, etc.

As such, there's been a set of shims/wrappers in place to work with
within bitbake/oe-core. These were the *me scripts: createme, updateme,
patchme and configme.

With this commit, we strip that legacy code and use the tools directly.
This means less complexity, fewer corner cases .. and no surprises
when the tools are arunning. As another benefit, the tools consume
much less time during a typical build and have no noticeable impact
on the overall build time.

Existing .scc files, features, and processing are not impacted as
these tools are compatible with existing feature descriptions and
kerne configuration fragments.

The audit of kernel configuration fragments is now detached
from the linux-yocto build structure and process. This means that
they can eventually be tweaked to offer kernel audit to any type of
kernel build and configuration process.

Additionally, the kernel symbol audit phase can now resolve symbol
dependencies and offer guidance when a symbol is missing:

WARNING: linux-yocto-4.4.15+gitAUTOINC+b030d96c7b_f5e2c49d58-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:

---------- CONFIG_BT_6LOWPAN -----------------
Config: CONFIG_BT_6LOWPAN
From: /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-source/.kernel-meta/configs/standard/features/bluetooth/bluetooth.cfg
Requested value: CONFIG_BT_6LOWPAN=y
Actual value:

Config 'BT_6LOWPAN' has the following conditionals:
BT_LE && 6LOWPAN (value: "n")
Dependency values are:
BT_LE [y] 6LOWPAN [n]

Signed-off-by: Bruce Ashfield
Signed-off-by: Ross Burton