openembedded-core.git/scripts/verify-bashisms, branch pyro

verify-bashisms: support warnings with more than one line of source code

2017-01-31T15:28:35+00:00

All warnings start with "possible bashism in", followed by one or more
(in the case of line continuation) lines of source code. To support
more than one line, we now split by matching against the known intro
text.

Example:

 $ verify-bashisms guile
 ...
 /.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb
  possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"):
         	echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \
 			> ${B}/guile-config.cross

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: check scripts only once, include original file and line

2017-01-31T15:28:35+00:00

Several scripts that are defined in .bbclass files end up in multiple
different recipes. It's better (faster, less repetitive error reports)
to check them only once.

In addition, the real information for the developer is where he can
find the script, not which recipe file uses it. verify-bashisms now
prints the original file instead of the recipe whenever possible
(i.e. 'filename' is set) and also bumps the line number so that it is
relative to the file and not the script.

Example with one real error and one added just for testing:

  $ verify-bashisms core-image-minimal core-image-sato
  Loading cache: 100% |#################################################################################| Time: 0:00:00
  Loaded 2935 entries from dependency cache.
  Parsing recipes: 100% |###############################################################################| Time: 0:00:01
  Parsing of 2137 .bb files complete (2101 cached, 36 parsed). 2935 targets, 412 skipped, 0 masked, 0 errors.
  Generating scripts...
  Scanning scripts...

  /.../openembedded-core/meta/classes/populate_sdk_ext.bbclass
   possible bashism in install_tools line 515 (should be 'b = a'):
  	if [ "${SDK_INCLUDE_TOOLCHAIN}" == "1" -a ! -e $unfsd_path ] ; then
   possible bashism in install_tools line 521 (type):
            type fixme

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: revise update-rc.d whitelist entry

2017-01-31T15:28:35+00:00

The actual code recently changed to:
   if ${@use_updatercd(d)} && type update-rc.d >/dev/null 2>/dev/null; then

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: fix problems with tinfoil2

2017-01-31T15:28:35+00:00

tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:

- The tinfoil instance and its data proxies can't be pickled, so
  all interaction with the bitbake server has to run in the main
  script process and only processing of the plain scripts can
  be done with multiprocessing:

  _pickle.PicklingError: Can't pickle : attribute lookup TinfoilRecipeCacheAdapter on bb.tinfoil failed

- The multiprocessing pool has to be created before initializing
  tinfoil, otherwise the pool workers end up trying to communicate
  with the bitbake server during shutdown:

  ERROR: UI received SIGTERM
  Process ForkPoolWorker-2:
  Traceback (most recent call last):
    File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
      util._exit_function()
    File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
      _run_finalizers(0)
    ...
    File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
      assert self._parent_pid == os.getpid(), 'can only test a child process'
   AssertionError: can only test a child process

- func() needs to defined before creating the pool to avoid:

  AttributeError: Can't get attribute 'func' on 

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: explicitly shut down server

2017-01-31T15:28:35+00:00

Current tinfoil2 requires manually shutting down the server.
Without that, the script hangs during exit. This might change
in the future.

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: point out where to get checkbashisms.pl

2017-01-31T15:28:35+00:00

The current SourceForge project seems to be unmaintained (last release
2.0.0.2 from 2015) while the copy used by Debian is quite active (last
commit 2016-09-30).

Ideally, checkbashisms.pl should get installed automatically via a
recipe, but for now at least provide the link for manual installation.

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

verify-bashisms: fix typo

2017-01-31T15:28:35+00:00

Variable was renamed, it's now called "output".

Signed-off-by: Patrick Ohly 
Signed-off-by: Richard Purdie

scripts: remove True option to getVarFlag calls

2016-12-16T08:30:03+00:00

getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

Signed-off-by: Joshua Lock 
Signed-off-by: Ross Burton

scripts: add tool to scan for bashisms recipe shell scripts

2016-09-16T14:19:31+00:00

Shell functions in bitbake are executed with /bin/sh so should be POSIX
compliant and not use Bash extensions, or at least only use extensions that are
implemented in both dash and ash (busybox).

This tool will extract all of the shell scripts from all recipes and run them
through checkbashisms (it assumes that checkbashisms is on $PATH).

There is a whitelist to filter out false-positives such as the use of $HOSTNAME
(a bashism) in functions where we have defined it, or using the 'type' builtin
which is supported by ash/dash.

[ YOCTO #8851 ]

Signed-off-by: Ross Burton 
Signed-off-by: Richard Purdie