openembedded-core.git/scripts/contrib/bb-perf, branch master

bb-perf: plot histograms base on buildstats data

2016-11-30T15:47:09+00:00

Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:

    * without -S: Produces a histogram listing top N recipes/tasks versus
      stats. The first stat defined in the -s parameter is the one taken
      into account for ranking
    * -S: Produces a histogram listing tasks versus stats.  In this case,
      the value of each stat is the sum for that particular stat in all recipes found.
      Stats values  are in descending order defined by the first stat defined on -s

EXAMPLES

1. Top recipes' tasks taking into account utime

    $ buildstats-plot.sh -s utime | gnuplot -p

2. Tasks versus utime:stime

    $ buildstats-plot.sh -s utime:stime -S | gnuplot -p

3. Tasks versus IO write_bytes:IO read_bytes

    $ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p

Signed-off-by: Leonardo Sandoval 
Signed-off-by: Ross Burton

scripts: Specify the stats to take into account

2016-11-23T11:02:30+00:00

There are many more stats on buildstats that 'Elapsed time', so make the script
more flexible to support all stats. Some cmd line examples:

$ buildstats.sh -s 'utime'

Buildstats' data covers proc's stats in different areas, including CPU times,
IO, program system resources and child program system resources. In order
to print values on each of these sets from command line, one can use the
following:

$ buildstats.sh -H -s 'TIME' | less

$ buildstats.sh -H -s 'IO' | less

and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system
resources.

Signed-off-by: Leonardo Sandoval 
Signed-off-by: Ross Burton

buildstats: Outputs 'task recipe elapsed-time' from each buildstats' recipe

2015-09-12T21:47:06+00:00

Given a 'buildstats' path (created by bitbake when setting
USER_CLASSES ?= "buildstats" on local.conf) and task names, outputs
'  ' for all recipes. Elapsed times are in
seconds, and task should be given without the 'do_' prefix.

Some useful pipelines

1. Tasks with largest elapsed times

    $ buildstats.sh -b  | sort -k3 -n -r | head
    do_compile perl-5.20.0-r1 221.82
    do_configure gettext-native-0.19.4-r0 140.34
    do_compile openssl-native-1.0.2a-r0 107.48
    do_compile openssl-1.0.2a-r0 102.10
    do_configure perl-native-5.20.0-r0 90.70
    do_configure gettext-0.19.4-r0 88.17
    do_compile gcc-cross-i586-4.9.2-r0 83.98
    do_configure m4-native-1.4.17-r0 83.44
    do_compile qemu-native-2.2.0-r1 71.69
    do_compile glibc-2.21-r0 60.88

2. Min, max, sum per task

    $ buildstats.sh | datamash -t' ' -g1 min 3 max 3 sum 3 | sort -k4 -n -r
    do_configure 0.03 140.34 1968.66
    do_compile 0.01 221.82 1664.44
    do_install 0.03 40.31 330.45
    do_populate_sysroot 0.11 34.45 229.23
    do_unpack 0.01 36.1 193.54
    do_patch 0.01 9.2 62.07
    do_fetch 0.01 6.66 32.13
    do_populate_lic 0.09 1.65 30.7

Signed-off-by: Leonardo Sandoval 
Signed-off-by: Ross Burton

bb-matrix-plot: Use interpolation for sparse data

2013-09-12T15:48:38+00:00

If not every combination of BB_NUMBER_THREADS and PARALLEL_MAKE have
been tested by bb-matrix.sh, e.g., by using BB_RANGE="04 08 10 12 16"
and PM_RANGE="04 08 10 12 16", then the graph that gnuplot generates by
default looks very jagged due to the missing data points. By using
splines to interpolate the missing data the graph looks a lot better.

This should not change graphs where all data points are available in any
way, only improve sparse graphs.

Signed-off-by: Peter Kjellerstedt 
Signed-off-by: Richard Purdie

bb-matrix: Clean before, rather than after, building

2013-09-12T15:48:38+00:00

This makes sure the the first build starts from a clean state. Otherwise
one could have the first build affected by any leftover state from
a previous build.

This also leaves a working state behind after the final build.

Signed-off-by: Peter Kjellerstedt 
Signed-off-by: Richard Purdie

bb-matrix: Fix min and max calculations

2013-01-21T10:20:21+00:00

The BB and PM ranges were originally intended to use leading 0s to
ensure all the values were the same string length, making for nice log
filenames and columnar dat files. However, not everyone will do this -
especially if it isn't documented.

Document the intent. Make the generation and parsing of dat files robust
to either method.

Signed-off-by: Darren Hart 
Signed-off-by: Saul Wold 
Signed-off-by: Richard Purdie

bb-matrix: correct BB and PM number canonicalization

2011-07-14T21:22:59+00:00

The bash string operation ${BB##*0} was greedy and in addition to converting
"02" to "2", also converted "20" to "", causing all builds for a BB value ending
in 0 to run with BB_NUMBER_THREADS=1.

Signed-off-by: Darren Hart 
Signed-off-by: Richard Purdie

bb-matrix: initial scripts to record TIME(1) metrics for BB and PM combinations

2011-07-12T14:06:04+00:00

The bb-matrix.sh script will run a bitbake command, building core-image-minimal
by default, for various combinations of BB_NUMBER_THREADS and PARALLEL_MAKE. It
records all relevant metrics of the TIME(1) command for each combination in a
data file.

The bb-matrix-plot.sh script can be used to visualize each of these metrics via
a 3d surface plot, either solid surface or wireframe with a value-map
projection on the XY plane.

Signed-off-by: Darren Hart 
Signed-off-by: Richard Purdie