openembedded-core.git/meta/lib/oeqa/buildperf, branch master-next2 Mirror of openembedded-core oeqa.buildperf: limit the length of error output 2017-03-26T12:17:03+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-03-24T14:43:25+00:00 3f1996cb016713295edf35edc32dd5e84888a5c7 Limit the length of error logs to 40 lines. We don't need to show/archive thousands of lines of bitbake logs if an error occurs. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Limit the length of error logs to 40 lines. We don't need to
show/archive thousands of lines of bitbake logs if an error occurs.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-build-perf-test: sum rusage in buildstats 2017-03-26T12:17:03+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-03-24T14:17:26+00:00 70c41bb721c00ed2abbb88d273eebc3a8bb01f5d Instead of separate rusage and child rusage values, only store their sum value in buildstats. This is a big reduction in data footprint without really losing any interesting data. Also, utilize OrderedDict to order data more logically. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Instead of separate rusage and child rusage values, only store their sum
value in buildstats. This is a big reduction in data footprint without
really losing any interesting data.

Also, utilize OrderedDict to order data more logically.

[YOCTO #10582]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-build-perf-test: pack all buildstat in one file 2017-03-26T12:17:02+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-03-24T14:17:25+00:00 a7f2e8915db379021f3409ca640de5d3b054a830 Write out all buildstats into one big json file, instead of using multiple per-measurement files. Individual buildstats will be indexed using "<test_name>.<measurement_name>" as the key. Also, changes the per-testcase working directories into temporary directories that will be removed after test execution as there are no more per-testcase data files to store permanently. [YOCTO #10582] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 
Write out all buildstats into one big json file, instead of using
multiple per-measurement files. Individual buildstats will be indexed
using "<test_name>.<measurement_name>" as the key. Also, changes the
per-testcase working directories into temporary directories that will be
removed after test execution as there are no more per-testcase data files
to store permanently.

[YOCTO #10582]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeqa/buildperf: don't archive stdout/stderr of commands 2017-02-16T04:06:27+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-27T14:54:53+00:00 e004664287ec03e7367a7bf553d9a3038444e82e Stop capturing output of the shell commands into <test>/commands.log. Redirecting output into a file prevented the unittest framework from capturing it, causing useless errors (with empty output) like: oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']' returned non-zero exit status 1 with output: In general, the console output of commands is only interesting when something fails. Also, dropping the commands.log file is a huge saving in disk space, and thus, repository size when results are archived in Git. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Stop capturing output of the shell commands into <test>/commands.log.
Redirecting output into a file prevented the unittest framework from
capturing it, causing useless errors (with empty output) like:
oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']'
returned non-zero exit status 1 with output:

In general, the console output of commands is only interesting when
something fails. Also, dropping the commands.log file is a huge saving
in disk space, and thus, repository size when results are archived in
Git.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: reword test descriptions 2017-02-05T09:20:23+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-02-03T08:27:07+00:00 7e7252108d5e3503888efe0e603b3883fdc793e8 In an attempt to make them more compact and coherent. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
In an attempt to make them more compact and coherent.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: store measurements as a dict (object) in the JSON report 2017-01-23T12:04:00+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-19T10:16:45+00:00 81065092f38c9631dcf5917d70a25809a21de5f4 Store measurements as a dict, instead of an array, in the JSON report. This change makes traversing of the report much easier. The change also disallows identically named measurements under one test, as a sanity check for the test cases. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Store measurements as a dict, instead of an array, in the JSON report.
This change makes traversing of the report much easier. The change also
disallows identically named measurements under one test, as a sanity
check for the test cases.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: change sorting in json report 2017-01-23T12:04:00+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-13T14:00:37+00:00 75e8aec0e0d81888be47b35c3c84df73edb91868 Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for more logical sorting of the values in the report. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for
more logical sorting of the values in the report.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oe-build-perf-test: remove unused imports and fix indent 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-12-28T15:04:02+00:00 0b1892fa9165407a156609ff7cb3708e21bacd8c [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oe-build-perf-test: save test metadata in a separate file 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-12-28T14:01:21+00:00 2036c646019660e32f1bc277fdec0cdbff0afdd4 The patch introduces a new metadata (.json or .xml) file in the output directory. All test meta data, e.g. git revision information and tester host information is now stored there. The JSON report format is slightly changed as the metadata is not present in results.json anymore. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
The patch introduces a new metadata (.json or .xml) file in the output
directory. All test meta data, e.g. git revision information and tester
host information is now stored there. The JSON report format is slightly
changed as the metadata is not present in results.json anymore.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: report results in chronological order 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-11-23T09:58:21+00:00 91ba6ea9fe2eb82f992a6516d7971b435e1cfd32 Write results in the report file in chronological order, instead of random order dependent on test statuses. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Write results in the report file in chronological order, instead of
random order dependent on test statuses.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>