openembedded-core.git/meta/lib/oeqa/buildperf, branch uninative-1.5 Mirror of openembedded-core oeqa/buildperf: don't archive stdout/stderr of commands 2017-02-16T04:06:27+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-27T14:54:53+00:00 e004664287ec03e7367a7bf553d9a3038444e82e Stop capturing output of the shell commands into <test>/commands.log. Redirecting output into a file prevented the unittest framework from capturing it, causing useless errors (with empty output) like: oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']' returned non-zero exit status 1 with output: In general, the console output of commands is only interesting when something fails. Also, dropping the commands.log file is a huge saving in disk space, and thus, repository size when results are archived in Git. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Stop capturing output of the shell commands into <test>/commands.log.
Redirecting output into a file prevented the unittest framework from
capturing it, causing useless errors (with empty output) like:
oeqa.utils.CommandError: Command '['bitbake', 'core-image-sato']'
returned non-zero exit status 1 with output:

In general, the console output of commands is only interesting when
something fails. Also, dropping the commands.log file is a huge saving
in disk space, and thus, repository size when results are archived in
Git.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: reword test descriptions 2017-02-05T09:20:23+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-02-03T08:27:07+00:00 7e7252108d5e3503888efe0e603b3883fdc793e8 In an attempt to make them more compact and coherent. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
In an attempt to make them more compact and coherent.

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: store measurements as a dict (object) in the JSON report 2017-01-23T12:04:00+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-19T10:16:45+00:00 81065092f38c9631dcf5917d70a25809a21de5f4 Store measurements as a dict, instead of an array, in the JSON report. This change makes traversing of the report much easier. The change also disallows identically named measurements under one test, as a sanity check for the test cases. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Store measurements as a dict, instead of an array, in the JSON report.
This change makes traversing of the report much easier. The change also
disallows identically named measurements under one test, as a sanity
check for the test cases.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: change sorting in json report 2017-01-23T12:04:00+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2017-01-13T14:00:37+00:00 75e8aec0e0d81888be47b35c3c84df73edb91868 Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for more logical sorting of the values in the report. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Use OrderedDict() instead of sort_keys=True (of json.dump()). Makes for
more logical sorting of the values in the report.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oe-build-perf-test: remove unused imports and fix indent 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-12-28T15:04:02+00:00 0b1892fa9165407a156609ff7cb3708e21bacd8c [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oe-build-perf-test: save test metadata in a separate file 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-12-28T14:01:21+00:00 2036c646019660e32f1bc277fdec0cdbff0afdd4 The patch introduces a new metadata (.json or .xml) file in the output directory. All test meta data, e.g. git revision information and tester host information is now stored there. The JSON report format is slightly changed as the metadata is not present in results.json anymore. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
The patch introduces a new metadata (.json or .xml) file in the output
directory. All test meta data, e.g. git revision information and tester
host information is now stored there. The JSON report format is slightly
changed as the metadata is not present in results.json anymore.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: report results in chronological order 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-11-23T09:58:21+00:00 91ba6ea9fe2eb82f992a6516d7971b435e1cfd32 Write results in the report file in chronological order, instead of random order dependent on test statuses. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Write results in the report file in chronological order, instead of
random order dependent on test statuses.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: extend xml report format with test description 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-11-14T10:54:06+00:00 7c23ddfeb4a46ee519cafdbd83ad1880621fba4d Add test description as an attribute to the <testcase> element. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Add test description as an attribute to the <testcase> element.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa.buildperf: extend xml format to contain measurement data 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-10-28T07:19:45+00:00 b7164d30fb125ff0c85a2ea508b0f1801aa57f66 Make the xml report format slightly non-standard by incorporating measurement data into it. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Make the xml report format slightly non-standard by incorporating
measurement data into it.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
oe-build-perf-test: enable xml reporting 2017-01-23T12:03:59+00:00 Markus Lehtonen markus.lehtonen@linux.intel.com 2016-10-27T14:52:37+00:00 21ae1c491b93675254b7733640662b566ed76f98 Add --xml command line option to oe-build-perf-test script for producing a test report in JUnit XML format instead of JSON. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> 
Add --xml command line option to oe-build-perf-test script for producing
a test report in JUnit XML format instead of JSON.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>