By default python logging module uses stderr as default stream
for output but is unix-like to use stdout instead, so change it.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>

The new OEQA framework aims to re-use code into the different Test
components.

The previous oe-selftest implements it-self loading, run, and list test
cases in a non-standard way (unittest base) and other functionalities
like logging that is now on oeqa core. This ends on a compact oe-selftest
script.

All needed command line options was migrated but there are some of them
pending of implementation and others deprecated.

Deprecated options:

list-tags: The tag functionality into the old oeqa framework isn't
    work, the selftest doesn't has tag decorators.
{run, list}-tests-by: Ambiguos options it accepts all the posibilites module,
    class, name, id or tag.

Remaining to implement:

coverage: It enables covrage reports over a test run, currently isn't on
    on use and some bugs [1], i filed a bug to add support to OEQA core module in
    this way other Test components could enable it.
repository: It push XML results into a git repository and isn't in use,
    i filed a bug to implement this into OEQA core module. [2]

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>

The oe-selftest creates include files to store custom configuration to
make specific tests, every class executes a different test and may be
uses custom configuration.

So move to case class in order to simplify oe-selftest script and later
implement later a build folder per class.

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>

Setting SANITY_TESTED_DISTROS or PRSERV_HOST are known to break oe-selftest.
Rather than have the user experience this, refuse to execute unless the
environment is correct.

Ideally we'd try and unset these but that is a more invasive change and this
at least makes people aware of the problem.

[YOCTO #11292]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

adding the meta-selftest layer is a small bump you always encounter
when you want to run selftest. Adding an automatic procedure to include
the layer if it is not yet present.

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

buildhistory interfers with various tests so error if the user has it enabled,
hence avoiding time spent running the tests only for them to fail.

For example, if there is an sstate cache and something comes from that cache,
you can get a "version went backwards" warning from buildhistory which would
then change the outcome of a test. Its safer/easier to disable it.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

Make it possible to store any bitbake config variables in the metadata.
Config values will be stored under a new config element in the xml report:
    <config>
        <variable name="MACHINE">qemux86</variable>
    </config>

The value of MACHINE is moved there instead of having a dedicated
<machine> element.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>

Use the same format, based on /etc/os-release, as for host distro
information.

[YOCTO #10590]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>