| Age | Commit message (Collapse) | Author | Files |
|---|
|
The purpouse of this function was to check dependencies for building a
hybrid iso and build them using bitbake if not found. Calling bitbake in
this context means this wic plugin itself cannot be instrumented inside
bitbake recipes which is undesirable, the benefits of this are clear:
there is no need to maintain outside scripts to generate an iso using wic
and the isohybrid building logic can be further abstracted away into an
isohybrid.bbclass in the future which can be easily inherited or something
similar.
So remove the function and add all dependencies to NATIVE_RECIPES so that
wic can print useful errors when they're not built.
To automate building the isohybrid image dependencies, add the following
somewhere in your image build inheritence hierarcy (or maybe create a
bbclass in the future to do these sort of things automatically):
DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \
parted-native dosfstools-native mtools-native grub-efi-native"
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Adding http module from Python's standard library. This allow use
of the http module without installing all python-misc modules.
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Current parsing code can wrongly interpret arbitrary lines
that are of 'key=value' format as legitimate bitbake variables.
Implemented more strict parsing of key=value pairs using
regular expressions.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since the gummiboot project is no longer being maintained
and we are using systemd-boot as a replacement instead,
we can now clean up all remaining gummiboot files.
[YOCTO #10332]
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you delete the log file that the oe-selftest.log symlink points to
but not the symlink itself, because we were using os.path.exists() here
the code assumed that the symlink didn't exist when in fact it still
did. Use os.path.lexists() instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
After systemd-boot was introduced, its been tested for a while with no major
issues being found until now, this patch completely replaces all gummiboot
instances with systemd-boot ones, taking the next step into cleaning
up systemd-boot/gummiboot.
[YOCTO #10332]
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:
d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')
which I've corrected (they happend to work by luck).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its possible something (like bitbake/tinfoil2) may mess around with the
environment and using the enviroment as a global variable store isn't
particularly nice anyway.
This patch changes the BUILDDIR usages so that the environment isn't used
as a global store and a global variable is used instead. Whilst that
is still not perfect, it does avoid the current double and triple backtraces
we're seeing where tinfoil2/bitbake has trampled the enviroment leading
to failures of failures making debugging even harder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It may be necessary to override the parameters gathered for the qemu
invocation. For example, the qemux86 machine configuration sets "-vga
vmware", but when using OVMF as BIOS, only "-vga std" is supported.
By putting the parameters derived from custom runqemu parameters like
"qemuparams" after the parameters derived from the machine
configuration the user gets the possibility to override those.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Added kickstart file to produce partitioned image for
MPC8315 reference hardware.
[YOCTO #8719]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Added new option --fixed-size to wks. The option can be used to indicate
the exact size of a partition. The option cannot be added together with
--size, in which case an error will be raised. Other options that
influence automatic partition size (--extra-space, --overhead-factor),
if specifiec along with --fixed-size, will raise an error.
If it partition data is larger than the amount of space specified with
--fixed-size option wic will raise an error.
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.
(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Currently wic looks for wks files in
<layer dir>/scripts/lib/wic/canned-wks/ directories.
This path is too nested and doesn't look consistent with the
naming scheme of layer directories.
Added <layer>/wic directory to the list of paths
to look for wks files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
git module is not included into standard Python
library and therefore causes import errors on the systems
where PythonGit is not installed.
As git module only used in the code implementing --repository
functionality it's better to import git only in the scope
that requires it.
[YOCTO #10821]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When no --size is specified for the rootfs in the .wks, we want to obey the
rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE
and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result
in image construction failure, if the size determined by du was insufficient
to hold the files without the aforementioned extra space.
This fallback from --size to ROOTFS_SIZE was already implemented when
--rootfs-dir is specified in the .wks, but it did not occur otherwise, neither
when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used.
This made a certain amount of sense, as this fallback logic happened at such
a level that it wasn't able to identify which partitions were rootfs
partitions otherwise. Rather than doing it at that level, we can do it in
prepare_rootfs(), which is run by the rootfs source plugins.
Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now
both be applied when --size isn't specified in the .wks. A warning is added
about this, though a user won't see it unless wic fails or they examine the
do_image_wic log.
Fixes [YOCTO #10815]
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.
Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This will add support to use qemu from the running host,
with this is possible to put qemu-native in ASSUME_PROVIDED
variable.
By default it will try to get qemu from the build sysroot,
and only if it fails will try to use the host's qemu.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If we don't catch this then attempting to run devtool in non-memres mode
when bitbake is already running will produce a traceback instead of just
an error message.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.
(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The autotools code imports oe.package; we weren't experiencing a problem
with this probably due to OE itself adding that path previously.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some variables in pkgdata files have a package-name override. When
the bare variable can not be found, try with the override-variant.
PKGSIZE is one such variable, and already had special code to handle this.
Test included.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Show usage text if script is not sourced.
Tested in bash, zsh and dash.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Created usage output for oe-find-native-sysroot script.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Created usage output for oe-git-proxy script.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Made usage output of oepydevshell-internal.py to look
similar to the output of other oe scripts.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Created usage output for oe-setup-builddir script.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Made usage output of oe-setup-rpmrepo to look similar to the
output of other oe scripts.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Created usage output for oe-trim-schemas script.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Made usage output of oe-run-native to look similar to the
output of other oe scripts.
[YOCTO #10751]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
To allow recipetool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/recipetool directories and then
scripts/lib/recipetool and load only the first found.
The previous search and load loop would load all found plugins with the
ones found later replacing any found before.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
To allow devtool plugins in one layer to shadow another in a well
defined way, first search BBPATH/lib/devtool directories and then
scripts/lib/devool and load only the first found.
The previous search and load loop would load all found plugins with the
ones found later replacing any found before.
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Short variant of wic command line option --skip-build-check
is incorretly named -p. It's named -s in wic help and Yocto
documentation.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Don't worth bother with logical partition on MBR partition type (aka
msdos) if disk image generated by wic should have 4 partitions.
Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This new option allows to commit the result to a git repository,
along with the results it will add a metadata file for information
of the current selftest run, such as: hostname, machine, distro,
distro version, host version, and layers.
This implementation will have a branch per different hostname,
testing branch, and machine.
To use this feature use:
oe-selftest <options> --repository <repository_link>
[YOCTO #9954]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Yet another instance of us expecting a string back from subprocess when
in Python 3 what you get back is bytes. Just decode the output within
run_command() so we avoid this everywhere.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.
buildstats.bbclass only writes the reduced logs now, but support for
the full /proc files is kept around as reference.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The internal representation after parsing now matches exactly
what the drawing code needs, thus speeding up drawing a bit.
However, the main motivation is to store exactly that required
information in a more compact file.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This adds a new, separate chart showing the amount of disk space used
over time for each volume monitored during the build. The hight of the
graph entries represents the delta between current usage and minimal
usage during the build.
That's more useful than showing just the current usage, because then a
graph showing changes in the order of MBs in a volume that is several
GB large would be just flat.
The legend shows the maximum of those deltas, i.e. maximum amount of
space needed for the build. Minor caveat: sampling of disk space usage
starts a bit later than the initial task, so the displayed value may
be slightly lower than the actual amount of space needed because
sampling does not record the actual initial state.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When matching fails, m.group(0) is invalid and can't be used in the
error message.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The only real change is the addition of two if checks that skips the
corresponding drawing code when there is no data.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This enables rendering of the original bootchart charts for CPU, disk
and memory usage. It depends on the /proc samples recorded by the
updated buildstats.bbclass. Currently, empty charts CPU and disk usage
charts are drawn if that data is not present; the memory chart already
gets skipped when there's no data, which will also have to be added
for the other two.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The code did not handle x scaling correctly when drawing starts at
some time larger than zero, i.e. it worked for normal bootchart data,
but not for the system statistics recorded by buildstats.bbclass.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Substracting curr_y when determining the hight of the process chart is
wrong because the height is independent of the position where the
chart is about to be drawn. It happens to work at the moment because
curr_y is always 10 when render_processes_chart() gets called. But it
leads to a negative height when other charts are drawn above it, and
then the grid gets drawn on top of those other charts.
Substracting some constant is relevant because otherwise the box is
slightly larger than the process bars. Not sure exactly where that
comes from (text height?); leg_s seems a suitable constant and happens
to be 10, so everything still gets rendered exactly as before.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When creating a patch set with cover letter using the
send-pull-request script, both the "In-Reply-To" and "References"
headers are appended twice in patch 2 and subsequent.
That's because git-format-patch already inserted them and then
git-send-email repeats that. Suppressing mail threading in
git-send-email with --no-thread avoids the problem and is the
right solution because it works regardless whether git-send-email is
called once or twicee.
Repeating these headers is a violation of RFC 2822 and can confuse
mail programs. For example, Patchwork does not detect a patch series
problem when there are these extra headers.
[YOCTO #10718]
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Scripts that produces script data to be consumed by gnuplot.
There are two possible plots depending if either the
-S parameter is present or not:
* without -S: Produces a histogram listing top N recipes/tasks versus
stats. The first stat defined in the -s parameter is the one taken
into account for ranking
* -S: Produces a histogram listing tasks versus stats. In this case,
the value of each stat is the sum for that particular stat in all recipes found.
Stats values are in descending order defined by the first stat defined on -s
EXAMPLES
1. Top recipes' tasks taking into account utime
$ buildstats-plot.sh -s utime | gnuplot -p
2. Tasks versus utime:stime
$ buildstats-plot.sh -s utime:stime -S | gnuplot -p
3. Tasks versus IO write_bytes:IO read_bytes
$ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
