| Age | Commit message (Collapse) | Author | Files |
|---|
|
We can't use subprocess.check_output() or subprocess.call() here since the one
who invokes runcmd() needs handle CmdError() exception (error out or ignore
it).
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* --no-signature saves unnecessary .patch modifications when executed on
host with different git version
* --no-numbered saves unnecessary .patch modifications when number of the
applied patches is changed (the number is still in the filename so the
order how they should be applied is still preserved)
* both options exist for very long time, I've tested them with git 1.9.1
from Ubuntu 14.04 and I'm quite sure they were available even in much
older releases, so there shouldn't be any issue on relatively new sanity
tested distros
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Ignoring patch context increases the chances of patches being
applied incorrectly. Depending on what code is being patched, this can go
completely unnoticed and create subtle bugs, sometimes with security implications.
Please see here for a specific example:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
On the other hand, we cannot simply force all patch context to match exactly:
doing this would break a lot of recipes suddenly, across all layers.
So let's try a softer approach: issue a warning, and gently update
patches over a longer span of time. When most of the warnings are eliminated,
we can start enforcing a strict patch application policy.
I do understand that this patch creates a lot of warnings all of a sudden, however
I believe the problem does need to be addressed. All of oe-core recipes have their
context already fixed.
Sample warning:
WARNING: vulkan-1.0.61.1-r0 do_patch:
Some of the context lines in patches were ignored. This can lead to incorrectly applied patches.
The context lines in the patches can be updated with devtool:
devtool modify <recipe>
devtool finish --force-patch-refresh <recipe> <layer_path>
Then the updated patches and the source tree (in devtool's workspace)
should be reviewed to make sure the patches apply in the correct place
and don't introduce duplicate lines (which can, and does happen
when some of the context is ignored).
Details:
Applying patch demos-Don-t-build-tri-or-cube.patch
patching file demos/CMakeLists.txt
Hunk #1 succeeded at 63 (offset 2 lines).
Hunk #2 succeeded at 76 with fuzz 1 (offset 2 lines).
[YOCTO #10450]
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
matching
Particularly, this was causing 'devtool modify' to erroneously add those
.orig files into commits. This was getting in the way, if the goal
was to amend/update those existing patches.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This module refers to oe.types, so it needs to actually import oe.types.
Fixes errors when parsing certain OE-Core recipes within the layer
index update script.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bb.data.expand(x, d) is deprecated API.
[YOCTO #10678]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add .xz to the list of extensions recognized by patch_path(), so that
compressed patches ending in .patch.xz or .diff.xz are automatically
applied.
(From OE-Core rev: f1a2c45765d14d3ca09657ad1f6b526554af2bb6)
Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Move patch_path(), src_patches() and should_apply() to oe.patch, making
them easier to call from elsewhere (particularly across the
UI/server boundary).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If a patch applied by a recipe has no header and we turn the recipe's
source into a git tree (when PATCHTOOL = "git" or when using devtool
extract / modify / upgrade), the commit message ends up consisting only
of the original filename marker ("%% original patch: filename.patch").
When we come to do turn the commits back into a set of patches in
extractPatches(), this first line ends up in the "Subject: " part of
the file, but we were ignoring it because the line didn't start with the
marker text. The end result was we weren't able to get the original
patch name. Strip off any "Subject [PATCH x/y]" part before looking for
the marker text to fix.
This caused "devtool modify openssl" followed by "devtool update-recipe
openssl" (without any changes in-between) to remove version-script.patch
because that patch has no header and we weren't able to determine the
original filename.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When PATCHTOOL = "git", if we need to manually apply a patch and then
commit it (i.e. when git am doesn't work) we try to extract the author /
date / shortlog from the patch header. Make the following improvements
to that extraction process:
* If there's no explicit Subject: but the first line is followed by a
blank line, isn't an Upstream-Status: or Index: marker and isn't too
long, then assume it's good enough to be the shortlog. This avoids
having too many patches with "Upgrade to version x.y" as the shortlog
(since that is often when patches get added).
* Add --follow to the command we use to find the commit that added the
patch, so we mostly get the commit that added the patch rather than
getting stuck on upgrade commits that last moved/renamed the patch
* Populate the date from the commit that added the patch if we were able
to get the author but not the date from the patch (otherwise you get
today's date which is less useful).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you leave "From <hash>" lines in the commit message it can actually
break git rebase because it tries to interpret the line in the context
of the current repository, and if the hash is invalid then a rebase
will blow up with:
fatal: git cat-file: could not get object info
or in newer git versions:
error: unable to find <hash>
fatal: git cat-file <hash>: bad file
(I hit this when I tried to do a devtool upgrade on openssl to 1.0.2i
the first time I did "git rebase --skip")
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When extracting patches from a git repository with PATCHTOOL = "git" we
cannot assume that all patches will be UTF-8 formatted, so as with other
places in this module, try latin-1 if utf-8 fails.
This fixes UnicodeDecodeError running devtool update-recipe or devtool
finish on the openssl recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With Python 3, the encoding of a file is significant; several recipes in
OE-Core have patches which are not fully utf-8 decodable e.g. man,
lrzsz, and gstreamer1.0-libav, leading to errors when using devtool's
modify, upgrade or extract subcommands on these recipes. To work around
this, try reading the patch file as utf-8 first and if that fails try
latin-1 before giving up.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
|
|
The syntax for octal values changed in python3, adapt to it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #9344]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Show the actual command that failed when raising a CmdError. Makes
figuring out what actually failed much easier.
[YOCTO #9344]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than modifying files in .git/hooks, which can be read-only
(e.g., if it is a link to a directory in /usr/share), move away the
entire .git/hooks directory temporarily.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
For patches that we have to extract the header information by hand (i.e.
will not apply with "git am"), make the following improvements:
* If we can't extract author/date/subject, then try to do so from the
commit that added the patch in git (assuming the metadata is tracked
by git)
* Take only first Signed-off-by line instead of last
* Accept any case for "Signed-off-by" in case author has typed it by
hand
* Improve conditional - we can skip the other cases if one matches
Implements [YOCTO #7624].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Makes it possible to define which paths are included in the patches.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
|
|
Unfortunately it appears that under certain circumstances, a failed
git am followed by git am --abort won't clean up any changes the patch
might have made - this was seen when running "devtool extract" on the
unzip recipe; unzip-6.0_overflow3.diff has a malformed date as far as
git am is concerned but it triggers this condition. Add a
git reset --hard HEAD followed by git clean -f in order to recover from
this scenario.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running "git add" on a directory with older git versions, deleted
files aren't added to what is to be committed unless you explicitly
specify the -A option. The result of this is that when applying a patch
from a recipe which doesn't apply with "git am" (and we fall back to
applying the patch through other means then "git add" following by
a "git commit") these deletes weren't committed with the patch, leaving
them sitting deleted but not committed at the end.
This should fix test_devtool_modify_native (which unwittingly exercises
this scenario by attempting to run "devtool modify" on apt-native) on
machines with older git versions.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Similarly to:
http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/lib/oe/patch.py?id=f205ccaf48ac36f4b26efc4aeb2e9d2939b28646
we need to fix patch application for source which is in a subdirectory.
Passing "." as the git directory or work-dir appears to work (or is ignored)
in some versions of git but does not work in others, probably quite correctly.
Since we have reporoot from the above patch, pass this in directly.
This bug caused this sanity test failure on some machines:
FAIL: test_devtool_modify_git (oeqa.selftest.devtool.DevtoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/devtool.py", line 390, in test_devtool_modify_git
self.assertEqual(result.output.strip(), "", 'Created git repo is not clean')
AssertionError: '?? util/mkelfImage/patches/' != '' : Created git repo is not clean
since git apply would fail, it would then fall back to quilt
and the git tree would be left unclean.
[YOCTO #7911]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Locating which recipe provides a file in an image that you want to
modify and then figuring out how to bbappend the recipe in order to
replace it can be a tedious process. Thus, add a new appendfile
subcommand to recipetool, providing the ability to create a bbappend
file to add/replace any file in the target system. Without the -r
option, it will search for the recipe packaging the specified file
(using pkgdata from previously built recipes). The bbappend will be
created at the appropriate path within the specified layer directory
(which may or may not be in your bblayers.conf) or if one already exists
it will be updated appropriately.
Fairly extensive oe-selftest tests are also provided.
Implements [YOCTO #6447].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
with open(...)... is preferred for reading/writing files as it is neater
and takes care of closing the file for you.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For recipes that have their actual source in a subdirectory of what is
fetched (e.g. mkelfimage), we need to find the root of the repository
within the GitApplyTree code that attempts to set up the required git
hooks and use that, rather than expecting the root to be the same as
${S}.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When patches from a recipe have been written out to a git tree, we also
want to be able to do the reverse so we can update the patches next to
the recipe. This is implemented by adding a comment to each commit
message (using git hooks) which we can extract later on.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we don't do this, you may still be in the git am resolution mode at
the end of applying patches, which is not desirable.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Preserving carriage returns is important where the patch contains them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When PATCHTOOL = "git", if we're not able to use "git am" to apply a
patch and fall back to "git apply" or "patch", it is desirable to
actually commit the changes, attempting to preserve (and interpret) the
patch header as part of the commit message if present. As a bonus, the
code for extracting the commit message is callable externally in case it
is useful elsewhere.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When PATCHTOOL = "git", git apply doesn't support fuzzy application, so
if a patch requires that it's better to be able to apply it rather than
just failing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is better to use "git am" when possible to preserve the commit messages and
the mail format in general for patches when those are present. A typical use
case is when developers would like to keep the changes on top of the latest
upstream, and they may occasionally need to rebase. This is not possible with
"git diff" and "diff" generated patches.
Since this is not always the case, the fallback would be the "git apply"
operation which is currently available.
Signed-off-by: Laszlo Papp <lpapp@kde.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets
add a wrapper and have this work in both worlds.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The commands module is removed in python3. Use the subprocess module instead
and the pipes module to replace the mkargs usage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update older code to use modern exception handling syntax which
is the form accepted by python 3.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Under the scenario where you have an existing source tree and you then
change one of the patches, maybe to be architecture or machine specific,
then rebuild, the build will fail since the symlink already exists
but should now point at a different file.
The easiest fix is to tell the system to remove and recreate the link
which is done with the force option.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
The bb and os modules are always imported so having these extra import calls
are a waste of space/execution time. They also set a bad example for people
copy and pasting code so clean them up.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using "1" with getVar is bad coding style and "True" is preferred.
This patch is a sed over the meta directory of the form:
sed \
-e 's:\(\.getVar([^,()]*, \)1 *):\1True):g' \
-e 's:\(\.getVarFlag([^,()]*, [^,()]*, \)1 *):\1True):g' \
-i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if PATCHRESOLVE is user and and PatchTree() is being used, you can
get backtraces if patch application fails. This is because even in the failure
case, self._current is incremented, meaning second time around, there are array
range issues.
This patch changes the code so _current is only incremented upon successful
patch application, thereby resolving this failure.
Secondly, if you bitbake -c patch -f a recipe using PatchTree(), the
clean method was unimplemented leading to patch failures.
The other part of this patch changes the logic so a series file and
set of applied patches are maintained in a quilt like fashion. This
means a the Clean method can be implemented correctly and rerunning
the patch task of an existing patches source now works reliably.
[YOCTO #2043 partially]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't need to see a Python stack backtrace when a patch does not
apply, just the error output from patch, so trap these kinds of errors
and ensure that we display the message and fail the task and nothing
else.
Fixes [YOCTO #1143]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the result of running the following over the metadata:
sed \
-e 's:bb.data.\(setVar([^,()]*,[^,()]*\), *\([^ )]*\) *):\2.\1):g' \
-e 's:bb.data.\(setVarFlag([^,()]*,[^,()]*,[^,()]*\), *\([^) ]*\) *):\2.\1):g' \
-e 's:bb.data.\(getVar([^,()]*\), *\([^(), ]*\) *,\([^)]*\)):\2.\1,\3):g' \
-e 's:bb.data.\(getVarFlag([^,()]*,[^,()]*\), *\([^(), ]*\) *,\([^)]*\)):\2.\1,\3):g' \
-e 's:bb.data.\(getVarFlag([^,()]*,[^,()]*\), *\([^() ]*\) *):\2.\1):g' \
-e 's:bb.data.\(getVar([^,()]*\), *\([^) ]*\) *):\2.\1):g' \
-i `grep -ril bb.data *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Unfortunately we can't access oe_terminal directly from patch.py
so we have to pass in the correct terminal function pointer.
[YOCTO #1587]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The dynamic message domain was introduced by Richard Purdie with the following patch:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=a6c48298b17e6a5844b3638b422fe226e3b67b89
Signed-off-by: Samuel Stirtzel <s.stirtzel@googlemail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if there is a failed do_patch the series files get appended
so if there were two patches
a.patch
b.patch
and b.patch failed during next run it would append a.patch again
a.patch
b.patch
a.patch
and this would keep growing.
We can remove series file in Clean() because we populate it in Import()
anyway
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Acked-by: Chris Larson <chris_larson@mentor.com>
Acked-by: Eric Bénard <eric@eukrea.com>
|
|
bb.mkdirhier should be bb.utils.mkdirhier
Signed-off-by: Khem Raj <raj.khem@gmail.com>
|
|
Signed-off-by: Chris Larson <chris_larson@mentor.com>
|
