summaryrefslogtreecommitdiff
path: root/bitbake/lib/bb/cooker.py
AgeCommit message (Collapse)AuthorFiles
2011-02-23bitbake: Remove in-tree versionKhem Raj1
Bitbake should be used by checking it out from its own repo Signed-off-by: Khem Raj <raj.khem@gmail.com>
2011-02-23bitbake/cooker: Fix parsing failure zombie problemRichard Purdie1
When parsing if a SystemExit event is triggered, it causes the parsing thread to exit and the main process hangs waiting for it to finish indefintely. Add code to catch BaseExceptions and raise these with the main process gracefully instead of just hanging indefinitely with zombie processes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-15bitbake/cooker: Ensure delays are accurately transfered to the idle loops ↵Richard Purdie1
from runqueue Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-07bitbake/cooker.py: Fix merge conflict with upstream which was preventing ↵Richard Purdie1
clean bitbake shutdown Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-04bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie1
bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Move LAYERDIR expansion hack into DataSmartChris Larson1
(Bitbake rev: 40778a6e9e82c7ea4673a74fc19574430fa63e8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Show the user progress when loading the cacheBob Foerster1
(Bitbake rev: bdd7813d8eecf7b6b636322e748ca6bf69118513) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: don't swallow recipe parse errorsChris Larson1
(Bitbake rev: 1f5d49ce64e5e1c7705edbfa3c8e19649c21edd2) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Rename command events, adjust compareRevisionsChris Larson1
- Moved the logic for comparing revisions from cooker into command - Removed 'Cooker' from the event names - Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to be a subclass of CommandExit (Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: add shutdown/stop methodsChris Larson1
(Bitbake rev: fc64eff03fe1f41e59128cb3df0ef2161e24a8cb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: merge cookerState and cookerActionChris Larson1
(Bitbake rev: c7c8945ef7ca9465312e630b7fa5f0a87ac8b6c7) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: use a pool, abort on first parse errorChris Larson1
(Bitbake rev: 9caf65e79f95fe0045e727391e974c4c1e7411ff) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix the <100 recipe progress fixChris Larson1
(Bitbake rev: 424428a764651183218f9cc93bc05496867aa5de) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: don't choke on <100 recipes to parseChris Larson1
(Bitbake rev: 2be5169a01e57bcfcb1f75cbd6bac3bf0f230edd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: no cached in progressbar and add ETAChris Larson1
Rather than updating the progress bar based on the recipe being processed (whether cached or parsed), consider only parsed recipes. This reduces the instability in progress rate introduced by the cached entries, and allows the ETA to be resurrected and be a bit more useful. (Bitbake rev: 618480f7739f6ae846f67a57bee5a78efb37839d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: pass back child exceptions to the serverChris Larson1
(Bitbake rev: 0f68f8bcd0e0aa944f76f88a4a85c9bcc1e42bee) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: ensure that the cache sync completesChris Larson1
Without explicitly joining the thread, it's possible for the process to end (e.g. after a bitbake -p) and kill off the thread without waiting for it to exit cleanly. So, register the thread join with atexit. (Bitbake rev: 97ce57e6f860d3e6f34cc7a603ed1eeac4f423d3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: get number of threads in constructorChris Larson1
(Bitbake rev: e7fd259d1528f48b0812fb7b0a54dd6d5d22f4a9) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: don't add info for skipped recipesChris Larson1
(Bitbake rev: ac596d55de4ad86e1cdc84dabfea1350dd54efd0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: show progress bar before initializing the cacheChris Larson1
This ensures that the time spent loading the cache from disk occurs with the progress bar up. Though the progress bar stays at 0% during this period, I think this is an improvement over the multi-second stall which occurred previously before the progress bar came up. Ideally, we'd integrate cache loading from disk into the progress display, but this is a first step. (Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: save progress chunk value (total/100)Chris Larson1
(Bitbake rev: 09333737cbeeb9875d938521ddcd519fc808bcc3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: stop loading the cache for -bChris Larson1
Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: don't fire unnecessary parse progress eventsBob Foerster1
(Bitbake rev: 7cf7d5f14405681496fced3640a50a20ef1acac1) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cache: sync the cache file to disk in the backgroundChris Larson1
This version uses a thread rather than a process, to avoid problems with waitpid handling. This gives slightly less overall build time reduction than the separate process for it did (this reduces a -c compile coreutils-native by about 3 seconds, while the process reduced it by 7 seconds), however this time is quite insignificant relative to a typical build. The biggest issue with non-backgrounded syncing is the perceived delay before work begins, and this resolves that without breaking anything, or so it seems. (Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Implement parallel parsing supportChris Larson1
This utilizes python's multiprocessing module. The default number of threads to be used is the same as the number of available processor cores, however, you can manually set this with the BB_NUMBER_PARSE_THREADS variable. (Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cache: create and use a RecipeInfo classChris Larson1
This class holds the particular pieces of information about a recipe which are needed for runqueue to do its job. By using it, I think we improve code clarity, reduce method sizes, reduce overuse of primitive types, and prepare for parallel parsing. In addition, this ditches the leaky abstraction whereby bb.cache attempted to hide the difference between cached data and a full recipe parse. This was a remnant from the way things used to be done, and the code using it had to know the difference anyway. If we choose to reimplement caching of the full recipes, we can do it in bb.parse, in a completely transparent way. (Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Prefer xrange over range for small performance gain.Bob Foerster1
range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Pre-explode rundeps/runrecs in CacheDataChris Larson1
(Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix GraphViz .dot output for rdepends and rrecsChris Larson1
Use bb.utils.explode_deps to break up the rdepends and rrecommends strings. This fixes the same issue which was fixed by a number of patches floating around, but uses explode_deps rather than regular expressions. (Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Resurrect display of failed filesChris Larson1
This was inadvertantly removed when trying to reduce the amount of duplicated information the user sees when a failure occurs. (Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix syntax issue and don't include 'd' in ui_queueChris Larson1
(Bitbake rev: bed8e09971dc577f5443ad3d89aa14634c54eb16) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Queue up events before the UI is spawnedChris Larson1
- Queue up any events fired to the UI before the UI exists - At exit, check if UIs exist, and if not, flush the queue of LogRecords to the console directly. - When establishing a connection from the UI to the server, flush the queue of events to the queue in the server connection, so the UI will receive them when it begins its event loop. (Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Simplify build exception handlingChris Larson1
- Drop EventException - Use FuncFailed as the primary function failure exception, using TaskFailed for the event (leaving it up to the process running exec_{func,task} to display the more detailed information available in the exception). - Switch InvalidTask to an exception rather than an event, as that's a critical issue. - Reduce the number of messages shown to the user when a task fails -- they don't need to be told it fails 12 times. Work remains in this area though. (Bitbake rev: 06b742aae2b8013cbb269cc30554cff89e3a5667) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Switch bitbake internals to use logging directly rather than bb.msgChris Larson1
We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09bitbake: Overhaul environment handlingRichard Purdie1
Currently, anything whitelisted in the environment makes it into the worker processes. This is undesireable and the worker environment should be as clean as possible. This patch adapts bitbake sosme variables are loaded into bitbake's datastore but not exported by default. Any variable can be exported by setting its export flag. Currently, this code only finalises the environment in he worker as doing so in the server means variables are unavailable in the worker. If we switch back to fork() calls instead of exec() this code will need revisting. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09bitbake/cooker: fix idle command processing in serversJoshua Lock1
idle command processing in each of the servers does not handle an explicit None return value, which means the goggle UI ends up repeatedly adding "Tasks Summary:" rows to the list. This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle. It may be that the correct way to fix this is to change the idle command processing in the servers. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-11-28bitbake: Simplfy pre and post cooker hooksRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13bitbake: Rewrite profiling code so its functional for both none and xmlrpc ↵Richard Purdie1
backends Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29bitbake/siggen: Ensure full signature data is not held unless needed, ↵Richard Purdie1
reducing memory consumption Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03bitbake: Implement signaturesRichard Purdie1
Includes functionality to find out what changes between two different singature data dumps. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31bitbake/codeparser: Implement persistent cacheRichard Purdie1
For a given input to this code, the output doesn't change to implement a persistent cache of the data to speed up parsing. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20bitbake/cooker.py: Don't init the fetcher in the worker caseRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run ↵Richard Purdie1
shell and python under a fakeroot environment Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake/cooker.py: Allow idle handlers to pass through a true valueRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-18bitbake: Split Runqueue into two classes, a data processor and the execution ↵Richard Purdie1
part Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-12bitbake/cooker: Make sure no locks are held before we start forking workersRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-04cooker.py: Fix case of -b option with a full filepathRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16bitbake: Add support for .bbappend files (see mailing lists for detais)Richard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16bitbake: Remove two layers of unnecessary functions from task executionRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02bitbake: Fix handling of empty variables in local config filesRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-02-06 01:01:41 +0000