summaryrefslogtreecommitdiff
path: root/bitbake/lib
AgeCommit message (Collapse)AuthorFiles
2011-01-04Fix bitbake -k issue introduced by build exception cleanupChris Larson1
A SystemExit from a python function wasn't being raised as a FuncFailed, which resulted in it not being caught by the exception handlers in the runqueue for the worker process, which resulted in a SystemExit exit, rather than os._exit, which causes all manner of problems when used in a forked process. This fixes it by ensuring we raise a FuncFailed when seeing exceptions which aren't instances of Exception. (Bitbake rev: dafe92fe9f387450d9f9e9ff41c99388998b7495) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Don't show tracebacks for SystemExit or KeyboardInterruptChris Larson1
(Bitbake rev: d71984b3934c3dd9791c3bc00f332b79a1985a05) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix exit code display for task failureChris Larson1
Per the python documentation, os.waitpid returns the exitcode shifted up by 8 bits, and we weren't compensating, resulting in a display of 'failed with 256' when a worker process exits with a code of 1. (Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04FIXUP: runqueue bb.debug refsRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Simplify build exception handlingChris Larson3
- Drop EventException - Use FuncFailed as the primary function failure exception, using TaskFailed for the event (leaving it up to the process running exec_{func,task} to display the more detailed information available in the exception). - Switch InvalidTask to an exception rather than an event, as that's a critical issue. - Reduce the number of messages shown to the user when a task fails -- they don't need to be told it fails 12 times. Work remains in this area though. (Bitbake rev: 06b742aae2b8013cbb269cc30554cff89e3a5667) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Turn Event into a new style classChris Larson1
(Bitbake rev: b42221cabeb1193ade134d1d3c0318203ab8eb93) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Work around issue with pickling of traceback objectsChris Larson1
(Bitbake rev: 82928613256bad92fde9f4071244a53e20fc89ee) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Apply fix for issue where the loggers were constructed with the wrong classChris Larson1
(Bitbake rev: 1b21daf052c49f3126dac001712ec01ad63c5f60) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix providerlog NameErrorChris Larson1
(Bitbake rev: 992e460f24d4da707c76d6e6d74d3684c9646279) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Drop 'fn' arguments to bb.msg functionsChris Larson1
(Bitbake rev: 8341458e3d21b45db84e46bd32f8ad270000ce3c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Add pending deprecation warnings to the bb.msg functionsChris Larson1
(Bitbake rev: 60293a42b5500b6139bcd912bf294f862ef9936b) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Switch bitbake internals to use logging directly rather than bb.msgChris Larson27
We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Use logging in the knotty ui, and pass the log record across directlyChris Larson3
This kills firing of Msg* events in favor of just passing along LogRecord objects. These objects hold more than just level and message, but can also have exception information, so the UI can decide what to do with that. As an aside, when using the 'none' server, this results in the log messages in the server being displayed directly via the logging module and the UI's handler, rather than going through the server's event queue. As a result of doing it this way, we have to override the event handlers of the base logger when spawning a worker process, to ensure they log via events rather than directly. (Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Use the python logging module under the hood for bb.msgChris Larson5
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Allow %20 in a file name in the SRC_URIC Michael Sundius1
(Bitbake rev: f7c181a0f6ab0b4d33bf80a0e24a788de441f82b) Signed-off-by: C Michael Sundius <msundius@sundius.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Make DataSmart inherit the MutableMapping ABCChris Larson1
Provide __len__, __iter__, and the getitem/setitem/delitem methods, and its mixed in versions of keys(), values(), items(), etc will automatically behave, making the DataSmart act more like a real mapping. (Bitbake rev: 89b5351c656d263b0ce513cee043bc046d20a01e) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Split out 'find next buildable task' into a separate generator functionChris Larson1
It needs to be a generator, so scheduler subclasses have the option to skip buildable tasks and return a later one. (Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Let the RunQueueScheduler control the number of active tasksChris Larson1
(Bitbake rev: 1387423e747f59866fd1cb99a7d90605e668823f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Add the ability to use runqueue schedulers from the metadataChris Larson1
If you create a runqueue scheduler class in a python module, available in the usual python search path, you can now make it available to bitbake via the BB_SCHEDULERS variable, and the user can then select it as they select any other scheduler. Example usage: In a test.py I placed appropriately: import bb.runqueue class TestScheduler(bb.runqueue.RunQueueScheduler): name = "myscheduler" In local.conf, to make it available and select it: BB_SCHEDULERS = "test.TestScheduler" BB_SCHEDULER = "myscheduler" (Bitbake rev: 4dd38d5cfb80f9bb72bc41a629c3320b38f7314d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix IndexError encountered when a 'depends' flag references a nonexistant taskChris Larson1
(Bitbake rev: 4b0fd70539e73d99282fa89d47ad2d5f642ca4f4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Make realfile optional for better_exec, using the existing fn in the code objectChris Larson1
(Bitbake rev: 499a2d28d578cdd6df7cd30ccb79cc2b2796fb65) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-31When shutting down, send SIGTERM, not SIGINTChris Larson1
SIGINT should be from the user, not a script. It also doesn't work as reliably to shut down processes, as it's not always interpreted as a termination request. In addition, it causes KeyboardInterrupt exceptions in the worker processes, which can interfere with our exception handling. (Bitbake rev: e5f6e0e9de4c6d1dfdd269d2bf7f83c00c415a27) Signed-off-by: Chris Larson <clarson@kergoth.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-30bb.utils: check if lock file is writable, to fix bug 606Yu Ke1
Bug 606 report that if $DL_DIR is read-only, do_fetch will simply hang without any error message. The root cause is that: bb.fetch.go()->bb.utils.lockfile() will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR} is read-only, it will cause IOError exception. Although lockfile() can catch the exception, currently code simply ignore all the exception and continue the loop. it make sense if the exception is caused by locking contention, but in the read-only $DL_DIR case, it cause endless waiting unfortunately. So this patch add read-only check for lockfile to avoid the silent hang. Fix [BUGID #606] Signed-off-by: Yu Ke <ke.yu@intel.com>
2010-12-22bitbake/depexp: fix early exitJoshua Lock1
Stupdi typo/thinko from me had depexp exiting once recipes had parsed as I'd used a return the while loop where I'd meant a continue... Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-21bitbake/fetch: Fix fetch errorRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21runqueue.py: set BB_TASKHASH approriately after moving to fork()Kevin Tian1
In previous exec() model, cooker is re-initialized from scratch with environmental variable exported accordingly. Now in fork() model, environmental variables are not exported again, and thus original method to export BB_TASKHASH doesn't apply now which breaks all sstate packages. Now we can set data variable directly instead. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-21bitbake/fetch: Only checksum downloads when they're first downloaded and a ↵Richard Purdie1
checksum is present Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21bitbake/runqueue.py: Somehow the python environment mapping is failing so do ↵Richard Purdie1
it manually Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-20bitbake/fetch: Add missing return so if a checksum isn't present, it isn't ↵Richard Purdie1
checked Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-20bitbake/fetch: Checksum validity fixesRichard Purdie1
If the checksum check failed, the .md5 stamp file would still have been created meaning subsequent builds would proceed with the corrupt file. Reorder the calls to avoid this. Also raise a specific error for the checksum not specified error case. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-20FetchData: add SRC_URI checksumYu Ke1
This patch add the per-recipe SRC_URI checksum verification. - SRC_URI format The format of SRC_URI checksum follow OE definition: 1. SRC_URI has single src SRC_URI = "http://some.domain/file.tar.gz" SRC_URI[md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" 2. SRC_URI has multiple src, every src need specify name SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \ http://some.domain/file2.tar.gz;name=name2 " SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" - SRC_URI checking invocation: the checksum checking is invoked in do_fetch phase, so it can be invoked manually by # bitbake -f -c fetch <recipe_name> if recipes has no SRC_URI checksum item, bitbake will show warning: " WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072" SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768" " thus recipe author can add it to recpie file after SRC_URI - control variable BB_STRICT_CHECKSUM when SRC_URI checksum is missing, this variable decide pass or not if BB_STRICT_CHECKSUM = "1", bitbake should fatal in this case, otherwise bitbake just pass Signed-off-by: Yu Ke <ke.yu@intel.com>
2010-12-20siggen.py: print taskname when seeing mismatched hash in cacheKevin Tian1
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-17bitbake Revert bitbake exec() and go back to fork() for performace wins ↵Richard Purdie2
(first draft) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17bitbake/runqueue: Revert previous setscene noexec changes and ensure ↵Richard Purdie1
setscene stamp exists for setscene noexec tasks Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17bitbake/runqueue: Clean up message log levelsRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-16siggen.py: make sure stamp directory existingKevin Tian1
or else "bitbake -S" from scratch may report "No such file or directory" error. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-16bitbake/runqueue.py: We should just failoutright in scenequeue for noexec ↵Richard Purdie1
tasks, not skip them so stamps are created Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-14runqueue.py: use correct task ID when checking validity of setscene tasksKevin Tian1
sstate hash validation is done at initialization of RunQueueExecuteScenequeue. However the index of 'valid' list returned from the validation doesn't correspond to setscene task ID. It's just an intermediate namespace between runqueue and sstate hash func. Use it as setscene task ID fully mess the flow. Previously this doesn't cause trouble because all setscene tasks are passed. Commit 58396a5d24c62710fd0a9f3780d84ac8a95d8e7c add 'noexec' concept to setscene tasks which grabs some tasks out of the list and thus trigger this problem Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly with a minimal build, even though existing sstate packages could accelerate them. there's another typo using wrong task ID in a debug message which further hide this issue Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-14bitbake: add optional expansion to getVarFlag()Paul Eggleton2
Add a parameter to getVarFlag() to auto-expand the value of the flag. This makes getVarFlag() more consistent with getVar(), and allows expansion of vardeps and vardepsexclude (which has been done in this commit). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2010-12-14bitbake: allow excluding dependencies of variablesPaul Eggleton1
Adds a vardepsexclude flag that can be used to exclude a dependency of a variable (the opposite of vardeps). This will allow the exclusion of variables from the hash generation much more selectively than blanket whitelisting using BB_HASHBASE_WHITELIST. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2010-12-09bitbake: Overhaul environment handlingRichard Purdie4
Currently, anything whitelisted in the environment makes it into the worker processes. This is undesireable and the worker environment should be as clean as possible. This patch adapts bitbake sosme variables are loaded into bitbake's datastore but not exported by default. Any variable can be exported by setting its export flag. Currently, this code only finalises the environment in he worker as doing so in the server means variables are unavailable in the worker. If we switch back to fork() calls instead of exec() this code will need revisting. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09bitbake/cooker: fix idle command processing in serversJoshua Lock1
idle command processing in each of the servers does not handle an explicit None return value, which means the goggle UI ends up repeatedly adding "Tasks Summary:" rows to the list. This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle. It may be that the correct way to fix this is to change the idle command processing in the servers. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/goggle: add a ProgressBar for parse progressJoshua Lock1
Construct a ProgressBar and pass it to the RunningBuild.handle_event() so that goggle users are notified of metadata parsing progress. UI's with status make users less nervous Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/goggle: automatically show most recently added messageJoshua Lock1
It seems likely that the user would want to view the most recently emitted messages so this patch sets the message dislay treeview to scroll to any newly added rows. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7Joshua Lock1
Python 2.7's library changes some of xmlrpclib's internal implementation such that interacting with a proxy to BitBakes SimpleXMLRPCServer would cause BitBake to crash. The issue was traced to changes in the xmlrpclib.Transport implementation and Python bug #8194 (http://bugs.python.org/issue8194). This patch introduces a workaround by create a subclass of xmlrpclib.Transport, which overrides the offending methods with the Python 2.6.6 implementation copy and pasted from the Python 2.6.6 xmlrpclib, and using this BBTransport implementation for both xmlrpclib.Server objects we create. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/crumbs: add optional pbar parameter to RunningBuild.handle_event()Joshua Lock1
Defaults to None, but if set will pass the ParseProgress sofar and total to pbar's update() method. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/depexp: Factor ProgressBar into a separate class in crumbs/Joshua Lock2
ProgressBar will be useful in other UI elements so make it it's own class. Make ProgressBar a subclass of gtk.Dialog, rather than gtk.Window, so that we can suggest the window manager parent the ProgressBar to the widget passed at as parent. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/goggle: interaction tweaksJoshua Lock1
Set the goggle window to a more sane default size (640x480) and hook up the close button. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/crumbs: fix the event name determinationJoshua Lock1
Due to some recent change *somewhere* we need to explicitly look at the name attribute on the instances class, rather than the name attribute of the instance. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07bitbake/crumbs: do the test for ignored messages soonerJoshua Lock1
Move the test for ignored messages to the start of the message handling loop to avoid doing work for messages which are only going to be ignored. Signed-off-by: Joshua Lock <josh@linux.intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-01-02 22:53:35 +0000