summaryrefslogtreecommitdiff
path: root/bitbake/lib/bb/runqueue.py
AgeCommit message (Collapse)AuthorFiles
2011-01-04bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie1
bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04runqueue: fix typo in documentation of calculate_task_weightsBernhard Reutner-Fischer1
(Bitbake rev: 5cde120003af97a5cf5c6eff2a02bb1480f9414b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04runqueue: use fewer newlines in error pathBernhard Reutner-Fischer1
(Bitbake rev: 7498466f0e42beea6f5f411209a892b636c7783c) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04logger usage cleanupChris Larson1
(Bitbake rev: 976e4f84a8147ad762442df7ff4820611a21d227) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Use os.devnull, not /dev/nullChris Larson1
(Bitbake rev: ce5ab0fc524a1c2c48c4c39d6fc8aae23019207b) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04bitbake: Transfer noexec runqueue messages to the UIRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Move the runq task start/failed messages to the UIChris Larson1
(Bitbake rev: ab831e867f09b47001cb8da2f8f060e04febf237) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04cooker: stop loading the cache for -bChris Larson1
Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Prefer xrange over range for small performance gain.Bob Foerster1
range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix exit code display for task failureChris Larson1
Per the python documentation, os.waitpid returns the exitcode shifted up by 8 bits, and we weren't compensating, resulting in a display of 'failed with 256' when a worker process exits with a code of 1. (Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04FIXUP: runqueue bb.debug refsRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Switch bitbake internals to use logging directly rather than bb.msgChris Larson1
We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Use logging in the knotty ui, and pass the log record across directlyChris Larson1
This kills firing of Msg* events in favor of just passing along LogRecord objects. These objects hold more than just level and message, but can also have exception information, so the UI can decide what to do with that. As an aside, when using the 'none' server, this results in the log messages in the server being displayed directly via the logging module and the UI's handler, rather than going through the server's event queue. As a result of doing it this way, we have to override the event handlers of the base logger when spawning a worker process, to ensure they log via events rather than directly. (Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Split out 'find next buildable task' into a separate generator functionChris Larson1
It needs to be a generator, so scheduler subclasses have the option to skip buildable tasks and return a later one. (Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Let the RunQueueScheduler control the number of active tasksChris Larson1
(Bitbake rev: 1387423e747f59866fd1cb99a7d90605e668823f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Add the ability to use runqueue schedulers from the metadataChris Larson1
If you create a runqueue scheduler class in a python module, available in the usual python search path, you can now make it available to bitbake via the BB_SCHEDULERS variable, and the user can then select it as they select any other scheduler. Example usage: In a test.py I placed appropriately: import bb.runqueue class TestScheduler(bb.runqueue.RunQueueScheduler): name = "myscheduler" In local.conf, to make it available and select it: BB_SCHEDULERS = "test.TestScheduler" BB_SCHEDULER = "myscheduler" (Bitbake rev: 4dd38d5cfb80f9bb72bc41a629c3320b38f7314d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix IndexError encountered when a 'depends' flag references a nonexistant taskChris Larson1
(Bitbake rev: 4b0fd70539e73d99282fa89d47ad2d5f642ca4f4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-31When shutting down, send SIGTERM, not SIGINTChris Larson1
SIGINT should be from the user, not a script. It also doesn't work as reliably to shut down processes, as it's not always interpreted as a termination request. In addition, it causes KeyboardInterrupt exceptions in the worker processes, which can interfere with our exception handling. (Bitbake rev: e5f6e0e9de4c6d1dfdd269d2bf7f83c00c415a27) Signed-off-by: Chris Larson <clarson@kergoth.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-21runqueue.py: set BB_TASKHASH approriately after moving to fork()Kevin Tian1
In previous exec() model, cooker is re-initialized from scratch with environmental variable exported accordingly. Now in fork() model, environmental variables are not exported again, and thus original method to export BB_TASKHASH doesn't apply now which breaks all sstate packages. Now we can set data variable directly instead. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-21bitbake/runqueue.py: Somehow the python environment mapping is failing so do ↵Richard Purdie1
it manually Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17bitbake Revert bitbake exec() and go back to fork() for performace wins ↵Richard Purdie1
(first draft) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17bitbake/runqueue: Revert previous setscene noexec changes and ensure ↵Richard Purdie1
setscene stamp exists for setscene noexec tasks Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17bitbake/runqueue: Clean up message log levelsRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-16bitbake/runqueue.py: We should just failoutright in scenequeue for noexec ↵Richard Purdie1
tasks, not skip them so stamps are created Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-14runqueue.py: use correct task ID when checking validity of setscene tasksKevin Tian1
sstate hash validation is done at initialization of RunQueueExecuteScenequeue. However the index of 'valid' list returned from the validation doesn't correspond to setscene task ID. It's just an intermediate namespace between runqueue and sstate hash func. Use it as setscene task ID fully mess the flow. Previously this doesn't cause trouble because all setscene tasks are passed. Commit 58396a5d24c62710fd0a9f3780d84ac8a95d8e7c add 'noexec' concept to setscene tasks which grabs some tasks out of the list and thus trigger this problem Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly with a minimal build, even though existing sstate packages could accelerate them. there's another typo using wrong task ID in a debug message which further hide this issue Signed-off-by: Kevin Tian <kevin.tian@intel.com>
2010-12-09bitbake: Overhaul environment handlingRichard Purdie1
Currently, anything whitelisted in the environment makes it into the worker processes. This is undesireable and the worker environment should be as clean as possible. This patch adapts bitbake sosme variables are loaded into bitbake's datastore but not exported by default. Any variable can be exported by setting its export flag. Currently, this code only finalises the environment in he worker as doing so in the server means variables are unavailable in the worker. If we switch back to fork() calls instead of exec() this code will need revisting. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28bitbake/runqueue: Teach scenequeue about noexec tasks (since they don't do ↵Richard Purdie1
anything we can assume they're always successful Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28bitbake/runqueue.py: Fix stamp handling for noexec tasksRichard Purdie1
This means the noexec messages are only shown once as the stamp files are now correctly created. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-07bitbake.runqueue.py: Fix no exec task handling to correct counter housekeepingRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06bitbake: Add support for 'noexec' tasksRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-06bitbake/siggen: Allow siggen code to control final stampfile formatRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22pseudo/fakeroot: Move the pseudo directory creation into bitbakeRichard Purdie1
If sstate was used to accelerate a build, the pseudo directory might not have been created leading to subsequent task failures. Also, sstate packages were not being installed under pseudo context meaning file permissions could have been lost. Fix these problems by creating a FAKEROOTDIRS variable which bitbake ensures exists before running tasks and running the appropriate setscene tasks under fakeroot context. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-14bitbake-runtask: Ensure logging settings take effect in the worker process ↵Richard Purdie1
(and pass verbose setting) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-12bitbake/runqueue: make the runtask program configurableJoshua Lock1
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-11bitbake-runtask: Ensure logging options are inherited from the parentRichard Purdie1
[BUGID #291] Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07bitbake/runqueue.py: Fix invalid variable reference fixing the -f option ↵Richard Purdie1
with setscene tasks Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05bitbake/sstate: Implement a lookup function to speed up setscene processingRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28bitbake: Pass task hash information to subprocessesRichard Purdie1
Pass task has informaiton to work processes, allowing full manipulation of the hash data in the task context allowing checksums to be usable. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13bitbake/runqueue: Remove now unneeded indentationRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13bitbake/runqueue.py: Optimise delay values to avoid unneeded delaysRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-08bitbake: save the initial PATH for use when starting a subprocessJoshua Lock1
It was possible for bitbake-runtime to be run against a semi-installed python-native resulting in tracebacks with ImportError's. To prevent this we stash the initial PATH in the BBConfiguration when bitbake is started and then set this in the env when launching bitbake-runtask through subprocesses Popen() call. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-03bitbake: Implement signaturesRichard Purdie1
Includes functionality to find out what changes between two different singature data dumps. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-25bitbake/runqueue.py: Ensure rqexe always exists and that empty task lists ↵Richard Purdie1
cause a graceful exit Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20bitbake: Correctly route events from the worker to the serverRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake/runqueue.py: Ensure the flush happens as late as possible to avoid ↵Richard Purdie1
data mixups Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake/runqueue.py: Fix runqueue UI issuesRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run ↵Richard Purdie1
shell and python under a fakeroot environment Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake: Initial scenequeue implementation (needs major fixes)Richard Purdie1
bitbake: scenequeue: Skip setscene if the underlying task already ran bitbake/setscene: Make sure uneeded dependencies are removed recursively Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake/runqueue.py: Factor task skipping code into a functionRichard Purdie1
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19bitbake/runqueue.py: Create RunQueueExecute and RunQueueExecuteTasks ↵Richard Purdie1
classes, further splitting up runqueue Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-05-30 14:19:03 +0000