Class TaskManager
Parallel task manager for shared-memory multiprocessor machines
This class provides a rather simple way to profit from shared-memory
multiprocessor machines by running several tasks in parallel. The calling
program decides how many execution threads should run at any given time,
and then feeds compute tasks to the task manager, who runs them as soon
as possible without exceeding the maximum number of threads.
The major limitation of this approach lies in Python's Global
Interpreter Lock. This effectively means that no more than one Python
thread can run at the same time. Consequently, parallelization can only
be achieved if the tasks to be parallelized spend significant time in C
extension modules that release the Global Interpreter Lock.
|
|
|
|
|
runTask(self,
function,
args)
Run a task as soon as processing capacity becomes available |
|
|
|
|
terminate(self)
Wait until all tasks have finished |
|
|
__init__(self,
nthreads)
(Constructor)
|
|
- Parameters:
nthreads (int) - the maximum number of compute threads that should run in
parallel. Note: This does not include the main thread which
generated and feeds the task manager!
|
runTask(self,
function,
args)
|
|
Run a task as soon as processing capacity becomes available
- Parameters:
function (callable) - the function that will be executed as the body of the taskargs (tuple) - the arguments that will be passed to function when it is called.
An additional argument will be added at the end: a lock object
that the task can use to get temporarily exclusive access to data
shared with other tasks.
|