Package Scientific :: Package DistributedComputing :: Module TaskManager :: Class TaskManager

Class TaskManager

Pyro.core.ObjBase --+
                    |
                   TaskManager

Manager for computational tasks.

A TaskManager accepts task requests and hands them out to other processes. It stores the results that can then be picked up by the requester. A TaskManager also keeps track of its compute processes. If a process disappears, its running tasks are re-scheduled for execution by another compute process. TaskManangers are thread-safe.

Instance Methods

__init__(self)

str

activeProcessInfo(self, pid)
Returns: information about the active process number pid

str

addTaskRequest(self, tag, parameters, process_id=None)
Returns: the task id

deleteData(self, label)

tuple

Returns the result of an arbitrary finished task. The task is removed from the list of finished tasks.

getAnyResult(self)
Returns: the task id, the task tag, and the result of the task

tuple

Returns a waiting task of arbitrary tag. The task is removed from the list of waiting tasks and added to the list of running tasks.

getAnyTask(self, process_id=None)
Returns: the task id, the task tag, and the parameters

tuple

Returns the result of a finished task that has the given tag. The task is removed from the list of finished tasks.

getResultWithTag(self, tag)
Returns: the task id and the result of the task

tuple

Returns a waiting task with the given tag. The task is removed from the list of waiting tasks and added to the list of running tasks.

getTaskWithTag(self, tag, process_id=None)
Returns: the task id and the parameters

int

numberOfActiveProcesses(self)
Returns: the number of active processes

tuple

numberOfTasks(self)
Returns: a tuple of dictionaries containing the number of waiting tasks, the number of running tasks, and the number of results waiting to be retrieved.

ping(self, process_id)

int

Registers a process with the task manager. All processes must call this method before making any other task manager calls.

registerProcess(self, watchdog_period=None, info=None)
Returns: a unique process id

retrieveData(self, label)

returnTask(self, task_id)

storeData(self, **kw)

storeException(self, task_id, exception, traceback)

storeResult(self, task_id, result)

terminate(self)
Signals that no more tasks or results will be requested.

unregisterProcess(self, process_id)

Inherited from Pyro.core.ObjBase: GUID, Pyro_dyncall, delegateTo, getAttrProxy, getDaemon, getLocalStorage, getProxy, remote_retrieve_code, remote_supply_code, setCodeValidator, setDaemon, setGUID

Method Details

init(self)
(Constructor)

Overrides: Pyro.core.ObjBase.__init__

activeProcessInfo(self, pid)

Parameters:

pid (int) - the number of an active process

Returns: str

information about the active process number pid

addTaskRequest(self, tag, parameters, process_id=None)

Parameters:

tag (str) - the tag of the task being requested
parameters - the parameters to be passed to the task
process_id (int) - the id of the requesting process (optional)

Returns: str

the task id

deleteData(self, label)

Parameters:

label (str) - the label of the data item to be deleted

getAnyResult(self)

Returns: tuple Returns the result of an arbitrary finished task. The task is removed from the list of finished tasks.: the task id, the task tag, and the result of the task

getAnyTask(self, process_id=None)

Parameters:

process_id (int) - the id of the retrieving process (optional)

Returns: tuple Returns a waiting task of arbitrary tag. The task is removed from the list of waiting tasks and added to the list of running tasks.

the task id, the task tag, and the parameters

getResultWithTag(self, tag)

Parameters:

tag - a task tag

Returns: tuple Returns the result of a finished task that has the given tag. The task is removed from the list of finished tasks.

the task id and the result of the task

getTaskWithTag(self, tag, process_id=None)

Parameters:

tag (str) - a task tag
process_id (int) - the id of the retrieving process (optional)

Returns: tuple Returns a waiting task with the given tag. The task is removed from the list of waiting tasks and added to the list of running tasks.

the task id and the parameters

numberOfActiveProcesses(self)

Returns: int: the number of active processes

numberOfTasks(self)

Returns: tuple: a tuple of dictionaries containing the number of waiting tasks, the number of running tasks, and the number of results waiting to be retrieved. Each dictionary contains the count for each tag.

ping(self, process_id)

Parameters:

process_id (int
Tells the task manager that a process is still alive.
) - the id of the process

registerProcess(self, watchdog_period=None, info=None)

Parameters:

watchdog_period (int or NoneType) - the period at which the registering process promises to ping the task manager to signal that is still alive. If None, no pings are expected.
info (str) - an information string telling something about the machine running the process

Returns: int Registers a process with the task manager. All processes must call this method before making any other task manager calls.

a unique process id

retrieveData(self, label)

Parameters:

label (str) - the label of the data item to be retrieved

returnTask(self, task_id)

Parameters:

task_id (str
Removes a task from the list of running tasks and put its back at the beginning of the list of waiting tasks. This method should be called by a process that has obtained a task but cannot handle it.
) - the id of the task for which the result is provided

storeData(self, **kw)

Parameters:

kw (dict
This routine permits processes to exchange arbitrary data items through the task manager.
) - a keyword list of data items to be stored

storeException(self, task_id, exception, traceback)

Parameters:

task_id (str) - the id of the task for which the result is provided
exception - the exception raised by the task
traceback (str
Stores the exception associated with the task. The task is removed from the list of running tasks and added to the list of finished tasks. When the result is retrieved by another process, TaskRaisedException is raised.
) - a text version of the stack traceback at the time of the exception

storeResult(self, task_id, result)

Parameters:

task_id (str) - the id of the task for which the result is provided
result - the result of the task
Stores the result associated with the task. The task is removed from the list of running tasks and added to the list of finished tasks.

terminate(self)

Signals that no more tasks or results will be requested. All waiting threads will be terminated by raising TaskManagerTermination.

unregisterProcess(self, process_id)

Parameters:

process_id (int
Removes the process from the task manager's process list. All processes should unregister when they are no longer available for accepting tasks. The task manager will also unregister processes itself if they do not ping the task manager at the promised frequency.
) - the id of the process

Class TaskManager

__init__(self) (Constructor)

activeProcessInfo(self, pid)

addTaskRequest(self, tag, parameters, process_id=None)

deleteData(self, label)

getAnyResult(self)

getAnyTask(self, process_id=None)

getResultWithTag(self, tag)

getTaskWithTag(self, tag, process_id=None)

numberOfActiveProcesses(self)

numberOfTasks(self)

ping(self, process_id)

registerProcess(self, watchdog_period=None, info=None)

retrieveData(self, label)

returnTask(self, task_id)

storeData(self, **kw)

storeException(self, task_id, exception, traceback)

storeResult(self, task_id, result)

terminate(self)

unregisterProcess(self, process_id)

init(self)
(Constructor)