.. include:: header.txt

===============
 Process Pools
===============

The `processing.pool` module has one public class:

    **class** `Pool(processes=None, initializer=None, initargs=())`
        A class representing a pool of worker processes.  

        Tasks can be offloaded to the pool and the results dealt with
        when they become available.  

        Note that tasks can only be submitted (or retrieved) by the
        process which created the pool object.

        `processes` is the number of worker processes to use.  If
        `processes` is `None` then the number returned by `cpuCount()`
        is used.  If `initializer` is not `None` then each worker
        process will call `initializer(*initargs)` when it starts.


Pool objects
============

`Pool` has the following public methods:

    `__init__(processes=None)`
        The constructor creates and starts `processes` worker
        processes.  If `processes` is `None` then `cpuCount()` is used
        to find a default or 1 if `cpuCount()` raises `NotImplemented`.

    `apply(func, args=(), kwds={})`
        Equivalent of the `apply()` builtin function.  It blocks till
        the result is ready.

    `applyAsync(func, args=(), kwds={}, callback=None)`
        A variant of the `apply()` method which returns a
        result object --- see `Asynchronous result objects`_.

        If `callback` is specified then it should be a callable which
        accepts a single argument.  When the result becomes ready
        `callback` is applied to it (unless the call failed).
        `callback` should complete immediately since otherwise the
        thread which handles the results will get blocked.

    `map(func, iterable, chunksize=None)`
        A parallel equivalent of the `map()` builtin function.  It
        blocks till the result is ready.  

        This method chops the iterable into a number of chunks which
        it submits to the process pool as separate tasks.  The
        (approximate) size of these chunks can be specified by setting
        `chunksize` to a positive integer.

    `mapAsync(func, iterable, chunksize=None, callback=None)`
        A variant of the `map()` method which returns a result object
        --- see `Asynchronous result objects`_.

        If `callback` is specified then it should be a callable which
        accepts a single argument.  When the result becomes ready
        `callback` is applied to it (unless the call failed).
        `callback` should complete immediately since otherwise the
        thread which handles the results will get blocked.
        
    `imap(func, iterable, chunksize=1)`
        An equivalent of `itertools.imap()`.

        The `chunksize` argument is the same as the one used by the
        `map()` method.  For very long iterables using a large value
        for `chunksize` can make make the job complete **much** faster
        than using the default value of `1`.

        Also if `chunksize` is `1` then the `next()` method of the
        iterator returned by the `imap()` method has an optional
        `timeout` parameter: `next(timeout)` will raise
        `processing.TimeoutError` if the result cannot be returned
        within `timeout` seconds.

    `imapUnordered(func, iterable, chunksize=1)`
        The same as `imap()` except that the ordering of the results
        from the returned iterator should be considered arbitrary.
        (Only when there is only one worker process is the order
        guaranteed to be "correct".)

    `close()`
        Prevents any more tasks from being submitted to the pool.
        Once all the tasks have been completed the worker processes
        will exit.

    `terminate()`
        Stops the worker processes immediately without completing
        outstanding work.  When the pool object is garbage collected
        `terminate()` will be called immediately.

    `join()`
        Wait for the worker processes to exit.  One must call
        `close()` or `terminate()` before using `join()`.


Asynchronous result objects
===========================

The result objects returns by `applyAsync()` and `mapAsync()` have
the following public methods:

    `get(timeout=None)`
        Returns the result when it arrives.  If `timeout` is not
        `None` and the result does not arrive within `timeout` seconds
        then `processing.TimeoutError` is raised.  If the remote call
        raised an exception then that exception will be reraised by `get()`.

    `wait(timeout=None)`
        Waits until the result is available or until `timeout` seconds
        pass.

    `ready()`
        Returns whether the call has completed.

    `successful()`
        Returns whether the call completed without raising an
        exception.  Will raise `AssertionError` if the result is not
        ready.


Examples
========

The following example demonstrates the use of a pool::

    from processing import Pool

    def f(x):
        return x*x

    if __name__ == '__main__':
        pool = Pool(processes=4)              # start 4 worker processes

        result = pool.applyAsync(f, (10,))    # evaluate "f(10)" asynchronously
        print result.get(timeout=1)           # prints "100" unless your computer is *very* slow

        print pool.map(f, range(10))          # prints "[0, 1, 4,..., 81]"

        it = pool.imap(f, range(10))
        print it.next()                       # prints "0"
        print it.next()                       # prints "1"
        print it.next(timeout=1)              # prints "4" unless your computer is *very* slow

        import time
        result = pool.applyAsync(time.sleep, (10,))
        print result.get(timeout=1)           # raises `TimeoutError`
                    


See also `ex_pool.py <../examples/ex_pool.py>`_.

.. _Prev: proxy-objects.html
.. _Up: processing-ref.html
.. _Next: sharedctypes.html