Skip to content

Commit 94cc0b9

Browse files
committed
Add doc for buffersize argument of imap and imap_unordered
1 parent dff1167 commit 94cc0b9

File tree

2 files changed

+21
-2
lines changed

2 files changed

+21
-2
lines changed

Doc/library/multiprocessing.rst

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2448,7 +2448,7 @@ with the :class:`Pool` class.
24482448
Callbacks should complete immediately since otherwise the thread which
24492449
handles the results will get blocked.
24502450

2451-
.. method:: imap(func, iterable[, chunksize])
2451+
.. method:: imap(func, iterable[, chunksize[, buffersize]])
24522452

24532453
A lazier version of :meth:`.map`.
24542454

@@ -2462,7 +2462,18 @@ with the :class:`Pool` class.
24622462
``next(timeout)`` will raise :exc:`multiprocessing.TimeoutError` if the
24632463
result cannot be returned within *timeout* seconds.
24642464

2465-
.. method:: imap_unordered(func, iterable[, chunksize])
2465+
The *iterable* is collected immediately rather than lazily, unless a
2466+
*buffersize* is specified to limit the number of submitted tasks whose
2467+
results have not yet been yielded. If the buffer is full, iteration over
2468+
the *iterables* pauses until a result is yielded from the buffer.
2469+
To fully utilize pool's capacity, set *buffersize* to the number of
2470+
processes in pool (to consume *iterable* as you go) or even higher
2471+
(to prefetch *buffersize - processes* arguments).
2472+
2473+
.. versionadded:: 3.15
2474+
Added the *buffersize* parameter.
2475+
2476+
.. method:: imap_unordered(func, iterable[, chunksize[, buffersize]])
24662477

24672478
The same as :meth:`imap` except that the ordering of the results from the
24682479
returned iterator should be considered arbitrary. (Only when there is
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is
5+
full, iteration over the *iterables* pauses until a result is yielded from
6+
the buffer. To fully utilize pool's capacity, set *buffersize* to the number
7+
of processes in pool (to consume *iterable* as you go) or even higher (to
8+
prefetch *buffersize - processes* arguments).

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy