Skip to content

Commit fc37cb4

Browse files
committed
Add doc for buffersize argument of imap and imap_unordered
1 parent c6e6074 commit fc37cb4

File tree

3 files changed

+29
-1
lines changed

3 files changed

+29
-1
lines changed

Doc/library/multiprocessing.rst

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2408,7 +2408,7 @@ with the :class:`Pool` class.
24082408
Callbacks should complete immediately since otherwise the thread which
24092409
handles the results will get blocked.
24102410

2411-
.. method:: imap(func, iterable[, chunksize])
2411+
.. method:: imap(func, iterable[, chunksize[, buffersize]])
24122412

24132413
A lazier version of :meth:`.map`.
24142414

@@ -2422,6 +2422,17 @@ with the :class:`Pool` class.
24222422
``next(timeout)`` will raise :exc:`multiprocessing.TimeoutError` if the
24232423
result cannot be returned within *timeout* seconds.
24242424

2425+
The *iterable* is collected immediately rather than lazily, unless a
2426+
*buffersize* is specified to limit the number of submitted tasks whose
2427+
results have not yet been yielded. If the buffer is full, iteration over
2428+
the *iterables* pauses until a result is yielded from the buffer.
2429+
To fully utilize pool's capacity, set *buffersize* to the number of
2430+
processes in pool (to consume *iterable* as you go) or even higher
2431+
(to prefetch *buffersize - processes* arguments).
2432+
2433+
.. versionadded:: 3.15
2434+
Added the *buffersize* parameter.
2435+
24252436
.. method:: imap_unordered(func, iterable[, chunksize])
24262437

24272438
The same as :meth:`imap` except that the ordering of the results from the
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is full,
5+
iteration over the *iterables* pauses until a result is yielded from the buffer.
6+
7+
To fully utilize pool's capacity, set *buffersize* to the number of
8+
processes in pool (to consume *iterable* as you go) or even higher
9+
(to prefetch *buffersize - processes* arguments).
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Add the optional ``buffersize`` parameter to
2+
:meth:`multiprocessing.pool.Pool.imap` and
3+
:meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
4+
submitted tasks whose results have not yet been yielded. If the buffer is
5+
full, iteration over the *iterables* pauses until a result is yielded from
6+
the buffer. To fully utilize pool's capacity, set *buffersize* to the number
7+
of processes in pool (to consume *iterable* as you go) or even higher (to
8+
prefetch *buffersize - processes* arguments).

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy