Skip to content

Commit 607ae4a

Browse files
committed
Reword, expand, and clarify the limitation, highlighting the REPL case.
1 parent dd2d43e commit 607ae4a

File tree

2 files changed

+36
-13
lines changed

2 files changed

+36
-13
lines changed

Doc/library/concurrent.futures.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -342,6 +342,11 @@ that :class:`ProcessPoolExecutor` will not work in the interactive interpreter.
342342
Calling :class:`Executor` or :class:`Future` methods from a callable submitted
343343
to a :class:`ProcessPoolExecutor` will result in deadlock.
344344

345+
Note that the restrictions on functions and arguments needing to picklable as
346+
per :class:`multiprocessing.Process` apply when using :meth:`~Executor.submit`
347+
and :meth:`~Executor.map` on a :class:`ProcessPoolExecutor`. A function defined
348+
in a REPL or a lambda should not be expected to work.
349+
345350
.. class:: ProcessPoolExecutor(max_workers=None, mp_context=None, initializer=None, initargs=(), max_tasks_per_child=None)
346351

347352
An :class:`Executor` subclass that executes calls asynchronously using a pool

Doc/library/multiprocessing.rst

Lines changed: 31 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -233,9 +233,12 @@ processes for a different context. In particular, locks created using
233233
the *fork* context cannot be passed to processes started using the
234234
*spawn* or *forkserver* start methods.
235235

236-
A library which wants to use a particular start method should probably
237-
use :func:`get_context` to avoid interfering with the choice of the
238-
library user.
236+
Libraries using :mod:`multiprocessing` or
237+
:class:`~concurrent.futures.ProcessPoolExecutor` should be designed to allow
238+
their users to provide their own multiprocessing context. Using a specific
239+
context of your own within a library can lead to incompatibilities with the
240+
rest of the library user's application. Always document if your library
241+
requires a specific start method.
239242

240243
.. warning::
241244

@@ -546,18 +549,33 @@ The :mod:`multiprocessing` package mostly replicates the API of the
546549

547550
.. note::
548551

549-
Starting with Python 3.14, ``'fork'`` is no longer the default start
550-
method on any operating system. When creating a new ``Process`` object
551-
in a REPL session, with a start method such as ``'spawn'`` or ``'forkserver'``
552-
(other than ``'fork'``), the *target* argument must be
553-
a callable object **mandatorily** defined in a module.
552+
In general, all arguments to :meth:`Process.__init__` must be picklable.
553+
This is particularly notable when trying to create a :class:`Process` or
554+
use a :class:`~concurrent.futures.ProcessPoolExecutor` from a REPL with a
555+
locally defined *target* function.
554556

555-
Using a callable object defined in the current REPL session raises
556-
an :exc:`AttributeError` exception when starting the process,
557-
although this is still possible when the start method is ``'fork'``.
557+
Passing a callable object defined in the current REPL session raises an
558+
:exc:`AttributeError` exception when starting the process as such as
559+
*target* must have been defined within an importable module to under to be
560+
unpickled.
558561

559-
This also applies to the use of the
560-
:class:`concurrent.futures.ProcessPoolExecutor` class.
562+
Example::
563+
564+
>>> import multiprocessing as mp
565+
>>> def knigit():
566+
... print("knee!")
567+
...
568+
>>> mp.Process(target=knigit).start()
569+
>>> Traceback (most recent call last):
570+
File ".../multiprocessing/spawn.py", line ..., in spawn_main
571+
File ".../multiprocessing/spawn.py", line ..., in _main
572+
AttributeError: module '__main__' has no attribute 'knigit'
573+
574+
See :ref:`multiprocessing-programming-spawn`.
575+
576+
While this restriction is not true if using the ``"fork"`` start method,
577+
as of Python ``3.14`` that is no longer the default on any platform. See
578+
:ref:`multiprocessing-start-methods`.
561579

562580
.. method:: run()
563581

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy