Skip to content

Some pre-finalization callbacks can create other callbacks #136003

@ZeroIntensity

Description

@ZeroIntensity

Bug report

Bug description:

Currently, "pre-finalization" (callbacks executed while the interpreter is still fully intact) looks like this:

cpython/Python/pylifecycle.c

Lines 2023 to 2038 in 642e5df

wait_for_thread_shutdown(tstate);
// Make any remaining pending calls.
_Py_FinishPendingCalls(tstate);
/* The interpreter is still entirely intact at this point, and the
* exit funcs may be relying on that. In particular, if some thread
* or exit func is still waiting to do an import, the import machinery
* expects Py_IsInitialized() to return true. So don't say the
* runtime is uninitialized until after the exit funcs have run.
* Note that Threading.py uses an exit func to do a join on all the
* threads created thru it, so this also protects pending imports in
* the threads created via Threading.
*/
_PyAtExit_Call(tstate->interp);

Threads are joined first, then pending calls are executed, and finally atexit callbacks are executed. The problem is that any of these three can create one another, such as a pending call creating a thread, or an atexit handler scheduling a pending call. The best way to demonstrate this is by creating a new thread inside of an atexit callback:

import atexit
import threading
import time


def run():
    print(24)
    time.sleep(1)
    print(42)

@atexit.register
def start_thread():
    threading.Thread(target=run).start()

On 3.13+ (and probably earlier), you'll get one of these three outputs:

  1. 24 is printed.
  2. Nothing is printed.
  3. A fatal error occurs due to the stdout lock being unavailable.

This is because the thread created is incorrectly non-daemon, because the call to threading._shutdown (which should join it) has already happened.

@ericsnowcurrently, I discussed this with you at the PyCon sprints. I think we came to the agreement that the best way to fix this is by looping until all three of them execute no callbacks.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-C-APItype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy