Legacy-mode subinterpreters in Python 3.12: import _tkinter leads to shutdown crash #115649
Description
Bug report
Bug description:
Hello everyone,
I am working on an application embedding multiple Python subinterpreters - for which the Python version should be upgraded from Python 3.10 to Python 3.12.1. For the time being, the "legacy version" of subinterpreters (i.e., using a global, shared GIL) should be used, since all Python extensions (including _tkinter and all extensions with single-phase initialization) should be supported.
If I understand the docs correctly, using the legacy Py_NewInterpreter() method should preserve the existing behavior. Still, the following application crashes at shutdown:
#include <Python.h>
class PythonInterpreter {
public:
PythonInterpreter(PyThreadState* parent, int id)
: mParent(parent)
, mId(id) {
PyEval_RestoreThread(mParent);
mThread = Py_NewInterpreter();
PyObject* globals = PyModule_GetDict(PyImport_AddModule("__main__"));
PyRun_String("import _tkinter", Py_single_input, globals, globals);
PyEval_SaveThread();
std::cout << "Subinterpreter " << id << " done" << std::endl;
}
virtual ~PythonInterpreter() {
std::cout << "destructor " << mId << std::endl;
PyThreadState_Swap(mThread);
Py_EndInterpreter(mThread);
}
private:
PyThreadState* mParent;
PyThreadState* mThread;
int mId;
};
int main(int /* argc */, char** /* argv[] */) {
PyConfig config;
PyConfig_InitPythonConfig(&config);
PyStatus status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
PyConfig_Clear(&config);
Py_ExitStatusException(status);
} else {
PyConfig_Clear(&config);
}
PyThreadState* s0 = PyThreadState_Get();
PyEval_SaveThread();
PythonInterpreter* i0 = new PythonInterpreter(s0, 0);
PythonInterpreter* i1 = new PythonInterpreter(s0, 1);
delete i0;
delete i1;
PyEval_RestoreThread(s0);
Py_Finalize();
}When compiling and running this program with a debug build of Python 3.12.1 (or later) on Linux, I get this output:
Subinterpreter 0 done
Subinterpreter 1 done
destructor 0
destructor 1
main: Objects/dictobject.c:283: unicode_get_hash: Assertion `Py_IS_TYPE(((PyObject*)(((o)))), (&PyUnicode_Type))' failed.
With a non-debug build, the program exits with a segmentation fault.
The gdb backtrace looks as follows:
#0 0x00007ffff6311387 in raise () from /lib64/libc.so.6
#1 0x00007ffff6312a78 in abort () from /lib64/libc.so.6
#2 0x00007ffff630a1a6 in __assert_fail_base () from /lib64/libc.so.6
#3 0x00007ffff630a252 in __assert_fail () from /lib64/libc.so.6
#4 0x00007ffff6a9baf5 in unicode_get_hash (o=<optimized out>) at Objects/dictobject.c:2143
#5 _PyDict_Next (op=op@entry=0x7fffecfd2270, ppos=ppos@entry=0x7fffffffd2a8, pkey=pkey@entry=0x7fffffffd2a0, pvalue=pvalue@entry=0x7fffffffd298,
phash=phash@entry=0x0) at Objects/dictobject.c:2142
#6 0x00007ffff6a9c0d8 in PyDict_Next (op=op@entry=0x7fffecfd2270, ppos=ppos@entry=0x7fffffffd2a8, pkey=pkey@entry=0x7fffffffd2a0,
pvalue=pvalue@entry=0x7fffffffd298) at Objects/dictobject.c:2189
#7 0x00007ffff6ab3751 in _PyModule_ClearDict (d=0x7fffecfd2270) at Objects/moduleobject.c:624
#8 0x00007ffff6ab40dd in _PyModule_Clear (m=m@entry=0x7fffed02ca70) at Objects/moduleobject.c:604
#9 0x00007ffff6c11bd4 in finalize_modules_clear_weaklist (interp=interp@entry=0x7fffed03c020, weaklist=weaklist@entry=0x7fffef912da0, verbose=verbose@entry=0)
at Python/pylifecycle.c:1526
#10 0x00007ffff6c125ef in finalize_modules (tstate=tstate@entry=0x7fffed099950) at Python/pylifecycle.c:1609
#11 0x00007ffff6c202da in Py_EndInterpreter (tstate=0x7fffed099950) at Python/pylifecycle.c:2220
#12 0x00000000004014b5 in PythonInterpreter::~PythonInterpreter (this=0x5092f0, __in_chrg=<optimized out>) at main.cc:25
#13 PythonInterpreter::~PythonInterpreter (this=0x5092f0, __in_chrg=<optimized out>) at main.cc:26
#14 0x00000000004012e2 in main () at main.cc:60
Digging further into the backtrace, it looks like the Python garbage collector is trying to decrease the reference counter to the _tkinter module twice, despite it having been increased only once. Oddly enough, the program runs just fine when destroying the interpreters in reverse order:
PythonInterpreter* i0 = new PythonInterpreter(s0, 0);
PythonInterpreter* i1 = new PythonInterpreter(s0, 1);
delete i1;
delete i0;Can anyone help me shed some light into this issue? Is there anything I am overlooking?
CPython versions tested on:
3.12.1, 3.12.2, 3.13.0a4
Operating systems tested on:
Linux, Windows
### Tasks