Crash in clear_weakref() in pydantic test suite, in 3.12.0rc1 and newer #108295
Description
Crash report
CPython versions tested on:
3.12, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.12.0rc1+ (heads/3.12:149d70c254, Aug 22 2023, 16:11:47) [GCC 13.2.0]
What happened?
The pydantic test suite started causing pytest to segfault on exit recently, with Python 3.12. I've been able to bisect it to pydantic commit pydantic/pydantic@f5e3aa9 that looks relatively harmless. I've also been able to bisect CPython into commit 58f9c88:
gh-106403: Restore weakref support for TypeVar and friends (GH-106418)
I have been able to reproduce the problem with tip of main as well, though I have to note it's a bit of heisenbug. I've basically had a very lucky day today that I've managed to correctly bisect on the second attempt.
To reproduce:
git clone https://github.com/pydantic/pydantic/
cd pydantic
python -m venv .venv
. .venv/bin/activate
pip install -e . pytest dirty-equals
# may need to be repeated a few times
python -m pytest -o addopts= tests/test_generics.py --deselect tests/test_generics.py::test_partial_specification_name -s
Error messages
Backtrace
Core was generated by `python -m pytest -o addopts= tests/test_generics.py --deselect tests/test_gener'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 clear_weakref (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:62
62 if (*list == self)
(gdb) bt
#0 clear_weakref (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:62
#1 0x0000557c3663d843 in _PyWeakref_ClearRef (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:102
#2 0x0000557c366ec77e in handle_weakrefs (unreachable=unreachable@entry=0x7ffeec923ad0,
old=old@entry=0x557c36a40f68 <_PyRuntime+76552>) at Modules/gcmodule.c:804
#3 0x0000557c366ecad3 in gc_collect_main (tstate=0x557c36a9e7b0 <_PyRuntime+459600>, generation=generation@entry=2,
n_collected=n_collected@entry=0x0, n_uncollectable=n_uncollectable@entry=0x0, nofail=nofail@entry=1) at Modules/gcmodule.c:1284
#4 0x0000557c366ed333 in _PyGC_CollectNoFail (tstate=tstate@entry=0x557c36a9e7b0 <_PyRuntime+459600>) at Modules/gcmodule.c:2135
#5 0x0000557c366c39bb in finalize_modules (tstate=tstate@entry=0x557c36a9e7b0 <_PyRuntime+459600>) at Python/pylifecycle.c:1602
#6 0x0000557c366c524e in Py_FinalizeEx () at Python/pylifecycle.c:1863
#7 0x0000557c366eb4f9 in Py_RunMain () at Modules/main.c:691
#8 0x0000557c366eb56e in pymain_main (args=args@entry=0x7ffeec923c10) at Modules/main.c:719
#9 0x0000557c366eb63d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:743
#10 0x0000557c3655979e in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15
(gdb) p self
$1 = (PyWeakReference *) 0x7f0a9a171df0
(gdb) p *self
$2 = {ob_base = {{ob_refcnt = 1, ob_refcnt_split = {1, 0}}, ob_type = 0x557c38bda680}, wr_object = 0x7f0a9a171c70, wr_callback = 0x0,
hash = -1, wr_prev = 0x0, wr_next = 0x0, vectorcall = 0x557c3663aa08 <weakref_vectorcall>}
(gdb) p *list
Cannot access memory at address 0xfe15355ba7e0
(gdb) p list
$3 = (PyWeakReference **) 0xfe15355ba7e0