[3.14] gh-136421: Load _datetime static types during interpreter initialization (GH-136583)
#136943
Conversation
…tialization (pythonGH-136583) `_datetime` is a special module, because it's the only non-builtin C extension that contains static types. As such, it would initialize static types in the module's execution function, which can run concurrently. Since static type initialization is not thread-safe, this caused crashes. This fixes it by moving the initialization of `_datetime`'s static types to interpreter startup (where all other static types are initialized), which is already properly protected through other locks. (cherry picked from commit a10960699a2b3e4e62896331c4f9cfd162ebf440) Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
|
🤖 New build scheduled with the buildbot fleet by @ZeroIntensity for commit 4912c10 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136943%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
@hugovk Just want to make sure that this one is good with you before landing it. It fixes crashes with the subinterpreter APIs, but is also a risky change in the sense that we're making |
|
@ZeroIntensity Go ahead when the buildbots are ready, thanks for the ping. |
|
Most of them have passed, I'm confident that this is good to go. |
|
For future reference, this did break our downstream build at astral-sh/python-build-standalone#710 — it's not a big deal but I was surprised to see a change like this in the release candidate. Seems like a decent trade-off given it was fixing crashes, but figure the signal might be helpful. |
|
Yes but that's because we're reimplementing Modules/Setup for complicated reasons, right? If I understand correctly, the build system in CPython was stable into the release candidate, we're just choosing to mess around with implementation details of the build system that I don't think CPython should be expected to keep stable, if I understand correctly. (And our build correctly failed because we have checks that our reimplementation matches what is actually distributed by CPython.) |
|
Yeah our situation is bespoke, but changes to how extension modules are built are still significant. I'm not really sure what there is to debate though, since I agree with you and was just leaving the note for reference :) |
|
Thanks for the heads up. Would it be helpful to perhaps document this in the whatsnew somewhere? |
|
I don't think that'd make a difference to me, I found this pull request pretty easily. Maybe another downstream will feel differently though. |
_datetimeis a special module, because it's the only non-builtin C extension that contains static types. As such, it would initialize static types in the module's execution function, which can run concurrently. Since static type initialization is not thread-safe, this caused crashes. This fixes it by moving the initialization of_datetime's static types to interpreter startup (where all other static types are initialized), which is already properly protected through other locks.(cherry picked from commit a109606)
Co-authored-by: Peter Bierma zintensitydev@gmail.com
_datetimein sub-interpreters in the same time may crash the process #136421