Skip to content

gh-136421: Load _datetime static types during interpreter initialization #136583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jul 21, 2025
Merged

gh-136421: Load _datetime static types during interpreter initialization #136583

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
77d117c
Load _datetime during interpreter initialization
ZeroIntensity Jul 12, 2025
963a9ee
Add a test case.
ZeroIntensity Jul 12, 2025
e16fb54
Add blurb.
ZeroIntensity Jul 12, 2025
ed65682
Move to pycore_init_types()
ZeroIntensity Jul 12, 2025
43b4843
Fix lint.
ZeroIntensity Jul 12, 2025
0ad304f
Add _datetime to the frozen modules.
ZeroIntensity Jul 12, 2025
d762ed5
Remove obsolete 'reloading' state.
ZeroIntensity Jul 20, 2025
db327e7
Some test refactoring and improvements.
ZeroIntensity Jul 20, 2025
9456147
Fix test.
ZeroIntensity Jul 20, 2025
6adafa1
Make tp_base stores atomic.
ZeroIntensity Jul 20, 2025
6e2f891
Revert "Make tp_base stores atomic."
ZeroIntensity Jul 20, 2025
a15843f
Set tp_base in the PyTypeObject definition.
ZeroIntensity Jul 20, 2025
d6064c4
Add a comment.
ZeroIntensity Jul 20, 2025
9a3b1c2
Move the test.
ZeroIntensity Jul 20, 2025
3520514
Remove dead tp_base code.
ZeroIntensity Jul 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Load _datetime during interpreter initialization
  • Loading branch information
@ZeroIntensity
ZeroIntensity committed Jul 12, 2025
commit 77d117c145ee7a2ced7f716e8b1815fb5d7cf587
1 change: 1 addition & 0 deletions Include/internal/pycore_pylifecycle.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ extern PyStatus _Py_HashRandomization_Init(const PyConfig *);

extern PyStatus _PyGC_Init(PyInterpreterState *interp);
extern PyStatus _PyAtExit_Init(PyInterpreterState *interp);
extern PyStatus _PyDateTime_Init(PyInterpreterState *interp);

/* Various internal finalizers */

Expand Down
2 changes: 2 additions & 0 deletions Modules/Setup.bootstrap.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ posix posixmodule.c
_signal signalmodule.c
_tracemalloc _tracemalloc.c
_suggestions _suggestions.c
# needs libm and on some platforms librt
_datetime _datetimemodule.c

# modules used by importlib, deepfreeze, freeze, runpy, and sysconfig
_codecs _codecsmodule.c
Expand Down
3 changes: 0 additions & 3 deletions Modules/Setup.stdlib.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,6 @@
@MODULE_CMATH_TRUE@cmath cmathmodule.c
@MODULE__STATISTICS_TRUE@_statistics _statisticsmodule.c

# needs libm and on some platforms librt
@MODULE__DATETIME_TRUE@_datetime _datetimemodule.c

# _decimal uses libmpdec
# either static libmpdec.a from Modules/_decimal/libmpdec or libmpdec.so
# with ./configure --with-system-libmpdec
Expand Down
17 changes: 5 additions & 12 deletions Modules/_datetimemodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include "pycore_object.h" // _PyObject_Init()
#include "pycore_time.h" // _PyTime_ObjectToTime_t()
#include "pycore_unicodeobject.h" // _PyUnicode_Copy()
#include "pycore_initconfig.h" // _PyStatus_OK()

#include "datetime.h"

Expand Down Expand Up @@ -7329,13 +7330,9 @@ clear_state(datetime_state *st)
}


static int
init_static_types(PyInterpreterState *interp, int reloading)
PyStatus
_PyDateTime_Init(PyInterpreterState *interp)
{
if (reloading) {
return 0;
}

// `&...` is not a constant expression according to a strict reading
// of C standards. Fill tp_base at run-time rather than statically.
// See https://bugs.python.org/issue40777
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR does not address the possible races in PyDateTime_*.tp_base = &PyDateTime_*Type; below, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not. I don't think there's an easy way to do this here, because atomically storing tp_base will continue to race with all the non-atomic reads elsewhere.

Do we even need to load it at runtime like this? We have other examples of directly storing it in the PyTypeObject structure:

cpython/Objects/methodobject.c

Line 396 in 958657b

.tp_base = &PyCFunction_Type,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not needed now because the module is statically linked, that issue happens only with dynamic loaded modules so you can define it statically now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess a _Py_IsMainInterPreter() check will suffice in this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not needed now because the module is statically linked, that issue happens only with dynamic loaded modules so you can define it statically now.

Ah, TIL. That's definitely the best option here then.

I guess a _Py_IsMainInterPreter() check will suffice in this PR?

For what?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ensuring tp_base is set only once after Py_Initialize(). But I missed the Kumar 's comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove PyDateTime_DateTimeType.tp_base = &PyDateTime_DateType; as well.

Expand All @@ -7347,11 +7344,11 @@ init_static_types(PyInterpreterState *interp, int reloading)
for (size_t i = 0; i < Py_ARRAY_LENGTH(capi_types); i++) {
PyTypeObject *type = capi_types[i];
if (_PyStaticType_InitForExtension(interp, type) < 0) {
return -1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR but as a follow up I think it would be better to now remove _PyStaticType_InitForExtension and just use _PyStaticType_InitBuiltin for it, there's a lot of special casing that could be removed.

return _PyStatus_ERR("could not initialize static types");
}
}

return 0;
return _PyStatus_OK();
}


Expand Down Expand Up @@ -7379,10 +7376,6 @@ _datetime_exec(PyObject *module)
}
/* We actually set the "current" module right before a successful return. */

if (init_static_types(interp, reloading) < 0) {
goto error;
}

for (size_t i = 0; i < Py_ARRAY_LENGTH(capi_types); i++) {
PyTypeObject *type = capi_types[i];
const char *name = _PyType_Name(type);
Expand Down
5 changes: 5 additions & 0 deletions Python/pylifecycle.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -915,6 +915,11 @@ pycore_interp_init(PyThreadState *tstate)
goto done;
}

status = _PyDateTime_Init(tstate->interp);
if (_PyStatus_EXCEPTION(status)) {
goto done;
}

const PyConfig *config = _PyInterpreterState_GetConfig(interp);

status = _PyImport_InitCore(tstate, sysmod, config->_install_importlib);
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy