Skip to content

gh-127065: Make methodcaller thread-safe and re-entrant #127245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 21 commits into from
Closed

gh-127065: Make methodcaller thread-safe and re-entrant #127245

Changes from 4 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
223d650
Make methodcalled thread-safe
eendebakpt Nov 23, 2024
b6d454a
check result of PyMem_Malloc
eendebakpt Nov 23, 2024
4ce1233
enable ft
eendebakpt Nov 23, 2024
cf6b79b
fix memory error
eendebakpt Nov 25, 2024
2476ce4
wip
eendebakpt Nov 28, 2024
5ecf876
add tests
eendebakpt Nov 28, 2024
709010d
wip
eendebakpt Nov 28, 2024
6bd2c2e
wip
eendebakpt Nov 28, 2024
8d40552
Merge branch 'main' into methodcaller_ft
eendebakpt Nov 28, 2024
c9e3898
wip
eendebakpt Nov 28, 2024
8ef7a04
fix memory error
eendebakpt Nov 29, 2024
b4f30d3
skip check on zero size for memcpy
eendebakpt Dec 1, 2024
6d06201
📜🤖 Added by blurb_it.
blurb-it[bot] Dec 1, 2024
56cdc1f
Merge branch 'methodcaller_ft' of github.com:eendebakpt/cpython into …
eendebakpt Dec 1, 2024
440eb0c
Merge branch 'main' into methodcaller_ft
eendebakpt Dec 1, 2024
ad66951
review comments
eendebakpt Dec 3, 2024
bc3fe2a
review comments
eendebakpt Dec 3, 2024
e9a1fa6
Update Modules/_operator.c
eendebakpt Dec 6, 2024
00ab654
Update Modules/_operator.c
eendebakpt Dec 6, 2024
5a7344b
review comments
eendebakpt Dec 6, 2024
f9f53fe
Merge branch 'methodcaller_ft' of github.com:eendebakpt/cpython into …
eendebakpt Dec 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 17 additions & 12 deletions Modules/_operator.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1602,7 +1602,7 @@ typedef struct {
vectorcallfunc vectorcall;
} methodcallerobject;

#ifndef Py_GIL_DISABLED

static int _methodcaller_initialize_vectorcall(methodcallerobject* mc)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
static int _methodcaller_initialize_vectorcall(methodcallerobject* mc)
static int
_methodcaller_initialize_vectorcall(methodcallerobject *mc)

{
PyObject* args = mc->xargs;
Expand Down Expand Up @@ -1659,13 +1659,26 @@ methodcaller_vectorcall(
}

assert(mc->vectorcall_args != 0);
mc->vectorcall_args[0] = args[0];

Py_ssize_t number_of_arguments = PyTuple_GET_SIZE(mc->xargs) +
(mc->vectorcall_kwnames? PyTuple_GET_SIZE(mc->vectorcall_kwnames):0) + 1;
size_t buffer_size = sizeof(PyObject *) * (number_of_arguments + 1);

// for number_of_arguments == 1 we could optimize by setting tmp_args equal to &args
PyObject **tmp_args = (PyObject **) PyMem_Malloc(buffer_size);
if (tmp_args == NULL) {
PyErr_NoMemory();
return NULL;
}
memcpy(tmp_args, mc->vectorcall_args, buffer_size);
tmp_args[0] = args[0];
return PyObject_VectorcallMethod(
mc->name, mc->vectorcall_args,
mc->name, tmp_args,
(PyTuple_GET_SIZE(mc->xargs)) | PY_VECTORCALL_ARGUMENTS_OFFSET,
mc->vectorcall_kwnames);

PyMem_Free(tmp_args);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyMem_Free is after the return statement

}
#endif


/* AC 3.5: variable number of arguments, not currently support by AC */
Expand Down Expand Up @@ -1704,15 +1717,7 @@ methodcaller_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
mc->kwds = Py_XNewRef(kwds);
mc->vectorcall_args = 0;


#ifdef Py_GIL_DISABLED
// gh-127065: The current implementation of methodcaller_vectorcall
// is not thread-safe because it modifies the `vectorcall_args` array,
// which is shared across calls.
mc->vectorcall = NULL;
#else
mc->vectorcall = (vectorcallfunc)methodcaller_vectorcall;
#endif

PyObject_GC_Track(mc);
return (PyObject *)mc;
Expand Down
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy