Skip to content

Commit e1becf4

Browse files
encukouMarcel Plchvstinner
authored
bpo-38787: C API for module state access from extension methods (PEP 573) (GH-19936)
Module C state is now accessible from C-defined heap type methods (PEP 573). Patch by Marcel Plch and Petr Viktorin. Co-authored-by: Marcel Plch <mplch@redhat.com> Co-authored-by: Victor Stinner <vstinner@python.org>
1 parent 4638c64 commit e1becf4

19 files changed

+797
-51
lines changed

‎Doc/c-api/structures.rst

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -147,23 +147,56 @@ Implementing functions and methods
147147
value of the function as exposed in Python. The function must return a new
148148
reference.
149149
150+
The function signature is::
151+
152+
PyObject *PyCFunction(PyObject *self,
153+
PyObject *const *args);
150154
151155
.. c:type:: PyCFunctionWithKeywords
152156
153157
Type of the functions used to implement Python callables in C
154158
with signature :const:`METH_VARARGS | METH_KEYWORDS`.
159+
The function signature is::
160+
161+
PyObject *PyCFunctionWithKeywords(PyObject *self,
162+
PyObject *const *args,
163+
PyObject *kwargs);
155164
156165
157166
.. c:type:: _PyCFunctionFast
158167
159168
Type of the functions used to implement Python callables in C
160169
with signature :const:`METH_FASTCALL`.
170+
The function signature is::
161171
172+
PyObject *_PyCFunctionFast(PyObject *self,
173+
PyObject *const *args,
174+
Py_ssize_t nargs);
162175
163176
.. c:type:: _PyCFunctionFastWithKeywords
164177
165178
Type of the functions used to implement Python callables in C
166179
with signature :const:`METH_FASTCALL | METH_KEYWORDS`.
180+
The function signature is::
181+
182+
PyObject *_PyCFunctionFastWithKeywords(PyObject *self,
183+
PyObject *const *args,
184+
Py_ssize_t nargs,
185+
PyObject *kwnames);
186+
187+
.. c:type:: PyCMethod
188+
189+
Type of the functions used to implement Python callables in C
190+
with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`.
191+
The function signature is::
192+
193+
PyObject *PyCMethod(PyObject *self,
194+
PyTypeObject *defining_class,
195+
PyObject *const *args,
196+
Py_ssize_t nargs,
197+
PyObject *kwnames)
198+
199+
.. versionadded:: 3.9
167200
168201
169202
.. c:type:: PyMethodDef
@@ -197,9 +230,7 @@ The :attr:`ml_flags` field is a bitfield which can include the following flags.
197230
The individual flags indicate either a calling convention or a binding
198231
convention.
199232
200-
There are four basic calling conventions for positional arguments
201-
and two of them can be combined with :const:`METH_KEYWORDS` to support
202-
also keyword arguments. So there are a total of 6 calling conventions:
233+
There are these calling conventions:
203234
204235
.. data:: METH_VARARGS
205236
@@ -250,6 +281,19 @@ also keyword arguments. So there are a total of 6 calling conventions:
250281
.. versionadded:: 3.7
251282
252283
284+
.. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS
285+
286+
Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining
287+
class*, that is, the class that contains the method in question.
288+
The defining class might be a superclass of ``Py_TYPE(self)``.
289+
290+
The method needs to be of type :c:type:`PyCMethod`, the same as for
291+
``METH_FASTCALL | METH_KEYWORDS`` with ``defining_class`` argument added after
292+
``self``.
293+
294+
.. versionadded:: 3.9
295+
296+
253297
.. data:: METH_NOARGS
254298
255299
Methods without parameters don't need to check whether arguments are given if

‎Doc/c-api/type.rst

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,14 +109,38 @@ Type Objects
109109
110110
.. versionadded:: 3.4
111111
112+
.. c:function:: PyObject* PyType_GetModule(PyTypeObject *type)
113+
114+
Return the module object associated with the given type when the type was
115+
created using :c:func:`PyType_FromModuleAndSpec`.
116+
117+
If no module is associated with the given type, sets :py:class:`TypeError`
118+
and returns ``NULL``.
119+
120+
.. versionadded:: 3.9
121+
122+
.. c:function:: void* PyType_GetModuleState(PyTypeObject *type)
123+
124+
Return the state of the module object associated with the given type.
125+
This is a shortcut for calling :c:func:`PyModule_GetState()` on the result
126+
of :c:func:`PyType_GetModule`.
127+
128+
If no module is associated with the given type, sets :py:class:`TypeError`
129+
and returns ``NULL``.
130+
131+
If the *type* has an associated module but its state is ``NULL``,
132+
returns ``NULL`` without setting an exception.
133+
134+
.. versionadded:: 3.9
135+
112136
113137
Creating Heap-Allocated Types
114138
.............................
115139
116140
The following functions and structs are used to create
117141
:ref:`heap types <heap-types>`.
118142
119-
.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
143+
.. c:function:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)
120144
121145
Creates and returns a heap type object from the *spec*
122146
(:const:`Py_TPFLAGS_HEAPTYPE`).
@@ -127,8 +151,18 @@ The following functions and structs are used to create
127151
If *bases* is ``NULL``, the *Py_tp_base* slot is used instead.
128152
If that also is ``NULL``, the new type derives from :class:`object`.
129153
154+
The *module* must be a module object or ``NULL``.
155+
If not ``NULL``, the module is associated with the new type and can later be
156+
retreived with :c:func:`PyType_GetModule`.
157+
130158
This function calls :c:func:`PyType_Ready` on the new type.
131159
160+
.. versionadded:: 3.9
161+
162+
.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
163+
164+
Equivalent to ``PyType_FromModuleAndSpec(NULL, spec, bases)``.
165+
132166
.. versionadded:: 3.3
133167
134168
.. c:function:: PyObject* PyType_FromSpec(PyType_Spec *spec)

‎Include/cpython/methodobject.h

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#ifndef Py_CPYTHON_METHODOBJECT_H
2+
# error "this header file must not be included directly"
3+
#endif
4+
5+
PyAPI_DATA(PyTypeObject) PyCMethod_Type;
6+
7+
/* Macros for direct access to these values. Type checks are *not*
8+
done, so use with care. */
9+
#define PyCFunction_GET_FUNCTION(func) \
10+
(((PyCFunctionObject *)func) -> m_ml -> ml_meth)
11+
#define PyCFunction_GET_SELF(func) \
12+
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_STATIC ? \
13+
NULL : ((PyCFunctionObject *)func) -> m_self)
14+
#define PyCFunction_GET_FLAGS(func) \
15+
(((PyCFunctionObject *)func) -> m_ml -> ml_flags)
16+
#define PyCFunction_GET_CLASS(func) \
17+
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_METHOD ? \
18+
((PyCMethodObject *)func) -> mm_class : NULL)
19+
20+
typedef struct {
21+
PyObject_HEAD
22+
PyMethodDef *m_ml; /* Description of the C function to call */
23+
PyObject *m_self; /* Passed as 'self' arg to the C func, can be NULL */
24+
PyObject *m_module; /* The __module__ attribute, can be anything */
25+
PyObject *m_weakreflist; /* List of weak references */
26+
vectorcallfunc vectorcall;
27+
} PyCFunctionObject;
28+
29+
typedef struct {
30+
PyCFunctionObject func;
31+
PyTypeObject *mm_class; /* Class that defines this method */
32+
} PyCMethodObject;

‎Include/cpython/object.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,7 @@ typedef struct _heaptypeobject {
289289
PyBufferProcs as_buffer;
290290
PyObject *ht_name, *ht_slots, *ht_qualname;
291291
struct _dictkeysobject *ht_cached_keys;
292+
PyObject *ht_module;
292293
/* here are optional user slots, followed by the members. */
293294
} PyHeapTypeObject;
294295

‎Include/methodobject.h

Lines changed: 28 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ extern "C" {
1313

1414
PyAPI_DATA(PyTypeObject) PyCFunction_Type;
1515

16-
#define PyCFunction_Check(op) Py_IS_TYPE(op, &PyCFunction_Type)
16+
#define PyCFunction_Check(op) (Py_IS_TYPE(op, &PyCFunction_Type) || (PyType_IsSubtype(Py_TYPE(op), &PyCFunction_Type)))
1717

1818
typedef PyObject *(*PyCFunction)(PyObject *, PyObject *);
1919
typedef PyObject *(*_PyCFunctionFast) (PyObject *, PyObject *const *, Py_ssize_t);
@@ -22,21 +22,13 @@ typedef PyObject *(*PyCFunctionWithKeywords)(PyObject *, PyObject *,
2222
typedef PyObject *(*_PyCFunctionFastWithKeywords) (PyObject *,
2323
PyObject *const *, Py_ssize_t,
2424
PyObject *);
25+
typedef PyObject *(*PyCMethod)(PyObject *, PyTypeObject *, PyObject *const *,
26+
size_t, PyObject *);
27+
2528
PyAPI_FUNC(PyCFunction) PyCFunction_GetFunction(PyObject *);
2629
PyAPI_FUNC(PyObject *) PyCFunction_GetSelf(PyObject *);
2730
PyAPI_FUNC(int) PyCFunction_GetFlags(PyObject *);
2831

29-
/* Macros for direct access to these values. Type checks are *not*
30-
done, so use with care. */
31-
#ifndef Py_LIMITED_API
32-
#define PyCFunction_GET_FUNCTION(func) \
33-
(((PyCFunctionObject *)func) -> m_ml -> ml_meth)
34-
#define PyCFunction_GET_SELF(func) \
35-
(((PyCFunctionObject *)func) -> m_ml -> ml_flags & METH_STATIC ? \
36-
NULL : ((PyCFunctionObject *)func) -> m_self)
37-
#define PyCFunction_GET_FLAGS(func) \
38-
(((PyCFunctionObject *)func) -> m_ml -> ml_flags)
39-
#endif
4032
Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyCFunction_Call(PyObject *, PyObject *, PyObject *);
4133

4234
struct PyMethodDef {
@@ -52,6 +44,13 @@ typedef struct PyMethodDef PyMethodDef;
5244
PyAPI_FUNC(PyObject *) PyCFunction_NewEx(PyMethodDef *, PyObject *,
5345
PyObject *);
5446

47+
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
48+
#define PyCFunction_NewEx(ML, SELF, MOD) PyCMethod_New((ML), (SELF), (MOD), NULL)
49+
PyAPI_FUNC(PyObject *) PyCMethod_New(PyMethodDef *, PyObject *,
50+
PyObject *, PyTypeObject *);
51+
#endif
52+
53+
5554
/* Flag passed to newmethodobject */
5655
/* #define METH_OLDARGS 0x0000 -- unsupported now */
5756
#define METH_VARARGS 0x0001
@@ -84,15 +83,24 @@ PyAPI_FUNC(PyObject *) PyCFunction_NewEx(PyMethodDef *, PyObject *,
8483
#define METH_STACKLESS 0x0000
8584
#endif
8685

86+
/* METH_METHOD means the function stores an
87+
* additional reference to the class that defines it;
88+
* both self and class are passed to it.
89+
* It uses PyCMethodObject instead of PyCFunctionObject.
90+
* May not be combined with METH_NOARGS, METH_O, METH_CLASS or METH_STATIC.
91+
*/
92+
93+
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
94+
#define METH_METHOD 0x0200
95+
#endif
96+
97+
8798
#ifndef Py_LIMITED_API
88-
typedef struct {
89-
PyObject_HEAD
90-
PyMethodDef *m_ml; /* Description of the C function to call */
91-
PyObject *m_self; /* Passed as 'self' arg to the C func, can be NULL */
92-
PyObject *m_module; /* The __module__ attribute, can be anything */
93-
PyObject *m_weakreflist; /* List of weak references */
94-
vectorcallfunc vectorcall;
95-
} PyCFunctionObject;
99+
100+
#define Py_CPYTHON_METHODOBJECT_H
101+
#include "cpython/methodobject.h"
102+
#undef Py_CPYTHON_METHODOBJECT_H
103+
96104
#endif
97105

98106
#ifdef __cplusplus

‎Include/object.h

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,11 @@ PyAPI_FUNC(PyObject*) PyType_FromSpecWithBases(PyType_Spec*, PyObject*);
213213
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03040000
214214
PyAPI_FUNC(void*) PyType_GetSlot(PyTypeObject*, int);
215215
#endif
216+
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03090000
217+
PyAPI_FUNC(PyObject*) PyType_FromModuleAndSpec(PyObject *, PyType_Spec *, PyObject *);
218+
PyAPI_FUNC(PyObject *) PyType_GetModule(struct _typeobject *);
219+
PyAPI_FUNC(void *) PyType_GetModuleState(struct _typeobject *);
220+
#endif
216221

217222
/* Generic type check */
218223
PyAPI_FUNC(int) PyType_IsSubtype(PyTypeObject *, PyTypeObject *);

‎Lib/test/test_capi.py

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
import time
1414
import unittest
1515
import weakref
16+
import importlib.machinery
17+
import importlib.util
1618
from test import support
1719
from test.support import MISSING_C_DOCSTRINGS
1820
from test.support.script_helper import assert_python_failure, assert_python_ok
@@ -774,5 +776,76 @@ class PyMemDefaultTests(PyMemDebugTests):
774776
PYTHONMALLOC = ''
775777

776778

779+
class Test_ModuleStateAccess(unittest.TestCase):
780+
"""Test access to module start (PEP 573)"""
781+
782+
# The C part of the tests lives in _testmultiphase, in a module called
783+
# _testmultiphase_meth_state_access.
784+
# This module has multi-phase initialization, unlike _testcapi.
785+
786+
def setUp(self):
787+
fullname = '_testmultiphase_meth_state_access' # XXX
788+
origin = importlib.util.find_spec('_testmultiphase').origin
789+
loader = importlib.machinery.ExtensionFileLoader(fullname, origin)
790+
spec = importlib.util.spec_from_loader(fullname, loader)
791+
module = importlib.util.module_from_spec(spec)
792+
loader.exec_module(module)
793+
self.module = module
794+
795+
def test_subclass_get_module(self):
796+
"""PyType_GetModule for defining_class"""
797+
class StateAccessType_Subclass(self.module.StateAccessType):
798+
pass
799+
800+
instance = StateAccessType_Subclass()
801+
self.assertIs(instance.get_defining_module(), self.module)
802+
803+
def test_subclass_get_module_with_super(self):
804+
class StateAccessType_Subclass(self.module.StateAccessType):
805+
def get_defining_module(self):
806+
return super().get_defining_module()
807+
808+
instance = StateAccessType_Subclass()
809+
self.assertIs(instance.get_defining_module(), self.module)
810+
811+
def test_state_access(self):
812+
"""Checks methods defined with and without argument clinic
813+
814+
This tests a no-arg method (get_count) and a method with
815+
both a positional and keyword argument.
816+
"""
817+
818+
a = self.module.StateAccessType()
819+
b = self.module.StateAccessType()
820+
821+
methods = {
822+
'clinic': a.increment_count_clinic,
823+
'noclinic': a.increment_count_noclinic,
824+
}
825+
826+
for name, increment_count in methods.items():
827+
with self.subTest(name):
828+
self.assertEqual(a.get_count(), b.get_count())
829+
self.assertEqual(a.get_count(), 0)
830+
831+
increment_count()
832+
self.assertEqual(a.get_count(), b.get_count())
833+
self.assertEqual(a.get_count(), 1)
834+
835+
increment_count(3)
836+
self.assertEqual(a.get_count(), b.get_count())
837+
self.assertEqual(a.get_count(), 4)
838+
839+
increment_count(-2, twice=True)
840+
self.assertEqual(a.get_count(), b.get_count())
841+
self.assertEqual(a.get_count(), 0)
842+
843+
with self.assertRaises(TypeError):
844+
increment_count(thrice=3)
845+
846+
with self.assertRaises(TypeError):
847+
increment_count(1, 2, 3)
848+
849+
777850
if __name__ == "__main__":
778851
unittest.main()

‎Lib/test/test_sys.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1322,7 +1322,7 @@ def delx(self): del self.__x
13221322
'3P' # PyMappingMethods
13231323
'10P' # PySequenceMethods
13241324
'2P' # PyBufferProcs
1325-
'4P')
1325+
'5P')
13261326
class newstyleclass(object): pass
13271327
# Separate block for PyDictKeysObject with 8 keys and 5 entries
13281328
check(newstyleclass, s + calcsize("2nP2n0P") + 8 + 5*calcsize("n2P"))

‎Makefile.pre.in

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1104,6 +1104,7 @@ PYTHON_HEADERS= \
11041104
$(srcdir)/Include/cpython/initconfig.h \
11051105
$(srcdir)/Include/cpython/interpreteridobject.h \
11061106
$(srcdir)/Include/cpython/listobject.h \
1107+
$(srcdir)/Include/cpython/methodobject.h \
11071108
$(srcdir)/Include/cpython/object.h \
11081109
$(srcdir)/Include/cpython/objimpl.h \
11091110
$(srcdir)/Include/cpython/pyerrors.h \

‎Misc/NEWS.d/next/C API/2020-01-22-12-38-59.bpo-38787.HUH6hd.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Module C state is now accessible from C-defined heap type methods (:pep:`573`).
2+
Patch by Marcel Plch and Petr Viktorin.

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy