Skip to content

Commit 5807d80

Browse files
IvanIsCodingIvanIsCoding
authored
Add #[pyclass(generic)] (#4926)
* First attempt at supporting pyclass(generic) * Include class_geitem in macro * Minor formatting * Pass py argument to new() * Add tests covering pyclass(generic) * Add changelog entry * Add test covering __getitem__ * Fix compilation error tests * Fix Alias -> TypingAlias * Make test case slightly stronger * Add compilation error test * Do not allow pyclass(generic) for enums * Remove unused import * Add import only to 3.9+ * Add documentation * Ignore Python typing hints newly added Rust code * Fix clippy warnings for the test * Add newline * Update TRYBUILD tests * Move `generic` in table * TypingAlias is only available in 3.10+ * Update trybuild test
1 parent 283ba3f commit 5807d80

13 files changed

+436
-23
lines changed

‎guide/pyclass-parameters.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
| <span style="white-space: pre">`extends = BaseType`</span> | Use a custom baseclass. Defaults to [`PyAny`][params-1] |
1111
| <span style="white-space: pre">`freelist = N`</span> | Implements a [free list][params-2] of size N. This can improve performance for types that are often created and deleted in quick succession. Profile your code to see whether `freelist` is right for you. |
1212
| <span style="white-space: pre">`frozen`</span> | Declares that your pyclass is immutable. It removes the borrow checker overhead when retrieving a shared reference to the Rust struct, but disables the ability to get a mutable reference. |
13+
| `generic` | Implements runtime parametrization for the class following [PEP 560](https://peps.python.org/pep-0560/). |
1314
| `get_all` | Generates getters for all fields of the pyclass. |
1415
| `hash` | Implements `__hash__` using the `Hash` implementation of the underlying Rust datatype. |
1516
| `immutable_type` | Makes the type object immutable. Supported on 3.14+ with the `abi3` feature active, or 3.10+ otherwise. |

‎guide/src/python-typing-hints.md

Lines changed: 77 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ As we can see, those are not full definitions containing implementation, but jus
4141

4242
### What do the PEPs say?
4343

44-
At the time of writing this documentation, the `pyi` files are referenced in three PEPs.
44+
At the time of writing this documentation, the `pyi` files are referenced in four PEPs.
4545

4646
[PEP8 - Style Guide for Python Code - #Function Annotations](https://www.python.org/dev/peps/pep-0008/#function-annotations) (last point) recommends all third party library creators to provide stub files as the source of knowledge about the package for type checker tools.
4747

@@ -55,6 +55,8 @@ It contains a specification for them (highly recommended reading, since it conta
5555

5656
[PEP561 - Distributing and Packaging Type Information](https://www.python.org/dev/peps/pep-0561/) describes in detail how to build packages that will enable type checking. In particular it contains information about how the stub files must be distributed in order for type checkers to use them.
5757

58+
[PEP560 - Core support for typing module and generic types](https://www.python.org/dev/peps/pep-0560/) describes the details on how Python's type system internally supports generics, including both runtime behavior and integration with static type checkers.
59+
5860
## How to do it?
5961

6062
[PEP561](https://www.python.org/dev/peps/pep-0561/) recognizes three ways of distributing type information:
@@ -165,3 +167,77 @@ class Car:
165167
:return: the name of the color our great algorithm thinks is the best for this car
166168
"""
167169
```
170+
171+
### Supporting Generics
172+
173+
Type annotations can also be made generic in Python. They are useful for working
174+
with different types while maintaining type safety. Usually, generic classes
175+
inherit from the `typing.Generic` metaclass.
176+
177+
Take for example the following `.pyi` file that specifies a `Car` that can
178+
accept multiple types of wheels:
179+
180+
```python
181+
from typing import Generic, TypeVar
182+
183+
W = TypeVar('W')
184+
185+
class Car(Generic[W]):
186+
def __init__(self, wheels: list[W]) -> None: ...
187+
188+
def get_wheels(self) -> list[W]: ...
189+
190+
def change_wheel(self, wheel_number: int, wheel: W) -> None: ...
191+
```
192+
193+
This way, the end-user can specify the type with variables such as `truck: Car[SteelWheel] = ...`
194+
and `f1_car: Car[AlloyWheel] = ...`.
195+
196+
There is also a special syntax for specifying generic types in Python 3.12+:
197+
198+
```python
199+
class Car[W]:
200+
def __init__(self, wheels: list[W]) -> None: ...
201+
202+
def get_wheels(self) -> list[W]: ...
203+
```
204+
205+
#### Runtime Behaviour
206+
207+
Stub files (`pyi`) are only useful for static type checkers and ignored at runtime. Therefore,
208+
PyO3 classes do not inherit from `typing.Generic` even if specified in the stub files.
209+
210+
This can cause some runtime issues, as annotating a variable like `f1_car: Car[AlloyWheel] = ...`
211+
can make Python call magic methods that are not defined.
212+
213+
To overcome this limitation, implementers can pass the `generic` parameter to `pyclass` in Rust:
214+
215+
```rust ignore
216+
#[pyclass(generic)]
217+
```
218+
219+
#### Advanced Users
220+
221+
`#[pyclass(generic)]` implements a very simple runtime behavior that accepts
222+
any generic argument. Advanced users can opt to manually implement
223+
[`__class_geitem__`](https://docs.python.org/3/reference/datamodel.html#emulating-generic-types)
224+
for the generic class to have more control.
225+
226+
```rust ignore
227+
impl MyClass {
228+
#[classmethod]
229+
#[pyo3(signature = (key, /))]
230+
pub fn __class_getitem__(
231+
cls: &Bound<'_, PyType>,
232+
key: &Bound<'_, PyAny>,
233+
) -> PyResult<PyObject> {
234+
/* implementation details */
235+
}
236+
}
237+
```
238+
239+
Note that [`pyo3::types::PyGenericAlias`][pygenericalias] can be helfpul when implementing
240+
`__class_geitem__` as it can create [`types.GenericAlias`][genericalias] objects from Rust.
241+
242+
[pygenericalias]: {{#PYO3_DOCS_URL}}/pyo3/types/struct.pygenericalias
243+
[genericalias]: https://docs.python.org/3/library/types.html#types.GenericAlias

‎newsfragments/4926.added.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Introduced a new optional parameter with `#[pyclass(generic)]`. This new
2+
parameter makes classes support generic typing at the runtime following
3+
[PEP 560](https://peps.python.org/pep-0560/). It is an alternative
4+
to inheriting from [typing.Generic](https://docs.python.org/3/library/typing.html#typing.Generic).

‎pyo3-macros-backend/src/attributes.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ pub mod kw {
4646
syn::custom_keyword!(transparent);
4747
syn::custom_keyword!(unsendable);
4848
syn::custom_keyword!(weakref);
49+
syn::custom_keyword!(generic);
4950
syn::custom_keyword!(gil_used);
5051
}
5152

‎pyo3-macros-backend/src/pyclass.rs

Lines changed: 68 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ pub struct PyClassPyO3Options {
8383
pub subclass: Option<kw::subclass>,
8484
pub unsendable: Option<kw::unsendable>,
8585
pub weakref: Option<kw::weakref>,
86+
pub generic: Option<kw::generic>,
8687
}
8788

8889
pub enum PyClassPyO3Option {
@@ -107,6 +108,7 @@ pub enum PyClassPyO3Option {
107108
Subclass(kw::subclass),
108109
Unsendable(kw::unsendable),
109110
Weakref(kw::weakref),
111+
Generic(kw::generic),
110112
}
111113

112114
impl Parse for PyClassPyO3Option {
@@ -154,6 +156,8 @@ impl Parse for PyClassPyO3Option {
154156
input.parse().map(PyClassPyO3Option::Unsendable)
155157
} else if lookahead.peek(attributes::kw::weakref) {
156158
input.parse().map(PyClassPyO3Option::Weakref)
159+
} else if lookahead.peek(attributes::kw::generic) {
160+
input.parse().map(PyClassPyO3Option::Generic)
157161
} else {
158162
Err(lookahead.error())
159163
}
@@ -232,6 +236,7 @@ impl PyClassPyO3Options {
232236
);
233237
set_option!(weakref);
234238
}
239+
PyClassPyO3Option::Generic(generic) => set_option!(generic),
235240
}
236241
Ok(())
237242
}
@@ -429,6 +434,21 @@ fn impl_class(
429434
}
430435
}
431436

437+
let mut default_methods = descriptors_to_items(
438+
cls,
439+
args.options.rename_all.as_ref(),
440+
args.options.frozen,
441+
field_options,
442+
ctx,
443+
)?;
444+
445+
let (default_class_geitem, default_class_geitem_method) =
446+
pyclass_class_geitem(&args.options, &syn::parse_quote!(#cls), ctx)?;
447+
448+
if let Some(default_class_geitem_method) = default_class_geitem_method {
449+
default_methods.push(default_class_geitem_method);
450+
}
451+
432452
let (default_str, default_str_slot) =
433453
implement_pyclass_str(&args.options, &syn::parse_quote!(#cls), ctx);
434454

@@ -443,21 +463,9 @@ fn impl_class(
443463
slots.extend(default_hash_slot);
444464
slots.extend(default_str_slot);
445465

446-
let py_class_impl = PyClassImplsBuilder::new(
447-
cls,
448-
args,
449-
methods_type,
450-
descriptors_to_items(
451-
cls,
452-
args.options.rename_all.as_ref(),
453-
args.options.frozen,
454-
field_options,
455-
ctx,
456-
)?,
457-
slots,
458-
)
459-
.doc(doc)
460-
.impl_all(ctx)?;
466+
let py_class_impl = PyClassImplsBuilder::new(cls, args, methods_type, default_methods, slots)
467+
.doc(doc)
468+
.impl_all(ctx)?;
461469

462470
Ok(quote! {
463471
impl #pyo3_path::types::DerefToPyAny for #cls {}
@@ -472,6 +480,7 @@ fn impl_class(
472480
#default_richcmp
473481
#default_hash
474482
#default_str
483+
#default_class_geitem
475484
}
476485
})
477486
}
@@ -514,6 +523,10 @@ pub fn build_py_enum(
514523
bail_spanned!(enum_.brace_token.span.join() => "#[pyclass] can't be used on enums without any variants");
515524
}
516525

526+
if let Some(generic) = &args.options.generic {
527+
bail_spanned!(generic.span() => "enums do not support #[pyclass(generic)]");
528+
}
529+
517530
let doc = utils::get_doc(&enum_.attrs, None, ctx);
518531
let enum_ = PyClassEnum::new(enum_)?;
519532
impl_enum(enum_, &args, doc, method_type, ctx)
@@ -2008,6 +2021,46 @@ fn pyclass_hash(
20082021
}
20092022
}
20102023

2024+
fn pyclass_class_geitem(
2025+
options: &PyClassPyO3Options,
2026+
cls: &syn::Type,
2027+
ctx: &Ctx,
2028+
) -> Result<(Option<syn::ImplItemFn>, Option<MethodAndMethodDef>)> {
2029+
let Ctx { pyo3_path, .. } = ctx;
2030+
match options.generic {
2031+
Some(_) => {
2032+
let ident = format_ident!("__class_getitem__");
2033+
let mut class_geitem_impl: syn::ImplItemFn = {
2034+
parse_quote! {
2035+
#[classmethod]
2036+
fn #ident<'py>(
2037+
cls: &#pyo3_path::Bound<'py, #pyo3_path::types::PyType>,
2038+
key: &#pyo3_path::Bound<'py, #pyo3_path::types::PyAny>
2039+
) -> #pyo3_path::PyResult<#pyo3_path::Bound<'py, #pyo3_path::types::PyGenericAlias>> {
2040+
#pyo3_path::types::PyGenericAlias::new(cls.py(), cls.as_any(), key)
2041+
}
2042+
}
2043+
};
2044+
2045+
let spec = FnSpec::parse(
2046+
&mut class_geitem_impl.sig,
2047+
&mut class_geitem_impl.attrs,
2048+
Default::default(),
2049+
)?;
2050+
2051+
let class_geitem_method = crate::pymethod::impl_py_method_def(
2052+
cls,
2053+
&spec,
2054+
&spec.get_doc(&class_geitem_impl.attrs, ctx),
2055+
Some(quote!(#pyo3_path::ffi::METH_CLASS)),
2056+
ctx,
2057+
)?;
2058+
Ok((Some(class_geitem_impl), Some(class_geitem_method)))
2059+
}
2060+
None => Ok((None, None)),
2061+
}
2062+
}
2063+
20112064
/// Implements most traits used by `#[pyclass]`.
20122065
///
20132066
/// Specifically, it implements traits that only depend on class name,

‎tests/test_class_basics.rs

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -714,3 +714,34 @@ fn test_unsendable_dict_with_weakref() {
714714
);
715715
});
716716
}
717+
718+
#[cfg(Py_3_9)]
719+
#[pyclass(generic)]
720+
struct ClassWithRuntimeParametrization {
721+
#[pyo3(get, set)]
722+
value: PyObject,
723+
}
724+
725+
#[cfg(Py_3_9)]
726+
#[pymethods]
727+
impl ClassWithRuntimeParametrization {
728+
#[new]
729+
fn new(value: PyObject) -> ClassWithRuntimeParametrization {
730+
Self { value }
731+
}
732+
}
733+
734+
#[test]
735+
#[cfg(Py_3_9)]
736+
fn test_runtime_parametrization() {
737+
Python::with_gil(|py| {
738+
let ty = py.get_type::<ClassWithRuntimeParametrization>();
739+
py_assert!(py, ty, "ty[int] == ty.__class_getitem__((int,))");
740+
py_run!(
741+
py,
742+
ty,
743+
"import types;
744+
assert ty.__class_getitem__((int,)) == types.GenericAlias(ty, (int,))"
745+
);
746+
});
747+
}

‎tests/test_compile_error.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,10 @@ fn test_compile_errors() {
1010
t.compile_fail("tests/ui/invalid_pyclass_args.rs");
1111
t.compile_fail("tests/ui/invalid_pyclass_enum.rs");
1212
t.compile_fail("tests/ui/invalid_pyclass_item.rs");
13+
#[cfg(Py_3_9)]
14+
t.compile_fail("tests/ui/invalid_pyclass_generic.rs");
15+
#[cfg(Py_3_9)]
16+
t.compile_fail("tests/ui/pyclass_generic_enum.rs");
1317
t.compile_fail("tests/ui/invalid_pyfunction_signatures.rs");
1418
t.compile_fail("tests/ui/invalid_pyfunction_definition.rs");
1519
#[cfg(any(not(Py_LIMITED_API), Py_3_11))]

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy