-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PEP 544: Protocols #224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PEP 544: Protocols #224
Changes from 1 commit
3fa3e48
51b0ca0
993f7b3
e384336
7ea5d41
cdcf62f
1ffed9b
72ceae6
6bea2e8
d5972c3
57d375f
9d4d685
82258d5
5d9fb7c
a6e6d9e
3175013
cbff669
dfccd06
60f4d52
60e7f7f
c90aa1c
b008de1
02cca5c
7d89b6b
0f3732a
95fbf58
cb65bff
817bf2f
2d89ba9
0efcbff
ebd4b17
0de36be
767c58b
efc3154
7d714c3
d4ab050
d9d21c2
4dfbfb2
d51420e
f6240c8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -50,8 +50,8 @@ of the library. Moreover, extensive use of ABCs might impose additional | |
runtime costs. | ||
|
||
The intention of this PEP is to solve all these problems | ||
by allowing to write the above code without explicit base classes in | ||
the class definition, and ``Bucket`` would still be implicitly considered | ||
by allowing users to write the above code without explicit base classes in | ||
the class definition, allowing ``Bucket`` to be implicitly considered | ||
a subtype of both ``Sized`` and ``Iterable[int]`` by static type checkers | ||
using structural [wiki-structural]_ subtyping:: | ||
|
||
|
@@ -66,7 +66,7 @@ using structural [wiki-structural]_ subtyping:: | |
result: int = collect(Bucket()) # Passes type check | ||
|
||
The same functionality will be provided for user-defined protocols, as | ||
specified below. The code with a protocol class matches common Python | ||
specified below. The above code with a protocol class matches common Python | ||
conventions much better. It is also automatically extensible and works | ||
with additional, unrelated classes that happen to implement | ||
the required protocol. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Yes, this is the killer feature. |
||
|
@@ -158,18 +158,18 @@ approaches related to structural subtyping in Python and other languages: | |
The drawback of this approach is the necessity to either subclass | ||
the abstract class or register an implementation explicitly:: | ||
|
||
from abc import ABCMeta | ||
from abc import ABC | ||
|
||
class MyTuple(metaclass=ABCMeta): | ||
class MyTuple(ABC): | ||
pass | ||
|
||
MyTuple.register(tuple) | ||
|
||
assert issubclass(tuple, MyTuple) | ||
assert isinstance((), MyTuple) | ||
|
||
As mentioned in `rationale`_, we want to avoid such necessity, especially | ||
in static context. However, in runtime context, ABCs are good candidates for | ||
As mentioned in the `rationale`_, we want to avoid such necessity, especially | ||
in static context. However, in a runtime context, ABCs are good candidates for | ||
protocol classes and they are already used extensively in | ||
the ``typing`` module. | ||
|
||
|
@@ -319,9 +319,9 @@ Protocol members | |
There are two ways to define methods that are protocol members: | ||
|
||
* Use the ``@abstractmethod`` decorator | ||
* Leave the method body empty, i.e. it should be either a single ``pass`` | ||
statement, a single ``...`` ellipsis literal, or a single | ||
``raise NotImplementedError`` statement. | ||
* Leave the method body empty, i.e. it should be either docstring, | ||
a single ``pass`` statement, a single ``...`` ellipsis literal, | ||
or a single ``raise NotImplementedError`` statement. | ||
|
||
A combination of both may be used if necessary for runtime purposes. | ||
If the method body is not empty but decorated by the ``@abstractmethod`` | ||
|
@@ -383,7 +383,7 @@ annotation should be used as specified by PEP 526. | |
Explicitly declaring implementation | ||
----------------------------------- | ||
|
||
To explicitly declare that a certain class implements a given protocols, they | ||
To explicitly declare that a certain class implements the given protocols, they | ||
can be used as regular base classes. In this case: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm worried this could be pretty confusing: you could have a class that explicitly implements some protocols and implicitly implements others. I don't know of any other language that allows this. |
||
|
||
* A class could use implementations of some methods in protocols, both | ||
|
@@ -545,7 +545,7 @@ relationships are subject to the following rules: | |
if and only if ``X`` implements all protocol members of ``P``. In other | ||
words, subtyping with respect to a protocol is always structural. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I agree. |
||
* Edge case: for recursive protocols, structural subtyping is decided | ||
positively for all situations that are type safe. Continuing | ||
positively for all situations that are type safe. Continuing the | ||
previous example:: | ||
|
||
class Tree(Generic[T]): | ||
|
@@ -590,7 +590,7 @@ classes. For example:: | |
|
||
In addition, we propose to add another special type construct | ||
``All`` that represents intersection types. Although for normal types | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd consider leaving out There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we could include this for three reasons:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with Jukka -- I think it's confusing for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was pushing for I would still like to have it, but that can be a separate PEP. I agree it's complex to implement for concrete types, and better yet, for combinations of both. |
||
it is not very useful, there are many situation where a variable should | ||
it is not very useful, there are many situations where a variable should | ||
implement more than one protocol. Annotation by ``All[Proto1, Proto2, ...]`` | ||
means that a given variable or parameter must implement all protocols | ||
``Proto1``, ``Proto2``, etc. either implicitly or explicitly. Example:: | ||
|
@@ -630,7 +630,7 @@ The same rule applies to variables:: | |
var = Concrete # OK | ||
var().meth() # OK | ||
|
||
Assigning protocol class to a variable is allowed if it is not explicitly | ||
Assigning a protocol class to a variable is allowed if it is not explicitly | ||
typed, and such assignment creates a type alias. For non-protocol classes, | ||
the behavior of ``Type[]`` is not changed. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
No, we want to make ABCs behave the same way. |
||
|
||
|
@@ -672,8 +672,8 @@ at runtime. | |
However, it should be possible for protocol types to implement custom | ||
instance and class checks when this makes sense, similar to how ``Iterable`` | ||
and other ABCs in ``collections.abc`` and ``typing`` already do it, | ||
but this is limited to non-generic and to unsubscripted generic protocols | ||
(statically ``Iterable`` is equivalent to ``Iterable[Any]`). | ||
but this is limited to non-generic and unsubscripted generic protocols | ||
(``Iterable`` is statically equivalent to ``Iterable[Any]`). | ||
The ``typing`` module will define a special ``@runtime`` class decorator that | ||
automatically adds a ``__subclasshook__()`` that provides the same semantics | ||
for class and instance checks as for ``collections.abc`` classes:: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't love the automatic class P(Protocol):
@abstractmethod
def common_method_name(self, x: int) -> int: ...
class X:
<a bunch of methods>
def common_method_name(self) -> None: ...
def do_stuff(o: Union[P, X]) -> int:
if isinstance(o, P):
return o.common_method_name(1) # oops what if it's an X instance instead?
else:
return (do something with an X) Maybe a type checker can warn about this example, but more generally the Another potentially problematic case: class P(Protocol):
x: int
class C:
def initialize(self) -> None:
self.x = 0
c1 = C()
c2 = C()
c2.initialize()
isinstance(c1, P) # False
isinstance(c2, P) # True
def f(x: Union[P, int]) -> None:
if isinstance(x, P):
# type of x is P here
...
else:
# type of x is int here?
print(x + 1)
f(C()) # oops Again, maybe we can live with this, but this could be pretty surprising. I'd argue that requiring an explicit class decorator would be better, since we can then attach warnings about problems like this in the documentation. The programmer would be able to evaluate whether the benefits outweigh the potential for confusion for each protocol and explicitly opt in -- but the default behavior would be safer. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is how I initially wrote this, since I intuitively felt that this could not be 100% reliable. Guido wanted to make it default, by now we have good examples to make this opt-in, so that I will revert this to class decorator. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure I understand how a class decorator would solve the problem. Wouldn't that mean that some classes that implement the protocol (i.e. the decorated ones) will work properly with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The decorator is applied to protocol, not to implementation. This was in a previous draft. At runtime the decorator will add same There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How would the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Basically, if a class has all necessary attributes, then it is a subclass. This is why I mention below that "type checkers will narrow types after such checks by the type erased This is how it is done in |
||
|
@@ -761,7 +761,7 @@ Implementation details | |
---------------------- | ||
|
||
The runtime implementation could be done in pure Python without any | ||
effects on the core interpreter and standard library except in | ||
effects on the core interpreter and standard library except in the | ||
``typing`` module: | ||
|
||
* Define class ``typing.Protocol`` similar to ``typing.Generic``. | ||
|
@@ -779,8 +779,8 @@ effects on the core interpreter and standard library except in | |
be provided at runtime. | ||
|
||
|
||
Changes in typing module | ||
------------------------ | ||
Changes in the typing module | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How do you propose to handle the backwards-incompatibility problem for all currently released versions of Python? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The idea is that there should be no backward incompatibility. The classes in class Whatever:
def __iter__(self): return []
assert isinstance(Whatever(), Iterable) as I mention below "Practically, few changes will be needed in Also, after I implement the changes proposed by Jukka, all code that passes mypy now, will still pass (unless I don't see some subtleties). |
||
---------------------------- | ||
|
||
The following classes in ``typing`` module will be protocols: | ||
|
||
|
@@ -896,8 +896,8 @@ Use assignments to check explicitly that a class implements a protocol | |
---------------------------------------------------------------------- | ||
|
||
In Go language the explicit checks for implementation are performed | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "In Go language" -> either "In Go" or "In the Go language" |
||
via dummy assignments [golang]_. Such way is also possible with the current | ||
proposal. Example:: | ||
via dummy assignments [golang]_. Such a way is also possible with the | ||
current proposal. Example:: | ||
|
||
class A: | ||
def __len__(self) -> float: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have any numbers? We know ABCs aren't free but I don't know if this is worth mentioning unless it's a major factor. The rationale for protocols to be the pythonic, dynamic and idiomatic version of ABCs is strong enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is 2x slower than
The numbers for
typing.Iterable
are even worse, but I have some ideas on how to improve those. The 2x slowdown is still not much, so I use "extensive" and "might". If you think it is not necessary we could remove this altogether.