Skip to content

Protocols #3132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 152 commits into from
Aug 16, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
884481d
Recognize protocols in semanal.py
Mar 26, 2017
c19b6d4
Add issubtype and basic constraints
ilevkivskyi Mar 26, 2017
83c1579
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 26, 2017
b01f5b0
Remove debugging code
ilevkivskyi Mar 26, 2017
16901fc
Minor fixes: semanal, constrains, lib-stub; add few tests
Mar 27, 2017
c8c1247
Allow mypy_extensions.runtime, revert change to test name.
Mar 27, 2017
f430c65
Fix typo
Mar 27, 2017
beea7d2
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 27, 2017
260237a
Enhance constraints, add join (meet seems to be already OK); minor fixes
ilevkivskyi Mar 27, 2017
5414eaf
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 27, 2017
7f8a514
Prohibit protocol instatiation except Type[]; start working on @runtime
ilevkivskyi Mar 28, 2017
92b71a2
Add some tests
ilevkivskyi Mar 28, 2017
83085c7
Restore docstring formstting
ilevkivskyi Mar 28, 2017
27b8570
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 28, 2017
41179c2
Fix serialization; add few more tests
ilevkivskyi Mar 28, 2017
5171844
Add notes for missing protocol members in assignment
ilevkivskyi Mar 30, 2017
db123a3
Add notes for missing members also for arguments and return types
ilevkivskyi Mar 30, 2017
31bf16c
Prohibit definition via self.x in protocol classes
ilevkivskyi Mar 30, 2017
03b5d72
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 30, 2017
65a8546
Make more things in fixtures structural subtypes
ilevkivskyi Mar 30, 2017
2768d76
Structural support for __call__; add more tests
ilevkivskyi Mar 30, 2017
93beb75
Fix lint and tests
ilevkivskyi Mar 30, 2017
5135fd9
Beautify error notes
ilevkivskyi Mar 30, 2017
fdeb89b
Bind self to potential subtype; add tests for structural self-types
ilevkivskyi Mar 31, 2017
1fbcb4a
One more test
ilevkivskyi Mar 31, 2017
d012e38
Move global data to nodes.TypeInfo
ilevkivskyi Mar 31, 2017
801770d
Add comments and docstrings; recognize and treat abstract variables
ilevkivskyi Mar 31, 2017
a625f37
Fix tests and lint
ilevkivskyi Mar 31, 2017
fb6b1ad
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Mar 31, 2017
adb68eb
Add two more tests for recursive
ilevkivskyi Mar 31, 2017
b9a0b2d
Add test for Sized
ilevkivskyi Mar 31, 2017
969f76f
Fix crash in typeanal.py; ignore positional arg names for subtyping; …
ilevkivskyi Apr 1, 2017
2e5de3e
Change order of structural and nominal checks to boost speed; minor f…
ilevkivskyi Apr 1, 2017
1daaf8d
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 1, 2017
2b6d198
Add structural support for tuples and named tuples; add tests
ilevkivskyi Apr 1, 2017
937c629
Add support for staticmethod, classmethod, property, ClassVar + tests…
ilevkivskyi Apr 1, 2017
e8e6661
Speed improvements; more carefull treatment of strucutral inference
ilevkivskyi Apr 2, 2017
8e6306f
Minor fixes and one more test
ilevkivskyi Apr 2, 2017
3f2d707
Start adding some docs
ilevkivskyi Apr 2, 2017
a82413d
Merge remote-tracking branch 'upstream/master' into protocols
Apr 3, 2017
957c76d
Add one more test
Apr 3, 2017
8cf08ec
Try to sync typeshed
ilevkivskyi Apr 3, 2017
e0083d9
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 3, 2017
c9629da
Simplify one test
ilevkivskyi Apr 4, 2017
affec0c
Add more docs
ilevkivskyi Apr 4, 2017
01948dd
Allow incompatible __init__ in structural subtypes
ilevkivskyi Apr 4, 2017
78a62eb
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 4, 2017
0ae409a
Fix minor failure after merge
ilevkivskyi Apr 4, 2017
cb27279
Re-run Travis
ilevkivskyi Apr 4, 2017
5452227
Merge branch 'master' into protocols
ilevkivskyi Apr 5, 2017
798954a
Implement subtype cache
ilevkivskyi Apr 9, 2017
9db4e51
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 9, 2017
7d2327a
Restore correct typeshed commit
ilevkivskyi Apr 9, 2017
88d4fed
Typeshed again
ilevkivskyi Apr 9, 2017
d9187e2
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 13, 2017
3ee1c53
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 16, 2017
85082d5
Switch to NotImplemented instead of False in __eq__
ilevkivskyi Apr 16, 2017
2733e1a
Prohibit using NewType with protocols
ilevkivskyi Apr 17, 2017
232428f
Address CR
ilevkivskyi Apr 17, 2017
4c04e0b
Restore current typeshed commit
ilevkivskyi Apr 17, 2017
e81a3f9
Add some support for __dunder__ = None idiom
ilevkivskyi Apr 18, 2017
b4ca6f0
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 18, 2017
b063767
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 21, 2017
79d8e30
Trigger AppVeyor re-build
Apr 21, 2017
480b977
Disable covariant subtyping for mutable members
ilevkivskyi Apr 22, 2017
483f163
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 22, 2017
70c3ae0
Fix remaining test
ilevkivskyi Apr 22, 2017
509113a
Add much more detailed error messages
ilevkivskyi Apr 22, 2017
0cb0985
Fix one error message in one more test
ilevkivskyi Apr 22, 2017
9f554b6
Fix minor metaclass self-type bug, add various tests
ilevkivskyi Apr 23, 2017
70463a5
Add two more tests (including for Any compatibility in protocols)
ilevkivskyi Apr 23, 2017
78f2011
Merge branch 'master' into protocols
ilevkivskyi Apr 25, 2017
473a2d2
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Apr 27, 2017
8dfe8ea
Add two corner case tests: __getattr__; implicit types
ilevkivskyi Apr 27, 2017
f7e55fa
Merge branch 'master' into protocols
ilevkivskyi May 2, 2017
06a1680
Fix fixture after merge
ilevkivskyi May 2, 2017
204ec82
Merge branch 'master' into protocols
ilevkivskyi May 5, 2017
985d1f7
Add variance checks and modify tests accordingly
ilevkivskyi May 5, 2017
8d2e199
Minor update to docs
ilevkivskyi May 14, 2017
f0471c1
Merge branch 'master' into protocols
ilevkivskyi May 21, 2017
4dfa3e1
Fix trailing whitespace
ilevkivskyi May 21, 2017
6d05060
Detailed error messages also for structural subtyping of tuples and c…
ilevkivskyi May 22, 2017
759409f
Add more protocol variance checks
ilevkivskyi May 22, 2017
83501ff
Merge branch 'master' into protocols
ilevkivskyi May 25, 2017
166b6da
Fix careless merge (#N: formatting)
ilevkivskyi May 25, 2017
d25bcfc
Merge branch 'master' into protocols
ilevkivskyi Jun 1, 2017
d652a69
Fix merge
ilevkivskyi Jun 1, 2017
addac40
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Jun 4, 2017
803ce1e
Fix merge
ilevkivskyi Jun 4, 2017
1c9f6f9
Merge branch 'master' into protocols
ilevkivskyi Jun 8, 2017
3c0411c
Improve notes formatting: add offsets
ilevkivskyi Jun 11, 2017
491b31f
Add more incremental tests
ilevkivskyi Jun 11, 2017
98f0180
Add support for __setattr__ (and few docstrings)
ilevkivskyi Jun 11, 2017
0f55718
Simplify hashes
ilevkivskyi Jun 11, 2017
513c759
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Jun 11, 2017
36f3d6d
Restore typeshed commit
ilevkivskyi Jun 11, 2017
05b70ab
Add explanation of covariant mutable overriding to common issues docs
ilevkivskyi Jun 11, 2017
561856e
Merge branch 'master' into protocols
ilevkivskyi Jun 28, 2017
228f621
Fix brocken merge (now Mapping needs to be Iterable in typing-full fi…
ilevkivskyi Jun 28, 2017
0716b59
Merge branch 'master' into protocols
ilevkivskyi Jun 29, 2017
9cd4e29
Merge branch 'master' into protocols
ilevkivskyi Jul 4, 2017
eb06c55
Fix strict-optional
ilevkivskyi Jul 4, 2017
59aeed2
Fix other merge problems
ilevkivskyi Jul 4, 2017
73d2d69
Merge branch 'master' into protocols
ilevkivskyi Jul 5, 2017
ee18dde
Fix fixtures/dict.pyi fixture
ilevkivskyi Jul 5, 2017
34d1cd1
Fix TypedDict joins with protocols via fallback
ilevkivskyi Jul 5, 2017
fbbd169
Use typing_extensions instead of typing in the docs
Jul 5, 2017
3acd19e
Use pre-PEP-526 syntax in docs where possible
Jul 5, 2017
22ad771
Move generic protocols after nominal generics docs
Jul 5, 2017
4f2391e
Change recommendations about using protocols
Jul 5, 2017
359a43b
Restore correct typeshed commit
Jul 5, 2017
eacff5f
Switch from mypy_extensions to typing_extensions in semanal and in tests
ilevkivskyi Jul 6, 2017
afe1291
Fix lint
ilevkivskyi Jul 6, 2017
1858ed9
Move most protocol erro formatting to messages.py
ilevkivskyi Jul 6, 2017
057a871
Fix few error messages in tests
ilevkivskyi Jul 6, 2017
28c1b9d
Limit display of missing protocol members
ilevkivskyi Jul 6, 2017
ad2bcaa
Limit (and simplify) display of conflicting protocol member types
ilevkivskyi Jul 6, 2017
8af7248
Use pretty callable formatting. Credits to @markkohdev
ilevkivskyi Jul 6, 2017
9ca98b2
Truncate output (max 2 conflicts, max 2 overloads), better formatting…
ilevkivskyi Jul 6, 2017
0cb13b7
Fix lint
ilevkivskyi Jul 6, 2017
fd06408
Factor out some common code
ilevkivskyi Jul 6, 2017
969a64b
Remove some redundant code
ilevkivskyi Jul 6, 2017
e9cbba8
Merge branch 'master' into protocols
ilevkivskyi Jul 14, 2017
e93f839
Fix broken merge
Jul 14, 2017
9292971
First part of CR
Jul 16, 2017
4c3f4e2
Second part of CR (few more comments left)
Jul 16, 2017
e71dff0
Fix strict-optional check
ilevkivskyi Jul 17, 2017
d195964
Don't hash most types (including syntetic)
ilevkivskyi Jul 17, 2017
318bb70
Correct comment in test
ilevkivskyi Jul 17, 2017
3d8782f
Improve structural support for TypedDict and minor things
ilevkivskyi Jul 17, 2017
6e1100c
Don't show detailed erorrs for TypedDicts in simple cases
ilevkivskyi Jul 17, 2017
5bfa3b2
Add also an exception for NamedTuples
ilevkivskyi Jul 17, 2017
82f01b7
Add one more test
ilevkivskyi Jul 17, 2017
c7304bd
Dict is not a protocol
ilevkivskyi Jul 17, 2017
58e6b36
Two minor tweaks: more precise isinstance, require explicit types for…
ilevkivskyi Jul 18, 2017
eefe881
Restore typeshed commit
ilevkivskyi Jul 18, 2017
96a04ae
pythoneval tests should wait until typeshed PR is merged
ilevkivskyi Jul 18, 2017
b1c4d37
Merge branch 'master' into protocols
ilevkivskyi Jul 28, 2017
8b6006c
Fix a silly mistake in merge
ilevkivskyi Jul 28, 2017
f76349b
Replace one last mention of typing with typing_extensions
ilevkivskyi Jul 28, 2017
c3bfe2b
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Jul 29, 2017
e2f4f5d
Merge remote-tracking branch 'upstream/master' into protocols
ilevkivskyi Aug 10, 2017
91fe9fd
Add TypeOfAny
ilevkivskyi Aug 10, 2017
aaea344
Remove some ocassionally added files
ilevkivskyi Aug 10, 2017
27c3e36
Limit unnecessary conflicts display and fix conflict count (with test)
ilevkivskyi Aug 10, 2017
cfa539d
Remove some strange file
ilevkivskyi Aug 12, 2017
713db0c
Implememt first part of recent comments
ilevkivskyi Aug 12, 2017
40d635f
Address second part of recent comments
ilevkivskyi Aug 12, 2017
19073e5
First part of todays comments
ilevkivskyi Aug 14, 2017
50d98c0
Second part of yesterdays comments
ilevkivskyi Aug 15, 2017
9411255
Last part of review comments
ilevkivskyi Aug 15, 2017
f1c915e
One last missing comment
ilevkivskyi Aug 15, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Minor fixes and one more test
  • Loading branch information
ilevkivskyi committed Apr 2, 2017
commit 8e6306f11d7ad4cf590e1399e2828d40f898e322
4 changes: 3 additions & 1 deletion mypy/meet.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,11 +92,13 @@ class C(A, B): ...
return True
if s.type._promote and is_overlapping_types(s.type._promote, t):
return True
if t.type in s.type.mro or s.type in t.type.mro:
return True
if t.type.is_protocol and is_protocol_implementation(s, t):
return True
if s.type.is_protocol and is_protocol_implementation(t, s):
return True
return t.type in s.type.mro or s.type in t.type.mro
return False
if isinstance(t, UnionType):
return any(is_overlapping_types(item, s)
for item in t.items)
Expand Down
8 changes: 5 additions & 3 deletions mypy/semanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -1505,7 +1505,7 @@ def visit_assignment_stmt(self, s: AssignmentStmt) -> None:
allow_tuple_literal = isinstance(s.lvalues[-1], (TupleExpr, ListExpr))
s.type = self.anal_type(s.type, allow_tuple_literal)
if (self.type and self.type.is_protocol and isinstance(lval, NameExpr) and
isinstance(s.rvalue, TempNode)) and s.rvalue.no_rhs:
isinstance(s.rvalue, TempNode) and s.rvalue.no_rhs):
if isinstance(lval.node, Var):
lval.node.is_abstract_var = True
else:
Expand Down Expand Up @@ -1707,8 +1707,8 @@ def analyze_member_lvalue(self, lval: MemberExpr) -> None:
if self.is_self_member_ref(lval):
node = self.type.get(lval.name)
if node is None or isinstance(node.node, Var) and node.node.is_abstract_var:
# Protocol members can't be defined via self
if self.type.is_protocol and node is None:
# Protocol members can't be defined via self
self.fail("Protocol members cannot be defined via assignment to self", lval)
else:
# Implicit attribute definition in __init__.
Expand Down Expand Up @@ -3451,7 +3451,9 @@ def visit_file(self, file: MypyFile, fnam: str, mod_id: str, options: Options) -
('__debug__', bool_type),
])
else:
# We are running tests
# We are running tests without 'bool' in builtins.
# TODO: Find a permanent solution to this problem.
# Maybe add 'bool' to all fixtures?
literal_types.append(('True', AnyType()))

for name, typ in literal_types:
Expand Down
75 changes: 37 additions & 38 deletions mypy/subtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ def is_subtype(left: Type, right: Type,
else:
result = left.accept(SubtypeVisitor(right, type_parameter_checker,
ignore_pos_arg_names=ignore_pos_arg_names))
# Useful for debugging
# Useful for debugging:
# print(left, right, result)
return result

Expand Down Expand Up @@ -145,18 +145,17 @@ def visit_instance(self, left: Instance) -> bool:
rname = right.type.fullname()
# Always try a nominal check if possible,
# there might be errors that a user wants to silence *once*.
# Also, nominal checks are faster.
if not left.type.has_base(rname) and rname != 'builtins.object':
if right.type.is_protocol:
return is_protocol_implementation(left, right)
return False

# Map left type to corresponding right instances.
t = map_instance_to_supertype(left, right.type)

return all(self.check_type_parameter(lefta, righta, tvar.variance)
for lefta, righta, tvar in
zip(t.args, right.args, right.type.defn.type_vars))
if left.type.has_base(rname) or rname == 'builtins.object':
# Map left type to corresponding right instances.
t = map_instance_to_supertype(left, right.type)
nominal = all(self.check_type_parameter(lefta, righta, tvar.variance)
for lefta, righta, tvar in
zip(t.args, right.args, right.type.defn.type_vars))
if nominal:
return True
if right.type.is_protocol and is_protocol_implementation(left, right):
return True
return False
if isinstance(right, TypeType):
item = right.item
if isinstance(item, TupleType):
Expand All @@ -170,7 +169,7 @@ def visit_instance(self, left: Instance) -> bool:
else:
return False
if isinstance(right, CallableType):
# Special case: Instance can by a subtype of Callable.
# Special case: Instance can be a subtype of Callable.
call = find_member('__call__', left, left)
if call:
return is_subtype(call, right)
Expand Down Expand Up @@ -311,10 +310,8 @@ def pop_on_exit(stack: List[Tuple[Instance, Instance]],

def is_protocol_implementation(left: Instance, right: Instance, allow_any: bool = True) -> bool:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about changing allow_any to proper_subtype (and negate the value) or similar for consistency?

"""Check whether 'left' implements the protocol 'right'. If 'allow_any' is False, then
check for a proper subtype. Treat recursive protocols by using a global 'assuming'
structural subtype matrix (in sparse representation). If concurrent type checking
will be implemented, then every thread/process should use its own matrix
(see comment in nodes.TypeInfo).
check for a proper subtype. Treat recursive protocols by using the 'assuming'
structural subtype matrix (in sparse representation), see comment in nodes.TypeInfo.
"""
assert right.type.is_protocol
assuming = right.type.assuming if allow_any else right.type.assuming_proper
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment here explaining how assuming works for recursive protocols, and why will not get an infinite recursion.

Expand All @@ -334,7 +331,7 @@ def is_protocol_implementation(left: Instance, right: Instance, allow_any: bool
if not subtype:
return False
if allow_any:
# nominal check currently ignore arg names
# nominal check currently ignores arg names
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use capitalized 'nominal'.

is_compat = is_subtype(subtype, supertype, ignore_pos_arg_names=True)
else:
is_compat = is_proper_subtype(subtype, supertype)
Expand All @@ -350,6 +347,9 @@ def is_protocol_implementation(left: Instance, right: Instance, allow_any: bool
# this rule is copied from nominal check in checker.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use capitalized 'this'.

if IS_CLASS_OR_STATIC in superflags and IS_CLASS_OR_STATIC not in subflags:
return False
# This additional push serves as poor man's subtype cache.
# (This already gives a decent speed-up.)
# TODO: Implement a general fast subtype cache?
assuming.append((left, right))
return True

Expand Down Expand Up @@ -452,7 +452,7 @@ def map_method(method: FuncBase, itype: Instance, subtype: Instance) -> Type:
by 'find_var_type'.
"""
from mypy.checkmember import bind_self
signature = function_type(method, Instance(itype.type.mro[-1], []))
signature = function_type(method, fallback=Instance(itype.type.mro[-1], []))
signature = bind_self(signature, subtype)
itype = map_instance_to_supertype(itype, method.info)
return expand_type_by_instance(signature, itype)
Expand Down Expand Up @@ -715,24 +715,23 @@ def is_proper_subtype(t: Type, s: Type) -> bool:

if isinstance(t, Instance):
if isinstance(s, Instance):
if s.type.is_protocol:
return is_protocol_implementation(t, s, allow_any=False)
if not t.type.has_base(s.type.fullname()):
return False

def check_argument(left: Type, right: Type, variance: int) -> bool:
if variance == COVARIANT:
return is_proper_subtype(left, right)
elif variance == CONTRAVARIANT:
return is_proper_subtype(right, left)
else:
return sametypes.is_same_type(left, right)

# Map left type to corresponding right instances.
t = map_instance_to_supertype(t, s.type)

return all(check_argument(ta, ra, tvar.variance) for ta, ra, tvar in
zip(t.args, s.args, s.type.defn.type_vars))
if t.type.has_base(s.type.fullname()):
def check_argument(left: Type, right: Type, variance: int) -> bool:
if variance == COVARIANT:
return is_proper_subtype(left, right)
elif variance == CONTRAVARIANT:
return is_proper_subtype(right, left)
else:
return sametypes.is_same_type(left, right)
# Map left type to corresponding right instances.
t = map_instance_to_supertype(t, s.type)
nominal = all(check_argument(ta, ra, tvar.variance) for ta, ra, tvar in
zip(t.args, s.args, s.type.defn.type_vars))
if nominal:
return True
if s.type.is_protocol and is_protocol_implementation(t, s, allow_any=False):
return True
return False
if isinstance(s, CallableType):
call = find_member('__call__', t, t)
if call:
Expand Down
1 change: 1 addition & 0 deletions test-data/unit/check-namedtuple.test
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,7 @@ reveal_type(X._make([5, 'a'])) # E: Revealed type is 'Tuple[builtins.int, built
X._make('a b') # E: Argument 1 to X._make has incompatible type "str"; expected Iterable[Any]\
# N: 'builtins.str' missing following 'typing.Iterable' protocol members:\
# N: __iter__
# Note that the above will not fail with real stubs, since 'str', is a subtype of 'Iterable[str]'

-- # FIX: not a proper class method
-- x = None # type: X
Expand Down
Loading
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy