Skip to content

Commit 384f32c

Browse files
Michael0x2aMichael0x2a
authored
Make revealed type of Final vars distinct from non-Final vars (python#7955)
This diff changes how we format Instances with a last known value when displaying them with `reveal_type`. Previously, we would always ignore the `last_known_value` field: ```python x: Final = 3 reveal_type(x) # N: Revealed type is 'builtins.int' ``` Now, we format it like `Literal[3]?`. Note that we use the question mark suffix as a way of distinguishing the type from true Literal types. ```python x: Final = 3 y: Literal[3] = 3 reveal_type(x) # N: Revealed type is 'Literal[3]?' reveal_type(y) # N: Revealed type is 'Literal[3]' ``` While making this change and auditing our tests, I also discovered we were accidentally copying over the `last_known_value` in a few places by accident. For example: ```python from typing_extensions import Final a = [] a.append(1) a.append(2) # Got no error here? reveal_type(a) # Incorrect revealed type: got builtins.list[Literal[1]?] b = [0, None] b.append(1) # Got no error here? reveal_type(b) # Incorrect revealed type: got builtins.list[Union[Literal[0]?, None]] ``` The other code changes I made were largely cosmetic. Similarly, most of the remaining test changes were just due to places where we were doing something like `reveal_type(0)` or `reveal_type(SomeEnum.BLAH)`. The main motivation behind this diff is that once this lands, it should become much simpler for me to write some tests I'll need while revamping python#7169. It also helps make a somewhat confusing and implicit part of mypy internals more visible.
1 parent 3b5a62e commit 384f32c

20 files changed

+187
-111
lines changed

‎docs/source/literal_types.rst

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -121,13 +121,16 @@ you can instead change the variable to be ``Final`` (see :ref:`final_attrs`):
121121
122122
c: Final = 19
123123
124-
reveal_type(c) # Revealed type is 'int'
125-
expects_literal(c) # ...but this type checks!
124+
reveal_type(c) # Revealed type is 'Literal[19]?'
125+
expects_literal(c) # ...and this type checks!
126126
127127
If you do not provide an explicit type in the ``Final``, the type of ``c`` becomes
128-
context-sensitive: mypy will basically try "substituting" the original assigned
129-
value whenever it's used before performing type checking. So, mypy will type-check
130-
the above program almost as if it were written like so:
128+
*context-sensitive*: mypy will basically try "substituting" the original assigned
129+
value whenever it's used before performing type checking. This is why the revealed
130+
type of ``c`` is ``Literal[19]?``: the question mark at the end reflects this
131+
context-sensitive nature.
132+
133+
For example, mypy will type check the above program almost as if it were written like so:
131134

132135
.. code-block:: python
133136
@@ -138,11 +141,32 @@ the above program almost as if it were written like so:
138141
reveal_type(19)
139142
expects_literal(19)
140143
141-
This is why ``expects_literal(19)`` type-checks despite the fact that ``reveal_type(c)``
142-
reports ``int``.
144+
This means that while changing a variable to be ``Final`` is not quite the same thing
145+
as adding an explicit ``Literal[...]`` annotation, it often leads to the same effect
146+
in practice.
147+
148+
The main cases where the behavior of context-sensitive vs true literal types differ are
149+
when you try using those types in places that are not explicitly expecting a ``Literal[...]``.
150+
For example, compare and contrast what happens when you try appending these types to a list:
151+
152+
.. code-block:: python
153+
154+
from typing_extensions import Final, Literal
155+
156+
a: Final = 19
157+
b: Literal[19] = 19
158+
159+
# Mypy will chose to infer List[int] here.
160+
list_of_ints = []
161+
list_of_ints.append(a)
162+
reveal_type(list_of_ints) # Revealed type is 'List[int]'
163+
164+
# But if the variable you're appending is an explicit Literal, mypy
165+
# will infer List[Literal[19]].
166+
list_of_lits = []
167+
list_of_lits.append(b)
168+
reveal_type(list_of_lits) # Revealed type is 'List[Literal[19]]'
143169
144-
So while changing a variable to be ``Final`` is not quite the same thing as adding
145-
an explicit ``Literal[...]`` annotation, it often leads to the same effect in practice.
146170
147171
Limitations
148172
***********

‎mypy/checker.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4099,6 +4099,7 @@ def named_generic_type(self, name: str, args: List[Type]) -> Instance:
40994099
the name refers to a compatible generic type.
41004100
"""
41014101
info = self.lookup_typeinfo(name)
4102+
args = [remove_instance_last_known_values(arg) for arg in args]
41024103
# TODO: assert len(args) == len(info.defn.type_vars)
41034104
return Instance(info, args)
41044105

‎mypy/checkexpr.py

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
import mypy.checker
4040
from mypy import types
4141
from mypy.sametypes import is_same_type
42-
from mypy.erasetype import replace_meta_vars, erase_type
42+
from mypy.erasetype import replace_meta_vars, erase_type, remove_instance_last_known_values
4343
from mypy.maptype import map_instance_to_supertype
4444
from mypy.messages import MessageBuilder
4545
from mypy import message_registry
@@ -3045,12 +3045,13 @@ def check_lst_expr(self, items: List[Expression], fullname: str,
30453045
self.named_type('builtins.function'),
30463046
name=tag,
30473047
variables=[tvdef])
3048-
return self.check_call(constructor,
3049-
[(i.expr if isinstance(i, StarExpr) else i)
3050-
for i in items],
3051-
[(nodes.ARG_STAR if isinstance(i, StarExpr) else nodes.ARG_POS)
3052-
for i in items],
3053-
context)[0]
3048+
out = self.check_call(constructor,
3049+
[(i.expr if isinstance(i, StarExpr) else i)
3050+
for i in items],
3051+
[(nodes.ARG_STAR if isinstance(i, StarExpr) else nodes.ARG_POS)
3052+
for i in items],
3053+
context)[0]
3054+
return remove_instance_last_known_values(out)
30543055

30553056
def visit_tuple_expr(self, e: TupleExpr) -> Type:
30563057
"""Type check a tuple expression."""

‎mypy/checkmember.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -691,7 +691,12 @@ def analyze_class_attribute_access(itype: Instance,
691691

692692
if info.is_enum and not (mx.is_lvalue or is_decorated or is_method):
693693
enum_literal = LiteralType(name, fallback=itype)
694-
return itype.copy_modified(last_known_value=enum_literal)
694+
# When we analyze enums, the corresponding Instance is always considered to be erased
695+
# due to how the signature of Enum.__new__ is `(cls: Type[_T], value: object) -> _T`
696+
# in typeshed. However, this is really more of an implementation detail of how Enums
697+
# are typed, and we really don't want to treat every single Enum value as if it were
698+
# from type variable substitution. So we reset the 'erased' field here.
699+
return itype.copy_modified(erased=False, last_known_value=enum_literal)
695700

696701
t = node.type
697702
if t:

‎mypy/erasetype.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -140,9 +140,12 @@ class LastKnownValueEraser(TypeTranslator):
140140
Instance types."""
141141

142142
def visit_instance(self, t: Instance) -> Type:
143-
if t.last_known_value:
144-
return t.copy_modified(last_known_value=None)
145-
return t
143+
if not t.last_known_value and not t.args:
144+
return t
145+
return t.copy_modified(
146+
args=[a.accept(self) for a in t.args],
147+
last_known_value=None,
148+
)
146149

147150
def visit_type_alias_type(self, t: TypeAliasType) -> Type:
148151
# Type aliases can't contain literal values, because they are

‎mypy/types.py

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -830,13 +830,14 @@ def deserialize(cls, data: Union[JsonDict, str]) -> 'Instance':
830830

831831
def copy_modified(self, *,
832832
args: Bogus[List[Type]] = _dummy,
833+
erased: Bogus[bool] = _dummy,
833834
last_known_value: Bogus[Optional['LiteralType']] = _dummy) -> 'Instance':
834835
return Instance(
835836
self.type,
836837
args if args is not _dummy else self.args,
837838
self.line,
838839
self.column,
839-
self.erased,
840+
erased if erased is not _dummy else self.erased,
840841
last_known_value if last_known_value is not _dummy else self.last_known_value,
841842
)
842843

@@ -1988,7 +1989,13 @@ def visit_deleted_type(self, t: DeletedType) -> str:
19881989
return "<Deleted '{}'>".format(t.source)
19891990

19901991
def visit_instance(self, t: Instance) -> str:
1991-
s = t.type.fullname or t.type.name or '<???>'
1992+
if t.last_known_value and not t.args:
1993+
# Instances with a literal fallback should never be generic. If they are,
1994+
# something went wrong so we fall back to showing the full Instance repr.
1995+
s = '{}?'.format(t.last_known_value)
1996+
else:
1997+
s = t.type.fullname or t.type.name or '<???>'
1998+
19921999
if t.erased:
19932000
s += '*'
19942001
if t.args != []:

‎test-data/unit/check-columns.test

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -308,7 +308,7 @@ if int():
308308

309309
[case testColumnRevealedType]
310310
if int():
311-
reveal_type(1) # N:17: Revealed type is 'builtins.int'
311+
reveal_type(1) # N:17: Revealed type is 'Literal[1]?'
312312

313313
[case testColumnNonOverlappingEqualityCheck]
314314
# flags: --strict-equality

‎test-data/unit/check-enum.test

Lines changed: 33 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ class Medal(Enum):
66
gold = 1
77
silver = 2
88
bronze = 3
9-
reveal_type(Medal.bronze) # N: Revealed type is '__main__.Medal*'
9+
reveal_type(Medal.bronze) # N: Revealed type is 'Literal[__main__.Medal.bronze]?'
1010
m = Medal.gold
1111
if int():
1212
m = 1 # E: Incompatible types in assignment (expression has type "int", variable has type "Medal")
@@ -20,7 +20,7 @@ class Medal(metaclass=EnumMeta):
2020
# Without __init__ the definition fails at runtime, but we want to verify that mypy
2121
# uses `enum.EnumMeta` and not `enum.Enum` as the definition of what is enum.
2222
def __init__(self, *args): pass
23-
reveal_type(Medal.bronze) # N: Revealed type is '__main__.Medal'
23+
reveal_type(Medal.bronze) # N: Revealed type is 'Literal[__main__.Medal.bronze]?'
2424
m = Medal.gold
2525
if int():
2626
m = 1 # E: Incompatible types in assignment (expression has type "int", variable has type "Medal")
@@ -34,7 +34,7 @@ class Medal(Achievement):
3434
bronze = None
3535
# See comment in testEnumFromEnumMetaBasics
3636
def __init__(self, *args): pass
37-
reveal_type(Medal.bronze) # N: Revealed type is '__main__.Medal'
37+
reveal_type(Medal.bronze) # N: Revealed type is 'Literal[__main__.Medal.bronze]?'
3838
m = Medal.gold
3939
if int():
4040
m = 1 # E: Incompatible types in assignment (expression has type "int", variable has type "Medal")
@@ -53,7 +53,7 @@ class Truth(Enum):
5353
false = False
5454
x = ''
5555
x = Truth.true.name
56-
reveal_type(Truth.true.name) # N: Revealed type is 'builtins.str'
56+
reveal_type(Truth.true.name) # N: Revealed type is 'Literal['true']?'
5757
reveal_type(Truth.false.value) # N: Revealed type is 'builtins.bool'
5858
[builtins fixtures/bool.pyi]
5959

@@ -246,7 +246,7 @@ class A:
246246
a = A()
247247
reveal_type(a.x)
248248
[out]
249-
main:8: note: Revealed type is '__main__.E@4*'
249+
main:8: note: Revealed type is '__main__.E@4'
250250

251251
[case testEnumInClassBody]
252252
from enum import Enum
@@ -270,9 +270,9 @@ reveal_type(E.bar.value)
270270
reveal_type(I.bar)
271271
reveal_type(I.baz.value)
272272
[out]
273-
main:4: note: Revealed type is '__main__.E*'
273+
main:4: note: Revealed type is 'Literal[__main__.E.foo]?'
274274
main:5: note: Revealed type is 'Any'
275-
main:6: note: Revealed type is '__main__.I*'
275+
main:6: note: Revealed type is 'Literal[__main__.I.bar]?'
276276
main:7: note: Revealed type is 'builtins.int'
277277

278278
[case testFunctionalEnumListOfStrings]
@@ -282,8 +282,8 @@ F = IntEnum('F', ['bar', 'baz'])
282282
reveal_type(E.foo)
283283
reveal_type(F.baz)
284284
[out]
285-
main:4: note: Revealed type is '__main__.E*'
286-
main:5: note: Revealed type is '__main__.F*'
285+
main:4: note: Revealed type is 'Literal[__main__.E.foo]?'
286+
main:5: note: Revealed type is 'Literal[__main__.F.baz]?'
287287

288288
[case testFunctionalEnumListOfPairs]
289289
from enum import Enum, IntEnum
@@ -294,10 +294,10 @@ reveal_type(F.baz)
294294
reveal_type(E.foo.value)
295295
reveal_type(F.bar.name)
296296
[out]
297-
main:4: note: Revealed type is '__main__.E*'
298-
main:5: note: Revealed type is '__main__.F*'
299-
main:6: note: Revealed type is 'builtins.int'
300-
main:7: note: Revealed type is 'builtins.str'
297+
main:4: note: Revealed type is 'Literal[__main__.E.foo]?'
298+
main:5: note: Revealed type is 'Literal[__main__.F.baz]?'
299+
main:6: note: Revealed type is 'Literal[1]?'
300+
main:7: note: Revealed type is 'Literal['bar']?'
301301

302302
[case testFunctionalEnumDict]
303303
from enum import Enum, IntEnum
@@ -308,10 +308,10 @@ reveal_type(F.baz)
308308
reveal_type(E.foo.value)
309309
reveal_type(F.bar.name)
310310
[out]
311-
main:4: note: Revealed type is '__main__.E*'
312-
main:5: note: Revealed type is '__main__.F*'
313-
main:6: note: Revealed type is 'builtins.int'
314-
main:7: note: Revealed type is 'builtins.str'
311+
main:4: note: Revealed type is 'Literal[__main__.E.foo]?'
312+
main:5: note: Revealed type is 'Literal[__main__.F.baz]?'
313+
main:6: note: Revealed type is 'Literal[1]?'
314+
main:7: note: Revealed type is 'Literal['bar']?'
315315

316316
[case testFunctionalEnumErrors]
317317
from enum import Enum, IntEnum
@@ -363,10 +363,10 @@ main:22: error: "Type[W]" has no attribute "c"
363363
from enum import Flag, IntFlag
364364
A = Flag('A', 'x y')
365365
B = IntFlag('B', 'a b')
366-
reveal_type(A.x) # N: Revealed type is '__main__.A*'
367-
reveal_type(B.a) # N: Revealed type is '__main__.B*'
368-
reveal_type(A.x.name) # N: Revealed type is 'builtins.str'
369-
reveal_type(B.a.name) # N: Revealed type is 'builtins.str'
366+
reveal_type(A.x) # N: Revealed type is 'Literal[__main__.A.x]?'
367+
reveal_type(B.a) # N: Revealed type is 'Literal[__main__.B.a]?'
368+
reveal_type(A.x.name) # N: Revealed type is 'Literal['x']?'
369+
reveal_type(B.a.name) # N: Revealed type is 'Literal['a']?'
370370

371371
# TODO: The revealed type should be 'int' here
372372
reveal_type(A.x.value) # N: Revealed type is 'Any'
@@ -381,7 +381,7 @@ class A:
381381
a = A()
382382
reveal_type(a.x)
383383
[out]
384-
main:7: note: Revealed type is '__main__.A.E@4*'
384+
main:7: note: Revealed type is '__main__.A.E@4'
385385

386386
[case testFunctionalEnumInClassBody]
387387
from enum import Enum
@@ -451,19 +451,19 @@ F = Enum('F', 'a b')
451451
[rechecked]
452452
[stale]
453453
[out1]
454-
main:2: note: Revealed type is 'm.E*'
455-
main:3: note: Revealed type is 'm.F*'
454+
main:2: note: Revealed type is 'Literal[m.E.a]?'
455+
main:3: note: Revealed type is 'Literal[m.F.b]?'
456456
[out2]
457-
main:2: note: Revealed type is 'm.E*'
458-
main:3: note: Revealed type is 'm.F*'
457+
main:2: note: Revealed type is 'Literal[m.E.a]?'
458+
main:3: note: Revealed type is 'Literal[m.F.b]?'
459459

460460
[case testEnumAuto]
461461
from enum import Enum, auto
462462
class Test(Enum):
463463
a = auto()
464464
b = auto()
465465

466-
reveal_type(Test.a) # N: Revealed type is '__main__.Test*'
466+
reveal_type(Test.a) # N: Revealed type is 'Literal[__main__.Test.a]?'
467467
[builtins fixtures/primitives.pyi]
468468

469469
[case testEnumAttributeAccessMatrix]
@@ -689,31 +689,31 @@ else:
689689

690690
if x is z:
691691
reveal_type(x) # N: Revealed type is 'Literal[__main__.Foo.A]'
692-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
692+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
693693
accepts_foo_a(z)
694694
else:
695695
reveal_type(x) # N: Revealed type is 'Union[Literal[__main__.Foo.B], Literal[__main__.Foo.C]]'
696-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
696+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
697697
accepts_foo_a(z)
698698
if z is x:
699699
reveal_type(x) # N: Revealed type is 'Literal[__main__.Foo.A]'
700-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
700+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
701701
accepts_foo_a(z)
702702
else:
703703
reveal_type(x) # N: Revealed type is 'Union[Literal[__main__.Foo.B], Literal[__main__.Foo.C]]'
704-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
704+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
705705
accepts_foo_a(z)
706706

707707
if y is z:
708708
reveal_type(y) # N: Revealed type is 'Literal[__main__.Foo.A]'
709-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
709+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
710710
accepts_foo_a(z)
711711
else:
712712
reveal_type(y) # No output: this branch is unreachable
713713
reveal_type(z) # No output: this branch is unreachable
714714
if z is y:
715715
reveal_type(y) # N: Revealed type is 'Literal[__main__.Foo.A]'
716-
reveal_type(z) # N: Revealed type is '__main__.Foo*'
716+
reveal_type(z) # N: Revealed type is 'Literal[__main__.Foo.A]?'
717717
accepts_foo_a(z)
718718
else:
719719
reveal_type(y) # No output: this branch is unreachable

‎test-data/unit/check-errorcodes.test

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ class A:
2828
pass
2929

3030
[case testErrorCodeNoteHasNoCode]
31-
reveal_type(1) # N: Revealed type is 'builtins.int'
31+
reveal_type(1) # N: Revealed type is 'Literal[1]?'
3232

3333
[case testErrorCodeSyntaxError]
3434
1 '' # E: invalid syntax [syntax]

‎test-data/unit/check-expressions.test

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1911,7 +1911,7 @@ from typing import Union
19111911
reveal_type(1 if bool() else 2) # N: Revealed type is 'builtins.int'
19121912
reveal_type(1 if bool() else '') # N: Revealed type is 'builtins.object'
19131913
x: Union[int, str] = reveal_type(1 if bool() else '') \
1914-
# N: Revealed type is 'Union[builtins.int, builtins.str]'
1914+
# N: Revealed type is 'Union[Literal[1]?, Literal['']?]'
19151915
class A:
19161916
pass
19171917
class B(A):
@@ -1934,7 +1934,7 @@ reveal_type(d if bool() else b) # N: Revealed type is '__main__.A'
19341934
[case testConditionalExpressionUnionWithAny]
19351935
from typing import Union, Any
19361936
a: Any
1937-
x: Union[int, str] = reveal_type(a if int() else 1) # N: Revealed type is 'Union[Any, builtins.int]'
1937+
x: Union[int, str] = reveal_type(a if int() else 1) # N: Revealed type is 'Union[Any, Literal[1]?]'
19381938
reveal_type(a if int() else 1) # N: Revealed type is 'Any'
19391939

19401940

@@ -2207,7 +2207,7 @@ d() # E: "D[str, int]" not callable
22072207
[builtins fixtures/dict.pyi]
22082208

22092209
[case testRevealType]
2210-
reveal_type(1) # N: Revealed type is 'builtins.int'
2210+
reveal_type(1) # N: Revealed type is 'Literal[1]?'
22112211

22122212
[case testRevealLocals]
22132213
x = 1

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy