Skip to content

Edit error marker comments on some tests #2048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 22, 2025
Merged

Edit error marker comments on some tests #2048

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f5930a5
edit error markers and regenerate results
yangdanny97 Jul 18, 2025
b0ffdc6
Merge branch 'main' into edit-error-comments
rchen152 Jul 22, 2025
db2544b
Add back newline
rchen152 Jul 22, 2025
8d29727
Add back newline
rchen152 Jul 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
edit error markers and regenerate results
  • Loading branch information
@yangdanny97
yangdanny97 committed Jul 18, 2025
commit f5930a56f247d721c6affb8d6ff5e1a40f75769e
2 changes: 1 addition & 1 deletion conformance/results/mypy/aliases_recursive.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ aliases_recursive.py:38: error: Incompatible types in assignment (expression has
aliases_recursive.py:39: error: Name "t6" already defined on line 38 [no-redef]
aliases_recursive.py:50: error: Dict entry 0 has incompatible type "str": "list[int]"; expected "str": "str | int | Mapping[str, RecursiveMapping]" [dict-item]
aliases_recursive.py:51: error: Dict entry 2 has incompatible type "str": "list[int]"; expected "str": "str | int | Mapping[str, RecursiveMapping]" [dict-item]
aliases_recursive.py:52: error: Dict entry 2 has incompatible type "str": "list[int]"; expected "str": "str | int | Mapping[str, RecursiveMapping]" [dict-item]
aliases_recursive.py:52: error: Dict entry 2 has incompatible type "str": "dict[str, str | int | list[int]]"; expected "str": "str | int | Mapping[str, RecursiveMapping]" [dict-item]
aliases_recursive.py:63: error: List item 0 has incompatible type "float"; expected "GenericTypeAlias1[str] | str" [list-item]
aliases_recursive.py:69: error: List item 0 has incompatible type "float"; expected "GenericTypeAlias2[str, int] | str | int" [list-item]
aliases_recursive.py:72: error: Invalid recursive alias: a union item of itself [misc]
Expand Down
2 changes: 1 addition & 1 deletion conformance/results/mypy/classes_classvar.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ classes_classvar.py:70: error: ClassVar can only be used for assignments in clas
classes_classvar.py:71: error: ClassVar can only be used for assignments in class body [misc]
classes_classvar.py:73: error: ClassVar can only be used for assignments in class body [misc]
classes_classvar.py:77: error: ClassVar can only be used for assignments in class body [misc]
classes_classvar.py:78: error: ClassVar[...] can't be used inside a type alias [valid-type]
classes_classvar.py:111: error: Cannot assign to class variable "stats" via instance [misc]
classes_classvar.py:140: error: Incompatible types in assignment (expression has type "ProtoAImpl", variable has type "ProtoA") [assignment]
classes_classvar.py:140: note: "ProtoAImpl" is missing following "ProtoA" protocol member:
Expand All @@ -30,6 +31,5 @@ classes_classvar.py:140: note: Protocol member ProtoA.y expected class variable,
conformance_automated = "Fail"
errors_diff = """
Line 47: Expected 1 errors
Line 78: Expected 1 errors
Line 67: Unexpected errors ['classes_classvar.py:67: error: Invalid type: ClassVar nested inside other type [valid-type]']
"""
19 changes: 2 additions & 17 deletions conformance/results/mypy/dataclasses_kwonly.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,9 @@
conformant = "Partial"
notes = """
Incorrectly rejects kw_only field with default before positional field.
"""
conformant = "Pass"
output = """
dataclasses_kwonly.py:23: error: Too many positional arguments for "DC1" [misc]
dataclasses_kwonly.py:29: error: Attributes without a default cannot follow attributes with one [misc]
dataclasses_kwonly.py:32: error: Too many positional arguments for "DC2" [misc]
dataclasses_kwonly.py:32: error: Too few arguments for "DC2" [call-arg]
dataclasses_kwonly.py:32: error: Argument 1 to "DC2" has incompatible type "str"; expected "int" [arg-type]
dataclasses_kwonly.py:35: error: "DC2" gets multiple values for keyword argument "b" [misc]
dataclasses_kwonly.py:35: error: Too few arguments for "DC2" [call-arg]
dataclasses_kwonly.py:35: error: Argument 1 to "DC2" has incompatible type "str"; expected "int" [arg-type]
dataclasses_kwonly.py:38: error: Too many positional arguments for "DC2" [misc]
dataclasses_kwonly.py:38: error: Argument 1 to "DC2" has incompatible type "str"; expected "int" [arg-type]
dataclasses_kwonly.py:38: error: Argument 2 to "DC2" has incompatible type "int"; expected "str" [arg-type]
dataclasses_kwonly.py:53: error: Too many positional arguments for "DC3" [misc]
"""
conformance_automated = "Fail"
conformance_automated = "Pass"
errors_diff = """
Line 29: Unexpected errors ['dataclasses_kwonly.py:29: error: Attributes without a default cannot follow attributes with one [misc]']
Line 32: Unexpected errors ['dataclasses_kwonly.py:32: error: Too many positional arguments for "DC2" [misc]', 'dataclasses_kwonly.py:32: error: Too few arguments for "DC2" [call-arg]', 'dataclasses_kwonly.py:32: error: Argument 1 to "DC2" has incompatible type "str"; expected "int" [arg-type]']
Line 35: Unexpected errors ['dataclasses_kwonly.py:35: error: "DC2" gets multiple values for keyword argument "b" [misc]', 'dataclasses_kwonly.py:35: error: Too few arguments for "DC2" [call-arg]', 'dataclasses_kwonly.py:35: error: Argument 1 to "DC2" has incompatible type "str"; expected "int" [arg-type]']
"""
2 changes: 1 addition & 1 deletion conformance/results/mypy/directives_reveal_type.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
conformant = "Pass"
output = """
directives_reveal_type.py:14: note: Revealed type is "Union[builtins.int, builtins.str]"
directives_reveal_type.py:14: note: Revealed type is "builtins.int | builtins.str"
directives_reveal_type.py:15: note: Revealed type is "builtins.list[builtins.int]"
directives_reveal_type.py:16: note: Revealed type is "Any"
directives_reveal_type.py:17: note: Revealed type is "directives_reveal_type.ForwardReference"
Expand Down
4 changes: 3 additions & 1 deletion conformance/results/mypy/enums_behaviors.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ conformant = "Pass"
errors_diff = """
"""
output = """
enums_behaviors.py:39: error: Cannot extend enum with existing members: "Shape" [misc]
enums_behaviors.py:28: error: Expression is of type "Color", not "Literal[Color.RED]" [assert-type]
enums_behaviors.py:32: error: Expression is of type "Color", not "Literal[Color.BLUE]" [assert-type]
enums_behaviors.py:44: error: Cannot extend enum with existing members: "Shape" [misc]
"""
conformance_automated = "Pass"
4 changes: 2 additions & 2 deletions conformance/results/mypy/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
version = "mypy 1.16.1"
test_duration = 2.2
version = "mypy 1.17.0"
test_duration = 1.9
4 changes: 3 additions & 1 deletion conformance/results/pyre/enums_behaviors.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,7 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
enums_behaviors.py:39:0 Invalid inheritance [39]: Cannot inherit from final enum `Shape`. Enums with defined members cannot be extended.
enums_behaviors.py:28:0 Assert type [70]: Expected `typing_extensions.Literal[Color.RED]` but got `Color`.
enums_behaviors.py:32:0 Assert type [70]: Expected `typing_extensions.Literal[Color.BLUE]` but got `Color`.
enums_behaviors.py:44:0 Invalid inheritance [39]: Cannot inherit from final enum `Shape`. Enums with defined members cannot be extended.
"""
4 changes: 0 additions & 4 deletions conformance/results/pyre/generics_paramspec_specialization.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ notes = """
Reports error for legitimate use of `...` to specialize ParamSpec
"""
output = """
generics_paramspec_specialization.py:32:14 Invalid type parameters [24]: Callable parameters expected for parameter specification `P2`, but a single type `...` was given for generic type ClassB.
generics_paramspec_specialization.py:36:14 Invalid type parameters [24]: Callable parameters expected for parameter specification `P1`, but a single type `...` was given for generic type ClassA.
generics_paramspec_specialization.py:44:14 Invalid type parameters [24]: Callable parameters expected for parameter specification `P1`, but a single type `int` was given for generic type ClassA.
generics_paramspec_specialization.py:53:4 Too many arguments [19]: PositionalOnly call expects 2 positional arguments, 3 were provided.
generics_paramspec_specialization.py:54:4 Too many arguments [19]: PositionalOnly call expects 2 positional arguments, 3 were provided.
Expand All @@ -15,8 +13,6 @@ generics_paramspec_specialization.py:61:4 Too many arguments [19]: PositionalOnl
"""
conformance_automated = "Fail"
errors_diff = """
Line 32: Unexpected errors ['generics_paramspec_specialization.py:32:14 Invalid type parameters [24]: Callable parameters expected for parameter specification `P2`, but a single type `...` was given for generic type ClassB.']
Line 36: Unexpected errors ['generics_paramspec_specialization.py:36:14 Invalid type parameters [24]: Callable parameters expected for parameter specification `P1`, but a single type `...` was given for generic type ClassA.']
Line 53: Unexpected errors ['generics_paramspec_specialization.py:53:4 Too many arguments [19]: PositionalOnly call expects 2 positional arguments, 3 were provided.']
Line 59: Unexpected errors ['generics_paramspec_specialization.py:59:4 Too many arguments [19]: PositionalOnly call expects 2 positional arguments, 3 were provided.']
"""
4 changes: 2 additions & 2 deletions conformance/results/pyre/qualifiers_final_decorator.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ qualifiers_final_decorator.py:126:0 Invalid inheritance [39]: `final` cannot be
"""
conformance_automated = "Fail"
errors_diff = """
Lines 80, 89: Expected error (tag 'Derived3')
Lines 94, 102: Expected error (tag 'Derived4')
Lines 80, 81, 89: Expected error (tag 'Derived3')
Lines 94, 95, 102: Expected error (tag 'Derived4')
Line 51: Unexpected errors ['qualifiers_final_decorator.py:51:4 Incompatible overload [43]: This definition does not have the same decorators as the preceding overload(s).']
"""
4 changes: 2 additions & 2 deletions conformance/results/pyre/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
version = "pyre 0.9.23"
test_duration = 10.7
version = "pyre 0.9.25"
test_duration = 7.4
4 changes: 3 additions & 1 deletion conformance/results/pyright/enums_behaviors.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ conformant = "Pass"
errors_diff = """
"""
output = """
enums_behaviors.py:39:21 - error: Enum class "Shape" is final and cannot be subclassed (reportGeneralTypeIssues)
enums_behaviors.py:28:13 - error: "assert_type" mismatch: expected "Literal[Color.RED]" but received "Color" (reportAssertTypeFailure)
enums_behaviors.py:32:13 - error: "assert_type" mismatch: expected "Literal[Color.BLUE]" but received "Color" (reportAssertTypeFailure)
enums_behaviors.py:44:21 - error: Enum class "Shape" is final and cannot be subclassed (reportGeneralTypeIssues)
"""
conformance_automated = "Pass"
4 changes: 2 additions & 2 deletions conformance/results/pyright/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
version = "pyright 1.1.402"
test_duration = 1.8
version = "pyright 1.1.403"
test_duration = 1.4
16 changes: 8 additions & 8 deletions conformance/results/results.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,14 +158,14 @@ <h3>Python Type System Conformance Test Results</h3>
</header>
<div class="table_container"><table><tbody>
<tr><th class="col1">&nbsp;</th>
<th class='tc-header'><div class='tc-name'>mypy 1.16.1</div>
<div class='tc-time'>2.2sec</div>
<th class='tc-header'><div class='tc-name'>mypy 1.17.0</div>
<div class='tc-time'>1.9sec</div>
</th>
<th class='tc-header'><div class='tc-name'>pyright 1.1.402</div>
<div class='tc-time'>1.8sec</div>
<th class='tc-header'><div class='tc-name'>pyright 1.1.403</div>
<div class='tc-time'>1.4sec</div>
</th>
<th class='tc-header'><div class='tc-name'>pyre 0.9.23</div>
<div class='tc-time'>10.7sec</div>
<th class='tc-header'><div class='tc-name'>pyre 0.9.25</div>
<div class='tc-time'>7.4sec</div>
</th>
</tr>
<tr><th class="column" colspan="4">
Expand Down Expand Up @@ -324,7 +324,7 @@ <h3>Python Type System Conformance Test Results</h3>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;generics_type_erasure</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Infers Node[Never] instead of Node[Any] when argument is not provided.</p><p>False negative on instance attribute access on type(node).</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Missing error regarding `type(instance).generic_attribute`.</p></span></div></th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not erase unspecified type variables to `Any` prior to `assert_type` handling.</p><p>False negatives on instance attribute access on the type.</p><p>Does not infer type of `DefaultDict` with explicit type parameters on constructor.</p><p>False negatives on assert_type uses.</p></span></div></th>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;generics_typevartuple_args</th>
Expand Down Expand Up @@ -423,7 +423,7 @@ <h3>Python Type System Conformance Test Results</h3>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;aliases_newtype</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>`NewType`s are considered classes, not functions.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>`NewType`s are considered classes, not functions.</p></span></div></th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not reject use of NewType in `isinstance` call.</p><p>Does not reject use of NewType in class definition statement.</p><p>Does not report inconsistency between name of NewType and assigned identifier name.</p><p>Does not reject use of NewType with generic class with TypeVar.</p><p>Does not reject use of NewType with protocol class.</p><p>Does not reject use of NewType with TypedDict class.</p><p>Does not reject use of NewType with Any.</p></span></div></th>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;aliases_recursive</th>
Expand Down
11 changes: 8 additions & 3 deletions conformance/tests/enums_behaviors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# Specification: https://typing.readthedocs.io/en/latest/spec/enums.html#enum-definition

from enum import Enum
from typing import assert_type
from typing import assert_type, Literal

# > Enum classes are iterable and indexable, and they can be called with a
# > value to look up the enum member with that value. Type checkers should
Expand All @@ -23,8 +23,13 @@ class Color(Enum):
# > constructor. Instead, the call performs a value-based lookup of an
# > enum member.

assert_type(Color["RED"], Color) # 'Literal[Color.RED]' is also acceptable
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

felt odd to assert_type here and say that another type is also acceptable. I tried to put both versions down, so that type checkers have discretion on which one to implement w/o the test failing.

I assume that if a checker emits an error on both lines the test will fail - could the maintainers please confirm this assumption?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right; this is documented in the "Test case syntax" section in conformance/README.

assert_type(Color(3), Color) # 'Literal[Color.BLUE]' is also acceptable
# 'Literal[Color.RED]' and 'Color' are both acceptable
assert_type(Color["RED"], Color) # E[red]
assert_type(Color["RED"], Literal[Color.RED]) # E[red]

# 'Literal[Color.BLUE]' and 'Color' are both acceptable
assert_type(Color(3), Color) # E[blue]
assert_type(Color(3), Literal[Color.BLUE]) # E[blue]


# > An Enum class with one or more defined members cannot be subclassed.
Expand Down
6 changes: 3 additions & 3 deletions conformance/tests/qualifiers_final_decorator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def method3() -> None: # E[method3]
pass

@overload # E[method4]
def method4(self, x: int) -> int:
def method4(self, x: int) -> int: # E[method4]
Copy link
Contributor Author

@yangdanny97 yangdanny97 Jul 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pyrefly emits diagnostics on the line of the def, not the line with the @overload

This is similar to what I did in #1980

...

@overload
Expand All @@ -78,7 +78,7 @@ def method4(self, x: int | str) -> int | str: # E[method4]

class Derived3(Base3):
@overload # E[Derived3]
def method(self, x: int) -> int:
def method(self, x: int) -> int: # E[Derived3]
...

@overload # E[Derived3-2]
Expand All @@ -92,7 +92,7 @@ def method(self, x: int | str) -> int | str: # E[Derived3]

class Derived4(Base4):
@overload # E[Derived4]
def method(self, x: int) -> int:
def method(self, x: int) -> int: # E[Derived4]
...

@overload
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy