Fix 7190:__init_subclass__ is not type-checked #7452

gantsevdenis · 2019-09-03T16:37:13Z

ilevkivskyi

Thank you for PR! I like this approach. I left few minor comments, could you also please move the logic to a separate method? (Like e.g. check_protocol_variance() below.)

mypy/checker.py

ilevkivskyi · 2019-09-04T12:31:18Z

mypy/checker.py

@@ -1647,6 +1647,24 @@ def visit_class_def(self, defn: ClassDef) -> None:
                with self.scope.push_class(defn.info):
                    self.accept(defn.defs)
            self.binder = old_binder
+            for base in typ.mro[1:]:
+                if base.name() != 'object' and base.defn.info:


Checking for base.defn.info looks suspicious, at this stage it must be always present, if it isn't, it is a bug elsewhere. Is this actually needed?

actually there are AssertionErrors during tests, but I didn't succeed to reproduce them myself..

mypy/checker.py

ilevkivskyi · 2019-09-04T12:35:33Z

test-data/unit/fixtures/object_with_init_subclass.pyi

+
+class object:
+    def __init__(self) -> None: pass
+    def __init_subclass__(cls, **kwargs) -> None: pass


This doesn't match typeshed, see https://github.com/python/typeshed/blob/master/stdlib/2and3/builtins.pyi. I would prefer to keep such subtle things in fixtures closer to the truth.

ilevkivskyi

Thanks for updates! I have few more comments, mostly just minor style tweak.

ilevkivskyi · 2019-09-05T13:35:39Z

mypy/checker.py

+            7   Child()
+
+        Base.__init_subclass__(thing=5) is called at line 4. This is what we simulate here
+        Child.__init_subclass__ is never called


Thanks for adding the docstring! Normally we use this style:

"""Short one line summary. The rest of the description after empty line. """

The summary in your case may be Check that keywords in a class definition are valid arguments for __init_subclass__().

ilevkivskyi · 2019-09-05T13:36:32Z

mypy/checker.py

+            # 'object.__init_subclass__ is a dummy method with no arguments, always defined
+            # there is no use to call it
+            if base.name() != 'object' \
+                    and base.defn.info:  # there are "NOT_READY" instances


This looks dangerous. Could you please post examples of tests that failed and the traceback?

I have pasted traceback below (quite long though)

ilevkivskyi · 2019-09-05T13:37:11Z

mypy/checker.py

+        # we skip the current class itself
+        for base in typ.mro[1:]:
+            # 'object.__init_subclass__ is a dummy method with no arguments, always defined
+            # there is no use to call it


In some sense it still makes sense to check this, for example:

>>> class C(test=True): ... ... Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: __init_subclass__() takes no keyword arguments

You are right. I made the changes accordingly

ilevkivskyi · 2019-09-05T14:01:22Z

mypy/checker.py

+                        call_expr.line = defn.line
+                        call_expr.column = defn.column
+                        call_expr.end_line = defn.end_line
+                        self.expr_checker.accept(call_expr,


Using accept() may actually be dangerous. For example if the base class is not in scope (defined in another module and not imported in the current one), this may trigger Name not defined or maybe even a crash. There are two possible solutions here:

Point name_expr.node to the base class TypeInfo before calling accept()

Use check_call() or similar methods

Please add a test for this.

Hm sorry, I don't understand. I tried to produce a mypy runtime exception with 2 modules, one of which defines a Base, and the other one doesn't import that Base, and it only gave me a nice error Name 'Base' is not defined. Which os OK I guess? this is what would happen at runtime anyway (NameError: name 'Base' is not defined)

I added a test for that

Which os OK I guess?

It is not what I wanted. I wanted something like this:

# file bases.py class Base: def __init_subclass__(cls, **kwargs) -> None: ... class MidBase(Base): ... # file main.py from bases import MidBase class Main(MidBase, test=False): ...

mypy/checker.py

ilevkivskyi · 2019-09-05T14:04:53Z

test-data/unit/check-classes.test

+        cls.default_name = default_name
+        return
+
+


I think it is better to have only one empty line here and below.

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

ilevkivskyi

This is almost ready, here are some more comments.

ilevkivskyi · 2019-09-11T20:45:17Z

mypy/checker.py

+        for base in typ.mro[1:]:
+            # there are "NOT_READY" instances
+            # during the tests, so I filter them out...
+            if base.defn.info:


I think I have an idea why this may be missing, but anyway this is probably not important since you don't need this anyway, everywhere where you use base.defn.info you can use just base, this is the same object.

understood!

ilevkivskyi · 2019-09-11T20:46:29Z

mypy/checker.py

+            # there are "NOT_READY" instances
+            # during the tests, so I filter them out...
+            if base.defn.info:
+                for method_name, method_symbol_node in base.defn.info.names.items():


You don't need this to be a cycle, just use method_symbol_node = base.names.get('__init_subclass__').

Or even simpler (if you are not going to use method_symbol_node):

if '__init_subclass__' not in base.names: continue

you are right, I have modified accordingly

ilevkivskyi · 2019-09-11T20:49:50Z

mypy/checker.py

+                        call_expr.line = defn.line
+                        call_expr.column = defn.column
+                        call_expr.end_line = defn.end_line
+                        self.expr_checker.accept(call_expr,


Which os OK I guess?

It is not what I wanted. I wanted something like this:

# file bases.py class Base: def __init_subclass__(cls, **kwargs) -> None: ... class MidBase(Base): ... # file main.py from bases import MidBase class Main(MidBase, test=False): ...

gantsevdenis · 2019-09-15T21:33:16Z

@ilevkivskyi Maybe it's better if I open another PR, with a clean history? this one starts being messy.. i am sorry about that

ilevkivskyi · 2019-09-17T11:05:11Z

Maybe it's better if I open another PR, with a clean history? this one starts being messy.. i am sorry about that

We squash commits anyway, so this doesn't really matter. I will take a look at this later today.

ilevkivskyi

Thanks, looks good! I can merge this after you will consider few style comments and undo the typeshed pin move.

ilevkivskyi · 2019-09-17T15:21:50Z

mypy/checker.py

@@ -1699,6 +1700,51 @@ def visit_class_def(self, defn: ClassDef) -> None:
        if typ.is_protocol and typ.defn.type_vars:
            self.check_protocol_variance(defn)

+    def check_init_subclass(self, defn: ClassDef) -> None:
+        """
+        Check that keywords in a class definition are valid arguments for __init_subclass__().


Move this to the line above, right after quotes. Essentially we just follow https://www.python.org/dev/peps/pep-0257/

mypy/checker.py

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

gantsevdenis · 2019-09-21T12:35:03Z

thank you very much for helping

srittau · 2019-09-23T09:32:51Z

This PR seems to have broken typeshed tests: https://travis-ci.org/python/typeshed/jobs/588255463 Seems an easy enough fix, but I wonder why this passes in this repository.

gantsevdenis added 8 commits September 3, 2019 12:09

simulate a Call to Base.__init_subclass__

a709e11

added fixture

dd3df66

update existing test; class B would fail at runtime

1dfae8c

added tests

081e19c

base, not typ

145d254

flake8

bd66bca

trailing space

1d12de7

fix type

7dccdb7

gantsevdenis changed the title ~~Fix 7190~~ Fix 7190:__init_subclass__ is not type-checked Sep 3, 2019

ilevkivskyi reviewed Sep 4, 2019

View reviewed changes

gantsevdenis added 5 commits September 4, 2019 13:19

added test for metaclass

f5cc79a

make signature conform to typeshed

1c62e37

refactor into a method + check for metaclass + added comments

6729fe8

space

502aa3f

missing return + annotation

dfb6551

ilevkivskyi reviewed Sep 5, 2019

View reviewed changes

gantsevdenis and others added 9 commits September 5, 2019 10:24

Update mypy/checker.py

b860c3a

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

Update mypy/checker.py

012b817

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

Update mypy/checker.py

a8fcb3e

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

Update mypy/checker.py

ef295b5

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

Update mypy/checker.py

ddc1dce

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

added test with imports

2c0c20f

check for "object" too

67a761b

stylistic changes

79208fd

added test

8eb7b8d

ilevkivskyi reviewed Sep 11, 2019

View reviewed changes

gantsevdenis added 2 commits September 15, 2019 22:03

simplify logic

9a7ab31

modify test

2281a86

gantsevdenis force-pushed the fix_7190 branch 2 times, most recently from a6c5089 to 2281a86 Compare September 15, 2019 20:34

gantsevdenis force-pushed the fix_7190 branch from 2fc04f3 to 2281a86 Compare September 15, 2019 21:02

gantsevdenis added 3 commits September 15, 2019 23:08

sync typeshed to master

f8f330a

Merge from upstream/master

0a16c0c

flake8

0850dff

ilevkivskyi approved these changes Sep 17, 2019

View reviewed changes

gantsevdenis and others added 4 commits September 21, 2019 14:00

Update mypy/checker.py

798ff33

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

Update mypy/checker.py

6f5bda9

Co-Authored-By: Ivan Levkivskyi <levkivskyi@gmail.com>

move comment line

ab8a6c8

undo pin move

7da5d11

ilevkivskyi merged commit 1f11538 into python:master Sep 21, 2019

Uh oh!

Fix 7190:__init_subclass__ is not type-checked #7452

Fix 7190:__init_subclass__ is not type-checked #7452

Uh oh!

Conversation

gantsevdenis commented Sep 3, 2019 • edited by ilevkivskyi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi Sep 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gantsevdenis Sep 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gantsevdenis commented Sep 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi commented Sep 17, 2019

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

gantsevdenis commented Sep 21, 2019

Uh oh!

gantsevdenis commented Sep 3, 2019 •

edited by ilevkivskyi

Loading

ilevkivskyi Sep 5, 2019 •

edited

Loading

gantsevdenis Sep 6, 2019 •

edited

Loading

gantsevdenis commented Sep 15, 2019 •

edited

Loading