Calling issubclass in __init_subclass__ with ABCMeta uses wrong _abc_cache. #116093
Description
Bug report
Bug description:
In the following minimal example, we get unexpected output:
from abc import ABCMeta
class BaseA(metaclass=ABCMeta):
...
class BaseB(metaclass=ABCMeta):
def __init_subclass__(cls):
issubclass(BaseA, BaseA)
issubclass(BaseA, BaseB)
class DerivedClass(BaseA, BaseB):
...
print(f"Is BaseA a subclass of BaseB? (expect False): {issubclass(BaseA, BaseB)}")Output:
Is BaseA a subclass of BaseB? (expect False): True
We can drill in a bit deeper with the following expanded example:
from abc import ABCMeta
class BaseA(metaclass=ABCMeta):
...
class BaseB(metaclass=ABCMeta):
def __init_subclass__(cls):
_DerivedClass = BaseB.__subclasses__()[0]
print("issubclass(BaseA, BaseA):", issubclass(BaseA, BaseA), "<-- Obviously true")
print("issubclass(BaseA, _DerivedClass):", issubclass(BaseA, _DerivedClass), "<-- This is unexpected!!!")
print("issubclass(_DerivedClass, BaseB)", issubclass(_DerivedClass, BaseB), "<-- This is expected")
# c.f. https://github.com/python/cpython/blob/3.11/Lib/_py_abc.py#L140
print("issubclass(BaseA, BaseB)", issubclass(BaseA, BaseB), "<-- This is true, because we end up checking issubclass(BaseA, _DerivedClass) within this check.")
print("And now the result issubclass(BaseA, BaseB) = True is cached, so...")
class DerivedClass(BaseA, BaseB):
...
print(f"Is BaseA a subclass of BaseB? (expect False): {issubclass(BaseA, BaseB)}")Output:
issubclass(BaseA, BaseA): True <-- Obviously true
issubclass(BaseA, _DerivedClass): True <-- This is unexpected!!!
issubclass(_DerivedClass, BaseB) True <-- This is expected
issubclass(BaseA, BaseB) True <-- This is true, because we end up checking issubclass(BaseA, _DerivedClass) within this check.
And now the result issubclass(BaseA, BaseB) = True is cached, so...
Is BaseA a subclass of BaseB? (expect False): True
Because issubclass(BaseA, _DerivedClass) is True, issubclass(BaseA, BaseB) is also True (c.f. checking subclasses at https://github.com/python/cpython/blob/3.11/Lib/_py_abc.py#L140).
To understand why issubclass(BaseA, _DerivedClass) is True, we can consider the following version of the script:
from abc import ABCMeta
class BaseA(metaclass=ABCMeta):
...
class BaseB(metaclass=ABCMeta):
def __init_subclass__(cls):
_DerivedClass = BaseB.__subclasses__()[0]
print("BaseA: ", BaseA._abc_impl)
print("DerivedClass:", _DerivedClass._abc_impl, "<-- Same as BaseA!")
print("BaseB: ", BaseB._abc_impl)
print()
issubclass(BaseA, BaseA)
ABCMeta._dump_registry(BaseA)
print()
ABCMeta._dump_registry(_DerivedClass)
issubclass(BaseA, _DerivedClass)
class DerivedClass(BaseA, BaseB):
...Output:
BaseA: <_abc._abc_data object at 0x7fa12e543f00>
DerivedClass: <_abc._abc_data object at 0x7fa12e543f00> <-- Same as BaseA!
BaseB: <_abc._abc_data object at 0x7fa12e541e80>
Class: __main__.BaseA
Inv. counter: 24
_abc_registry: set()
_abc_cache: {<weakref at 0x7fa12e510ef0; to 'ABCMeta' at 0x1c4d790 (BaseA)>}
_abc_negative_cache: set()
_abc_negative_cache_version: 24
Class: __main__.DerivedClass
Inv. counter: 24
_abc_registry: set()
_abc_cache: {<weakref at 0x7fa12e510ef0; to 'ABCMeta' at 0x1c4d790 (BaseA)>}
_abc_negative_cache: set()
_abc_negative_cache_version: 24
We can see that BaseA and DerivedClass are sharing the same ._abc_impl object! Because we have already called issubclass(BaseA, BaseA), this cache is populated, saying "BaseA is a subclass of BaseA". But because DerivedClass is sharing that same cache, the cache also implies "BaseA is a subclass of DerivedClass".
The reason BaseA and DerivedClass are sharing the same ._abc_impl has to do with the fact that we're inside the __init_subclass__ method. This method is called as part of type.__new__. Looking at ABCMeta.__new__(), we can see that type.__new__ is called before the code that sets up the various caches/registries (c.f. https://github.com/python/cpython/blob/3.11/Lib/_py_abc.py#L35).
As such, when looking for DerivedClass._abc_impl, we fall back to the parent class, BaseA._abc_impl.
Summary
The bug here is that ABCMeta.__new__ calls type.__new__ (which in turn calls the user supplied .__init_subclass__ before it's had a chance to set up the various pieces of ._abc_impl. This can cause issubclass to provide incorrect responses when called inside __init_subclass__, and for these incorrect response to be cached, and therefore propagated to calls outside of __init_subclass__.
CPython versions tested on:
3.11
Operating systems tested on:
macOS