@hauntsaninja Thanks for the comment.

I don't have a great idea of all the cases where you want to use different settings (not just a subset, but actually different). If there are a bunch, then I agree we should somehow make it clear that this needs a different config. I wonder if there are other ways we could make it more clear? Maybe we rename the arg to plugin-config-file, or have a big print statement ™ that tells the user that we are only processing plugins and mypy_path (if you pass in a config file).

In my use case, it's nice to have the config file because it automatically picks up on all of the following settings that we want to be the same:

[mypy]
mypy_path = path
plugins = plugin1, plugin2, plugin3

[mypy.plugins.django-stubs]
setting1 = "value1"
setting2 = "value2"

@hauntsaninja Could you elaborate on how this test would work? I spent some time today trying to get one running with named_callable.py which uses get_function_hook. I defined a test case that looked like:

def test_config_file(self) -> None:
    runtime = (
        "from typing import Callable\n"
        "def decorator1() -> Callable[..., Callable[..., int]]: pass\n"
        "def decorator2() -> Callable[..., Callable[..., int]]: pass\n"
        "@decorator1()\n"
        "def f() -> None: pass\n"
        "@decorator2()\n"
        "def g() -> None: pass\n"
    )
    stub = (
        "import builtins\n"
        "def f(*args: Any, **kwargs: Any) -> builtins.int: ...\n"
        "def g(*args: Any, **kwargs: Any) -> builtins.int: ...\n"
    )
    config_file = (
        "[mypy]\n"
        "plugins={}/test-data/unit/plugins/named_callable.py\n".format(root_dir)
    )
    output = run_stubtest(
        stub=stub,
        runtime=runtime,
        options=[],
        config_file=config_file,
    )

If you take a look at named_callable.py, it looks for a fullname of m.decorator1, and converts that function's return type to a function with a return type of str. Using the debugger, I found out that get_function_hook is not being called for the decorator1 or 2 functions at all - it's only being called with builtins.object and typing.runtime_checkable.

I tried similar approaches with testTypeAnalyzeHookPlugin (trying to modify the test case to fit the stubtest input format) to use the get_type_analyze_hook (as you mentioned), but still couldn't get it to work.

Is there an example test case you could point me to? Thanks for all your help!

stubtest uses mypy to build the stubs. The runtime is only imported and introspected using inspect and friends; stubtest doesn't use the runtime source code at all (and for C extensions, it isn't even available to us). You'll need to use the decorators you define in the stub.

Thanks, looks good!