Enable pretty by default #19510

null-dreams · 2025-07-25T21:02:48Z

This change enables mypy's "pretty" error reporting by default to provide a better out-of-the-box experience for users.

Fixes #19108.

Core Changes

The internal default for the pretty option is now True.
The command-line flags have been reconfigured so that --no-pretty is the primary, user-facing flag to disable pretty-printing.
The old --pretty flag is now a no-op (for backward compatibility) and has been hidden from the --help message.

Test Suite Modifications

Enabling pretty output by default caused widespread test failures, as most tests were written to expect the old plain output. Instead of modifying thousands of test case files, this PR introduces targeted fixes into the central test runners to default them to --no-pretty mode.

This required modifying several distinct test runners:

TypeCheckSuite: The main test runner's option parsing was updated in mypy/test/helpers.py.
CmdlineSuite: The raw command-line argument parser in mypy/test/testcmdline.py was updated.
PEP561Suite & others: Other smaller test runners (testpythoneval, testerrorstream) were also adapted.
DaemonSuite: This was the most complex fix due to the daemon's stateful, client-server architecture. The solution required a combination of central logic and manual test case edits:
- Central Logic: The run_cmd helper in mypy/test/testdaemon.py was updated to inject --no-pretty only into state-setting subcommands (run, check). This handles the majority of cases.
- Manual Overrides: The test cases in daemon.test were manually updated based on how they interact with the daemon's lifecycle:
  - For tests beginning with $ dmypy start, the --no-pretty flag was added to the server's default flags (e.g., start -- --no-pretty) to set the server's state for the entire run.
  - For tests that rely on an "implicit start" (i.e., the first command is $ dmypy run), the flag was passed via the -- separator (e.g., run -- --no-pretty test.py) to configure the new server instance.
  - For the few tests that expect pretty output, a `# NO-MODIFY "magic comment" was used to have the central logic skip them entirely.
Stubtest Unit Tests: The handful of stubtest tests were updated manually to assert against the new pretty-printed output, as their runner does not accept formatting flags.

This overall approach keeps the git diff minimal and isolates the test-related changes to the test infrastructure itself.

Documentation

The command-line and config file documentation has been updated to reflect that pretty-printing is the new default and to document the --no-pretty flag.

sterliakov · 2025-07-25T21:25:35Z

Supersedes #19122 and #19380. If any of those is merged, please ping me to contribute an envvar:) Anyway, seems like this feature is really wanted...

null-dreams · 2025-07-25T21:26:28Z

I'll make sure to ping.

null-dreams · 2025-07-25T22:49:32Z

Hi, looks like the Check documentation build / docs (pull_request) is failing consistently.
The error log points to a problem with intersphinx being unable to fetch the Cython documentation index:

/home/runner/work/mypy/mypy/docs/source/faq.rst:159: WARNING: unknown document: 'cython:index' [ref.doc]

This seems to be an issue with the CI environment and is unrelated to my code changes. I've re-triggered the CI once by pushing an empty commit, but the failure is persistent.

Could a maintainer with permissions please take a look or re-run the failed job? It seems to be the only thing blocking the CI from going green. Thanks!

sterliakov · 2025-07-26T00:27:08Z

Yep, this has nothing to do with your PR, docs.cython.org is down (domain not renewed), we should update the URL.

Fixes docs build failure discovered in #19510. Updated setuptools URL too because previous CI runs say that > intersphinx inventory has moved: https://setuptools.readthedocs.io/en/latest/objects.inv -> https://setuptools.pypa.io/en/latest/objects.inv

sterliakov · 2025-07-26T01:52:17Z

JFYI the docs issue is now fixed on master

Change the default mypy output to be pretty-printed, providing users with more readable error messages that include code context out-of-the-box. This commit implements the core logic for this feature: - In 'mypy/options.py', the default value for 'pretty' is set to 'True'. - In 'mypy/main.py', the command-line flags are reconfigured: - '--no-pretty' is now the primary, user-facing flag to disable the pretty printing. - The old '--pretty' flag is deprecated and hidden from help text to reflect that the pretty output is now default. This change intentionally breaks the test suite, which will be fixediIn subsequent commits by adapting the test runners. Part of python#19108

This commit adapts the main test runner (`TypeCheckSuite`) to handle the new pretty-by-default behaviour. - For tests with a `# flags: ` line `--no-pretty` is now injected unless `--pretty` is explicitly requested. - For tests without any flags, `options.pretty` is directly set to `False` to ensure plain output. This strategy avoids modifying thousands of individual test-data files.

This commit adapts the `CmdlineSuite` runner to handle the new pretty-by-default behavior. The `parse_args` helper in `mypy/test/testcmdline.py` is modified to inject the `--no-pretty` flag into the arguments parsed from '# cmd:' lines in test cases. This ensures that command-line-driven tests, which run in a separate subprocess and bypass other test helpers, are also executed in non-pretty mode by default. Part of python#19108.

This commit adapts the `pythoneval` test runner to handle the new pretty-by-default behavior. The `test_python_evaluation` function in `mypy/test/testpythoneval.py` now injects the `--no-pretty` flag by default into the mypy command line it constructs. Logic is also included to respect a `# flags: --pretty` directive, ensuring that tests designed to check pretty-printing still work correctly. Part of python#19108.

This commit modifies the `test_error_stream` runner to set `options.pretty = False`, ensuring its tests continue to pass with the new pretty-by-default behavior. Part of python#19108.

This commit adapts the `DaemonSuite` test runner and its corresponding data files to accommodate the new pretty-by-default behavior. This fix is nuanced due to the stateful, client-server nature of the daemon and required a two-part approach: 1. **Central Logic:** The `run_cmd` helper in `mypy/test/testdaemon.py` was modified to inject `--no-pretty` only into state-setting subcommands (`run`, `check`). This handles the majority of daemon tests. 2. **Manual Overrides:** A number of test cases in `daemon.test` were manually updated. This was necessary for tests that start the daemon with `dmypy start` or have other unique requirements, ensuring the server is correctly configured in non-pretty mode from the outset. This combined strategy ensures the entire daemon test suite passes. Part of python#19108.

This commit adapts the `PEP561Suite` test runner to handle the new pretty-by-default behavior. The `parse_mypy_args` helper in `mypy/test/testpep561.py` is modified to inject the `--no-pretty` flag into the arguments parsed from '# flags:' lines in test cases. Part of python#19108.

This commit adapts several unit tests in the `StubtestMiscUnit` suite to handle the new pretty-by-default behavior. Because the `stubtest` runner is a distinct command-line tool, injecting the `--no-pretty` flag was not feasible as it's an unrecognized argument. Instead, the `assert` statements in the failing tests have been updated to match the new, pretty-printed error output from mypy. Part of python#19108.

This commit updates the user-facing documentation to reflect that pretty-printing is now the default output format. - The `--pretty` flag documentation in `command_line.rst` has been replaced with documentation for the new `--no-pretty` flag. - The `config_file.rst` documentation for the `pretty` option has been updated to state that its default is now `True`. Fixes python#19108.

for more information, see https://pre-commit.ci

null-dreams · 2025-07-30T10:15:22Z

Hello, just wanted to gently bump this for review when you have a moment.

To summarize, this PR enables --pretty by default and adapts the various test runners to handle the new behavior. All checks are passing.

Let me know if there's any feedback or anything else I can do to help get this merged. No rush, and thanks for your time!

wyattscarpenter · 2025-07-31T11:02:48Z

Anyway, seems like this feature is really wanted...

You know, it's funny... even though I have no interest in this feature myself (besides the cursory interest of thinking it's probably a good idea for mypy), I've considered several times trying to solve it myself, just because it seems so simple and doable. (The main thing that stopped me was the fact that the mypy test suite already doesn't run green on my machine, so it would be hard for me to use --update-data without breaking something else, presumably.) I suspect most people who have tried to solve the problem thus far, no offense to them, have mostly been thinking along the same lines, especially because the issue is labeled good-first-issue, and the other attempted-solvers never got around to fixing the test problems, suggesting they were at the very least not desperate to get this functionality into a release of mypy...

Anyway, congrats to @null-dreams for going the distance on this one, and also congrats to him on his first ever PR according to github!

mypy/test/helpers.py

wyattscarpenter · 2025-07-31T11:27:14Z

I'm just some random guy with no authority on this project, but I do have some thoughts:

It's probably better to implement the "no-op" pretty flag using the add_invertible_flag function we already have, as this would be fewer lines of code. You can also keep most of its current documentation, maybe.
I think it's probably better to update the tests than to special-case this behavior, although maybe the maintainers will strongly prefer the smaller diffs instead of the simpler logic. (I would argue that a test should have as little special massaging as possible, in order to fulfill best the purpose of testing.)
In any case, this "# NO-MODIFY" business seems the worst of both worlds, as there's a big diff and also the default behavior is massaged. Maybe it's fine, whatever you think of the other test cases, for the daemon test cases to be changed to be pretty by default, because of the lengths one must go to to avoid this...
A comment I just made in the github code review: Do you really have to set flag_list = ["--no-pretty"] in addition to options.pretty = False in mypy/test/helpers.py? Weird.
There is an --update-data flag to pytest you can use, if you do decide just to update the tests. Probably you should use k with it, or something, if you decide to use it.
There must be a test, somewhere in the system, for --pretty itself. This test should be, uh, updated, I suppose.

In the world of my imagination, there is a triptych of tests in this project: one for explicit pretty, one for explicit no-pretty, and one for default prettiness; and only the last of these would have to be updated. And since the tests are massaged to continue using no-pretty, I guess nothing has to be updated.

Again, this is just my professional opinion as an interested 3rd party, and has no bearing on the mypy maintainers' opinion of this PR nor the speed at which they choose to adopt or comment upon it.

…tible_flag`

github-actions · 2025-07-31T14:49:18Z

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

null-dreams · 2025-07-31T17:32:24Z

Sorry for the late comment.
Thank you @wyattscarpenter for all the suggestions.

Originally when I attempted that fix, I was not very well versed with the functioning of add_invertible_flag. After experimenting with some values and failing, I decided to switch to add_argument. Thanks to your suggestion, I went back with a better understanding of the functions, I was able to implement it using add_invertible_flag. I have refactored main.py to use add_invertible_flag instead of add_argument.
I was initially ready to update all the test cases, but seeing that would be huge diff and also somehow --update-data flag was failing for some reason which I couldn't understand, I decided to make changes in the test runners itself. I'm open to updating the test cases if the maintainers prefer it, but aligned to small diff for review purpose.
DaemonSuite was the most challenging part of my 'fixes' which I tried.
- After making a central change in the test runner, there were some test cases which were still failing. On inspection, they were those cases in which every command didn't accept flags like those with dmypy recheck or those which expected pretty output. So in case of dmypy start, I had to manually insert the flag at the start of the server then had to make sure the other commands in that case wasn't modified. Best solution which I could come up with (being very new to OSS overall) was to somehow cosmetically mark those commands which I can avoid in the test runners.
- It is definitely the most noisiest part of the PR I understand, and I'm open to any better alternatives or suggestions for cleaner way of handling the state management,
The way I insert the --no-pretty flag is that first I check if the test case uses pretty flag which would be necessary based on the previous inclination of using that flag to get a pretty output. In case of the presence of the pretty flag, I will not insert the no-op in that test-case specifically.

sterliakov · 2025-07-31T17:42:47Z

mypy/test/testdaemon.py

@@ -79,6 +79,18 @@ def parse_script(input: list[str]) -> list[list[str]]:
 def run_cmd(input: str) -> tuple[int, str]:
    if input[1:].startswith("mypy run --") and "--show-error-codes" not in input:
        input += " --hide-error-codes"
+    is_pretty_test = "# NO-MODIFY" in input


Okay, I must be missing something - why won't just checking for --pretty flag do? Yes, it will deviate from normal behavior, but be consistent with all other tests - essentially sticking to old default unless requested otherwise.

Hello @sterliakov , this is still my main approach and it is also partly working in the this test-runner to maintain the consistent behavior with all other test-runners.However, I discovered a critical technical constraint: the dmypy client itself does not accept the --pretty or --no-pretty flags.

The formatting flags must be directly passed to the server, which only worked in these two ways for me:
- During startup: $ dmypy start -- --no-pretty
- During one-shot run: $ dmypy run -- --no-pretty

Because the testcases contains a mix of commands where the flag will and won't work, I had to make #NO-MODIFY as the magic comment to recognize where to not modify the command.

For demonstration:

[case testDaemonRecheck] $ dmypy start -- --follow-imports=error --no-error-summary Daemon started $ dmypy check foo.py bar.py $ dmypy recheck $ dmypy recheck --update foo.py --remove bar.py sir_not_appearing_in_this_film.py foo.py:1: error: Import of "bar" ignored [misc] foo.py:1: note: (Using --follow-imports=error, module not passed on command line) == Return code: 1 $ dmypy recheck --update bar.py $ dmypy recheck --update sir_not_appearing_in_this_film.py $ dmypy recheck --update --remove $ dmypy stop Daemon stopped [file foo.py]

In this case, my first approach was to insert the --pretty flag in all of them. But dmypy recheck threw an unrecognized argument error. So I realized I cannot insert any flag in the commands following dmypy start...
Therefore, I preferred to adding --pretty flag during startup. But then the test-runner logic detecting so such --pretty flag in the other commands automatically injects the --no-pretty flag which also throws an unrecognized argument error. This led to the realization, that I also need to mark these commands someway, that the test-runner doesn't inject --no-pretty in them. The best I could think of was a magic comment #NO-MODIFY which I can use to detect these lines where the test-runner shouldn't be interfering.

I understand that this is a very bare-bones workaround, and I am open to any suggestions to make it cleaner.

sterliakov · 2025-07-31T17:43:02Z

No matter what, we should definitely keep most of the tests non-pretty. Most of them use inline error comments (they are much easier to verify manually, compared to separate snapshots in [out] sections), and such inline diffs will become completely unreadable with --pretty. Yes, it is a bit less powerful (as we don't actually test parts printed by --pretty then), but we need to keep reliability and actual test maintainability in balance.

I agree, though, that probably more tests should be using --pretty since it's going to be on by default. I think we can defer that to a separate PR, current approach (disable in tests unless requested) is the most sensible one.

And yes, --update-data does not work on some test files (I noticed that a while ago, but did not care enough to look deeper), so some manual work might be necessary.

wyattscarpenter · 2025-07-31T18:44:43Z

Huh, it hadn't even occurred to me — there's no syntax for multi-line inline comments, is there? Nor for ignoring subsequent lines of a pretty comment in a test...

This comment has been minimized.

Sign in to view

null-dreams force-pushed the enable-pretty-by-default branch from 964f17f to 8214b19 Compare July 25, 2025 21:42

This comment has been minimized.

Sign in to view

null-dreams force-pushed the enable-pretty-by-default branch from e60744f to 8afae7a Compare July 25, 2025 22:35

This comment has been minimized.

Sign in to view

sterliakov mentioned this pull request Jul 26, 2025

docs: update cython and setuptools RTD URLs #19512

Merged

null-dreams added 11 commits July 26, 2025 10:04

fix(tests): Adapt errorstream suite to default to non-pretty output

9d498a8

This commit modifies the `test_error_stream` runner to set `options.pretty = False`, ensuring its tests continue to pass with the new pretty-by-default behavior. Part of python#19108.

style: Fix ruff linting error in testdaemon.py

7b79b5a

chore: Re-trigger CI

d67c367

null-dreams force-pushed the enable-pretty-by-default branch from 73c1b5b to d67c367 Compare July 26, 2025 06:04

[pre-commit.ci] auto fixes from pre-commit.com hooks

afc6585

for more information, see https://pre-commit.ci

This comment has been minimized.

Sign in to view

null-dreams marked this pull request as ready for review July 26, 2025 06:40

wyattscarpenter reviewed Jul 31, 2025

View reviewed changes

mypy/test/helpers.py Outdated Show resolved Hide resolved

null-dreams added 2 commits July 31, 2025 18:25

fix: Remove redundant flag_list in parse_options

23e5980

refactor(cli): Simplify flag implementation using inverted `add_inver…

66121c9

…tible_flag`

sterliakov reviewed Jul 31, 2025

View reviewed changes

Uh oh!

Enable pretty by default #19510

Are you sure you want to change the base?

Enable pretty by default #19510

Uh oh!

Conversation

null-dreams commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Core Changes

Test Suite Modifications

Documentation

Uh oh!

sterliakov commented Jul 25, 2025

Uh oh!

null-dreams commented Jul 25, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

null-dreams commented Jul 25, 2025

Uh oh!

This comment has been minimized.

sterliakov commented Jul 26, 2025

Uh oh!

sterliakov commented Jul 26, 2025

Uh oh!

This comment has been minimized.

null-dreams commented Jul 30, 2025

Uh oh!

wyattscarpenter commented Jul 31, 2025

Uh oh!

Uh oh!

wyattscarpenter commented Jul 31, 2025

Uh oh!

github-actions bot commented Jul 31, 2025

Uh oh!

null-dreams commented Jul 31, 2025

Uh oh!

sterliakov Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

null-dreams Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

sterliakov commented Jul 31, 2025

Uh oh!

wyattscarpenter commented Jul 31, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

null-dreams commented Jul 25, 2025 •

edited

Loading