[1.19 regression] output format overruled by .mypy_cache

[sjoerdjob-legalsense]

Bug Report

Starting with mypy 1.19.0, the --output argument seems to be ignored when a .mypy_cache directory is present.

To Reproduce

Minimum working reproduction:

Have an incorrectly typed Python file.

# foo.py
wrong: str = 5

Install the latest mypy version.

$ pip freeze
librt==0.6.3
mypy==1.19.0
mypy_extensions==1.1.0
pathspec==0.12.1
typing_extensions==4.15.0

Run mypy, with different formats:

$ rm -r .mypy_cache
$ mypy foo.py
$ mypy --output json foo.py
$ rm -r .mypy_cache
$ mypy --output json foo.py
$ mypy foo.py

Expected Behavior

Calls with --output json produce json output, calls without --output json do not (assuming nothing specified in config files).

Actual Behavior

The output format is taken from the first call

$ rm -r .mypy_cache
$ mypy foo.py
foo.py:1: error: Incompatible types in assignment (expression has type "int", variable has type "str")  [assignment]
Found 1 error in 1 file (checked 1 source file)
$ mypy --output json foo.py
foo.py:1: error: Incompatible types in assignment (expression has type "int", variable has type "str")  [assignment]
Found 1 error in 1 file (checked 1 source file)
$ rm -r .mypy_cache
$ mypy --output json foo.py
{"file": "foo.py", "line": 1, "column": 13, "message": "Incompatible types in assignment (expression has type \"int\", variable has type \"str\")", "hint": null, "code": "assignment", "severity": "error"}

Your Environment

• Mypy version used: 1.19.0
• Mypy command-line flags: --output
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.14.0

Extra remarks

In a larger project, I have seen even --exclude being ignored on successive calls, but could not reproduce this yet in a minimal examples.

[ilevkivskyi]

Previously files with errors were not written to cache, now they are. I simple fix would be to add this flag to OPTIONS_AFFECTING_CACHE, but then we would re-check everything. I think we should invalidate the cache conditionally: i.e. this flag changed, and there are some errors in the file.

[gooverdian]

We are experiencing the same issue here and I wanted to comment a bit on your porposal.

I think we should invalidate the cache conditionally: i.e. this flag changed, and there are some errors in the file.

Thing is if I saw it correctly - cache contents are stored in json internally, so the best way to me is some kind of partial invalidation, where you don't recheck the file itself, but use it's internal state to generate a report according to new --output. Dropping the cache for a file with errors will lead to full recheck of this file and it's imports to get the same errors.

Tested this with --no-incremental – it was as slow as just dropping .mypy_cache dir

[sjoerdjob-legalsense]

Previously files with errors were not written to cache, now they are. I simple fix would be to add this flag to OPTIONS_AFFECTING_CACHE, but then we would re-check everything. I think we should invalidate the cache conditionally: i.e. this flag changed, and there are some errors in the file.

To me, it seems weird that the 'output format' would even be part of the cached data. As in, the output would be a 'representation' of the errors, and the errors itself could be cached, but it seems weird to cache the output.

[ilevkivskyi]

As in, the output would be a 'representation' of the errors, and the errors itself could be cached, but it seems weird to cache the output.

Well, it is what we want to have, but this is much less simple change. The question is really how pressing is this issue? If you can wait for a proper solution, it will almost certainly be in 1.20 (i.e. late January).

[sjoerdjob-legalsense]

I'm not sure how pressing it would be for others.

A work-around is running mypy with MYPY_CACHE_DIR=/dev/null for cases where you're running mypy with different arguments in sequence against the same codebase. Running with MYPY_CACHE_DIR on very large codebases, or codebases with many dependencies could slow things down a lot; but I'm unsure as what urgency to qualify this as. "Has known workaround" seems to make this 'not as bad'?

I now also have a reproduction case for mypy not excluding excluded files:

# foo.py
import bar
hello: int = "world"

# bar.py
def pizza(val):
    return 5 + val

Then, run

(mypy-1.18.2) $ rm -r .mypy_cache/
(mypy-1.18.2) $ mypy --strict .
bar.py:1: error: Function is missing a type annotation  [no-untyped-def]
foo.py:2: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 2 errors in 2 files (checked 2 source files)
(mypy-1.18.2) $ mypy --strict --exclude "bar.py" .
foo.py:2: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 1 error in 1 file (checked 1 source file)

vs

(mypy-1.19.0) $ rm -r .mypy_cache/
(mypy-1.19.0) $ mypy --strict .
bar.py:1: error: Function is missing a type annotation  [no-untyped-def]
foo.py:2: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 2 errors in 2 files (checked 2 source files)
(mypy-1.19.0) $ mypy --strict --exclude "bar.py" .
bar.py:1: error: Function is missing a type annotation  [no-untyped-def]
foo.py:2: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 2 errors in 2 files (checked 1 source file)

You are correct in that 8bfecd4 the behavior changes.

For the output format differences: I think it can definitely wait.
For the --exclude not interacting well with the cached errors: I find that very weird, and probably the --exclude should also be lifted to a higher layer?

[gooverdian]

A work-around is running mypy with MYPY_CACHE_DIR=/dev/null for cases where you're running mypy with different arguments in sequence against the same codebase. Running with MYPY_CACHE_DIR on very large codebases, or codebases with many dependencies could slow things down a lot; but I'm unsure as what urgency to qualify this as. "Has known workaround" seems to make this 'not as bad'?

You can also just add --no-incremental to the params.

As for the pressing matter – we will just rollback to 1.18 for now. Since we are using a custom runner for mypy which expects json, and after it's done – we just can't run mypy itself, since it's output is json regardless of params. That way it's annoying but workable, other way around our runner just dies, since it tries interpret text as json.

We got a pretty big project, so dropping cache on every run is pretty inefficient.

I also can see a way for IDE plugins to suffer from this issue if your local run cache somehow contaminates IDE's cache, since I think those use json too

[ilevkivskyi]

About --exclude however, the old behavior looks like a bug, and the new behavior looks correct. According to the docs https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-exclude

In particular, --exclude does not affect mypy’s discovery of files via import following. You can use a per-module ignore_errors config option to silence errors from a given module, or a per-module follow_imports config option to additionally avoid mypy from following imports and checking code you do not wish to be checked.

So since bar is still brought into the build by import following, it should be checked.