Type errors caused by dependency changes are missed in incremental mode if the dependency is checked first

[sjelin]

Bug Report

If a file's dependency changes, and then mypy is run on just that dependency, then running mypy on the dependent file does not catch errors caused by the change to the dependency.

To Reproduce

$ mkdir /tmp/mypy_deps_bug
$ cd /tmp/mypy_deps_bug
$ echo 'foo = 5' > foo.py
$ printf 'from foo import foo\nbar: int = foo\n' > bar.py
$ mypy *
Success: no issues found in 2 source files
$ echo 'foo = None' > foo.py
$ mypy foo.py
Success: no issues found in 1 source file
$ mypy *
Success: no issues found in 2 source files
$ rm -rf .mypy_cache
$ mypy *
bar.py:2: error: Incompatible types in assignment (expression has type "None", variable has type "int")
Found 1 error in 1 file (checked 2 source files)

Your Environment

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.3 LTS
Release:	18.04
Codename:	bionic
$  mypy --version
mypy 0.782
$ python3 --version
Python 3.6.9

I don't think I have any relevant mypy.ini files or environmental configuration. I was also able to reproduce this on my MacBook, which I don't think even had mypy installed beforehand

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H2
$ mypy --version
mypy 0.782
$ python3 --version
Python 3.7.3

Additional Information

When mypy was run through my company's internal linter, this was slightly harder to reproduce. Basically, instead of mypy foo.py, the command to "trick" the cache needed to be ./internal_linter.py baz.py, where baz.py is some third file which also imports foo.py. I have no theory as to why this is the case, and stopped investigating once I reproduced the bug directly through mypy.

In some unusual configurations, the location where mypy puts files within the cache dir can be non-deterministic, causing this bug to be slightly inconsistent to produce. The repro in the "To Reproduce" section has been 100% reliable, however.

[msullivan]

Yeah this is pretty nasty if you are running mypy on file subsets. This was never really a focus for us, because it never seemed like what we wanted: if a change in foo breaks bar, we want to know as part of the linting/checking of foo, which means also running on bar right then. Obvious workaround is to check all of the relevant files in each invocation.

That said, it's still a bug.

The issue is that each module tracks its dependencies, and gets rechecked if any of its dependencies were stale. So when foo gets rechecked by itself, that updates its cache so it isn't stale, and bar doesn't get reprocessed. I think that the solution would be to add a dep_hashes field to CacheMeta (https://github.com/python/mypy/blob/master/mypy/build.py#L294) that tracks the interface_hash of each dependency. And then we would take that into account when we determine what is stale in process_graph. (https://github.com/python/mypy/blob/master/mypy/build.py#L2829)