Using reveal_type with no errors should return exit code 0

[achimnol]

Bug Report

When using pytest-mypy to test some dynamically determined types (with generics) results in a false-positive error in CI, because mypy's exit code becomes 1 when there are no errors but only reveal_type outputs.

To Reproduce

Just write an arbitrary Python file that has no type errors and put reveal_type() to any one of the identifiers.
Run mypy against that file and check the exit code.

I'd like to keep reveal_type() in my code, to test a mixture of runtime type validation and static typing results using typeguard, generics, and pytest-mypy plugin.

Expected Behavior

When there are no errors but only some reveal_type outputs, mypy executable should set the exit code 0.

Actual Behavior

It sets the exit code 1, breaking my CI.

Your Environment

• Mypy version used: 0.800
• Mypy command-line flags and options:

[mypy]
ignore_missing_imports = true
mypy_path = stubs,src
namespace_packages = true
explicit_package_bases = true

• Python version used: 3.9.1
• Operating system and version: Ubuntu 20.04

[emmatyping]

I do think since IIRC we moved reveal_type from an error to a note it would make sense to exit 0 if the only output is reveal_type. More generally exiting 0 if there are only notes makes sense to me.

[KotlinIsland]

The downside to this is that mypy will now report that invalid code is correct due to the fact that reveal_type and friends don't actually exist.

👉 echo "a = 1"\n"reveal_type(a)" > test.py
👉 mypy test.py
test.py:2: note: Revealed type is "int"
Success: no issues found in 1 source file
👉 python test.py
Traceback (most recent call last):
  File "/test.py", line 2, in <module>
    reveal_type(a)
    ^^^^^^^^^^^
NameError: name 'reveal_type' is not defined
[1]👉 

Where previously mypy would warn that there is something invalid:

👉 mypy test.py
test.py:2: note: Revealed type is "int"
Success: no issues found in 1 source file
[1]👉 echo $status
1

But this isn't the biggest issue in the world, as any linter(flake8/pylint/qodana) should pick this up.

[DetachHead]

reveal_type usages should always exit with 1 imo, because they will fail at runtime. if you want to test types you should use assert_type instead