We’ve just uploaded mypy 0.760 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
Type Signature Suggestions for Tools
Normally mypy only infers the types of variables, and you have to explicitly annotate functions. Mypy daemon can now generate a suggestion for a function or method signature using static analysis through dmypy suggest. This is a low-level feature that can be used by other tools and editor integrations to make it easier to annotate existing code. See the documentation for more information.
Type Inference Improvements
Mypy type inference got a few minor improvements. Mypy can now infer a variable with a collection type from two assignments:
items = [] # No type annotation needed
if check():
items = [2, 3]
Previously mypy required a type annotation for items. Mypy can also more often infer an OrderedDict type:
from collections import OrderedDict
data = OrderedDict() # No type annotation needed
data['key'] = 4
Mypy can infer the type of an attribute in more cases when it’s first initialized through self.x:
class Counts:
def __init__(self, n: int) -> None:
self.counts = {} # Type inferred from initialization
for i in range(n):
self.counts[i] = 0
Fixes to Regressions
Fix regression introduced in 0.750: union (non-)simplification should not affect type inference (PR 8095)
Breaking Changes
Make the error code of a binary operation involving overload operator, for consistency (PR 8124)
Other Notable Improvements and Bug Fixes
Generate error when assigning an Enum or TypedDict type to an attribute (Xuanda Yang, PR 8107)
Fix relative path calculation if path is on a different drive on Windows (Netzeband, PR 8094)
Don’t infer an abstract type object type from concrete type object values (Xuanda Yang, PR 8096)
Fix an inconsistency with type checking lambda expressions (PR 8080)
We’ve just uploaded mypy 0.750 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
More Powerful Self-types
This version adds support for more patterns involving explicit annotations on self argument in methods. In particular, the mixin pattern got basic support via a protocol as a self annotation that specifies what the mixin expects from any class that subclasses it:
class Lockable(Protocol):
@property
def lock(self) -> Lock: ...
class AtomicCloseMixin:
def atomic_close(self: Lockable) -> int:
with self.lock: # This is OK
... # perform some actions
class File(AtomicCloseMixin):
def __init__(self) -> None:
self.lock = Lock()
class Bad(AtomicCloseMixin):
pass
f = File()
b = Bad()
f.atomic_close() # OK
b.atomic_close() # Error: Invalid self type for "atomic_close"
Another somewhat common pattern that got support via explicit self-types is overloaded generic constructors (in particular in library stubs):
from typing import Any, AnyStr, Generic
from typing_extensions import Literal
class File(Generic[AnyStr]):
@overload
def __init__(self: File[bytes], raw: Literal[True]) -> None: ...
@overload
def __init__(self: File[str], raw: Literal[False]) -> None: ...
@overload
def __init__(self: File[Any], raw: bool = ...) -> None: ...
reveal_type(File()) # type is File[Any]
reveal_type(File(raw=True)) # type is File[bytes]
Stubgen, the automatic stub generator that comes with mypy, has many fixes and improvements. In particular:
Recover from various errors that broke stub generation previously, such as inconsistent MROs or modules that crash on import.
Export imported names that aren't referenced in the module.
Export all names imported from the current package by default. Use --export-less to disable this behavior.
Fix various issues that resulted in syntactically invalid stubs being generated.
Fix signatures of various C extension methods.
Make the docstring parser more robust.
Add various heuristics to skip internal modules, such as tests and vendored packages.
Allow tweaking the level of logging. Add --verbose and --quiet flags.
Generating stub files has never been so easy and robust! Please read more docs here and try it on the libraries you maintain or use that don’t yet have stubs in Typeshed. Please consider contributing any stubs you create to Typeshed so that the community can share and improve them.
Mypy Daemon is No Longer Experimental
The mypy daemon has been the default way of running mypy locally at Dropbox for well over a year and it's quite stable. It also got several additional commands and flags for better customization (and existing flags got better documentation). In particular, mypy daemon now supports basic integration with external file system event watchers for even faster, sub-second type checking results over tens of thousands of files.
Use dmypy recheck --update FILE and dmypy recheck --remove FILE to directly tell the daemon which files got edited/deleted to avoid waiting for the built-in file system watcher. For more details, see the docs.
Static Inference of Annotations (Experimental)
The mypy daemon now supports (as an experimental feature) statically inferring a draft type annotation for a given function or method. Running dmypy suggest FUNCTION will produce a suggested signature in this format:
(param_type_1, param_type_2, ...) -> ret_type
This is a low-level command that can be used by editors, IDEs, and other tools, such as the mypy plugin for PyCharm, to propose an annotation to the user, and/or to insert an annotation to a source file.
In this example, the function format_id() has no annotation:
This release includes three major mypy plugin API changes:
There is a new plugin hook for specifying per-module configuration data: report_config_data(). It can be used if the plugin has some sort of per-module configuration that can affect type checking.
*(breaking change)* New TypeAliasType type has been added and is now used for type aliases instead of eagerly expanding them. To get the expanded type use mypy.types.get_proper_type(). See this comment for more compatibility instructions.
*(breaking change)* Symbol node methods .fullname() and .name() were replaced with properties. See this comment for compatibility instructions.
These improvements are related to plugins:
Invoke the get_dynamic_class_hook() plugin hook also on method calls (Seth Yastrov, PR 7990)
Fix plugin invocation on class method calls through Type[...] (PR 7969)
Allow new plugins to be added by programs (like mypyc) that directly invoke build (PR 7875)
For more information on plugin hooks see the docs. Breaking API changes are announced in this GitHub issue. Please subscribe to it if you are an active mypy plugin developer.
Typeshed Updates
Many small improvements were made to typeshed — too many to list. Browse the typeshed commit log here.
Acknowledgments
First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.
Thanks to all mypy contributors who contributed to this release:
ahmadF
Alexandre Viau
beezee
Bob Whitelock
Brandt Bucher
Chad Dombrova
Daniel Hahler
David Euresti
David Tucker
Djedouas
Ekin Dursun
Emmanuel Nosa Evbuomwan
Ethan Smith
Gábor Lipták
henribru
hoefling
Ilaï Deutel
jag426
Lewis Cowles
Maxim Koltsov
Michael Lee
Michael R. Crusoe
Michael Wayne Goodman
Rubin Raithel
Saleem Rashid
Sebastian Rittau
Seth Yastrov
TH3CHARLie
Theodore Liu
Thomas Hisch
Tim Gates
Tudor Brindus
Tuomas Suutari
Vasily Zakharov
Vincent Barbaresi
Wuisch
Xuanda Yang
Zac Hatfield-Dodds
Additional thanks to all contributors to typeshed:
We’ve just uploaded mypy 0.740 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
This also fixes some corner cases in % formatting and allows flagging bad interactions between bytes and strings.
Here are some errors mypy can now catch:
Likely accidental formatting of bytes ("{}".format(b'123') will produce "b'123'" on Python 3 which is likely unintended — the !r specifier should be used when it is intended)
Invalid format string syntax
Invalid conversion type or specifier
Incorrect number of arguments or field names (One thing this will complain about is too many arguments for a format string—this does not cause an exception at runtime but is still usually a bug.)
Type mismatches for numeric formatting
Type errors in index or attribute expressions
Improved check_untyped_defs
The self argument to methods without annotations is now correctly given the object’s type (instead of Any) when using check_untyped_defs (PR 7530).
This allows substantially better checking of untyped methods as well as enabling inference of attribute types in some situations where they would not have been determined in the past.
Additionally, we now suppress most “needs type annotation” errors caused by assigning empty collections (i.e, data = []), in untyped functions.
More Precise Error Locations (Breaking Change)
Reported line and column numbers are now more precise; in particular, incompatible assignment errors are now reported as being an error with the right hand side rather than the left hand side (PR 7578, PR 7569).
In some cases these changes can break # type: ignores, but from our experience they don’t do it too often.
Stricter Treatment of Context Manager Objects
Mypy 0.730 added support for using Literal types in the return type from __exit__ to determine whether it is possible for a with statement to swallow exceptions, which can affect whether mypy thinks code after a with statement is reachable.
To better pinpoint the source of issues caused by this, mypy now gives an error when __exit__ methods are declared to return bool but always return False. See the documentation for more information.
Documentation and Other Error Message Improvements
Add many cross-references to standard library to mypy docs (Oleg Höfling)
Update examples of mutable and non-mutable mapping in cheat sheet (Marco, Gorelli, PR 7620)
Fix syntax errors in code examples (Oleg Höfling, PR 7651)
Always report dmypy crashes to the client (ideally this fix will affect nobody, but…. 😉 , PR 7593)
Fix some notes not being ignored when using # type: ignore[code] (PR 7573)
Check syntax errors in # type: ignore[code] comments (PR 7460)
Fix incorrect error message when a positional argument is missing in a function with multiple default arguments (PattenR, PR 7547)
Fix message when second argument to isinstance is an alias to Any (Cohen Karnell, PR 7504)
Other Notable Improvements and Bug Fixes
Only return a declared type from __new__ if it is a subtype (PR 7372)
Give better types for pathlib.Path.open (Michael Lee, PR 7643)
Fix a crash on missing self with --check-untyped-defs (PR 7669)
Treat __class_getitem__ as an implicit class method, similar to __init_subclass__ (ahmadF, PR 7647)
Allow TypedDict key with literal type during construction (PR 7645)
Support NamedTuple class syntax in all stubs (Sebastian Rittau, PR 7646)
Support eq and order arguments to attr.s (David Euresti, PR 7619)
Fix some false positives when checking raise statements (Patrick Williams, PR 7515)
Fix spurious errors with inherited dataclasses in incremental mode (PR 7596)
We’ve just uploaded mypy 0.730 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
More Precise Error Locations (Breaking Change)
If you call a function with an incompatible argument type, mypy now points the error message to the argument with the incompatible type. Previously mypy pointed to the function being called, which could be confusing in multi-line calls.
Note: You may have to move some # type: ignore comments as a result of this change, since their behavior depends on the line number of reported errors.
Error Codes
Mypy can now optionally display error codes. They are shown within square brackets after each error message:
prog.py:24: error: "str" has no attribute "trim" [attr-defined]
Enable error codes using --show-error-codes (or show_error_codes = True in a configuration file). See the documentation for more information (documentation on individual codes here and here).
Ignoring Specific Error Codes
You can ignore only errors with specific error codes on a particular line by using a # type: ignore[code, ...] comment. This reduces the risk of ignoring unexpected, serious errors when using # type: ignore comments that ignore (almost) all possible errors on a line.
Mypy now uses colored, more user-friendly output by default. Example:
$ mypy mypy
mypy/argmap.py:12: error: Incompatible return value type (got
"List[int]", expected "List[Optional[int]]")
mypy/argmap.py:12: note: "List" is invariant -- see
http://mypy.readthedocs.io/en/latest/common_issues.html#variance
mypy/argmap.py:12: note: Consider using "Sequence" instead,
which is covariant
mypy/checkstrformat.py:238: error: Argument 1 to
"checkers_for_type" has incompatible type "str"; expected "bytes"Found 2 errors in 2 files (checked 154 source files)$ mypy mypySuccess: no issues found in 154 source files
You can use --no-color to disable colored output. You can use --no-error-summary to hide the summary line with the number of errors.
Windows support was contributed by Ethan Smith (requires Windows 10).
Pretty Output Mode
You can use --pretty to display each line which had errors and a caret that points to the location of the error on each line. Example of pretty output:
$ mypy --pretty mypy
mypy/checkstrformat.py:238: error: Argument 1 to "checkers_for_type"
has incompatible type "str"; expected "bytes"c = self.checkers_for_type(specifier.type, ...^Found 1 error in 1 file (checked 154 source files)
This mode also wraps long error messages at word boundaries to improve readability.
Old Semantic Analyzer Removed
This release no longer includes the old semantic analyzer. See mypy 0.720 release notes for information on the new semantic analyzer.
Reachability and Context Managers
The --warn-unreachable option now behaves more correctly with “exception-swallowing” context managers. If a context manager is currently declared to return bool but it never swallows exceptions, you should annotate the return of __exit__ as Literal[False] instead of bool, or otherwise mypy may complain about missing return statements. Example:
from typing import Optional, Type
from types import TracebackType
from typing_extensions import Literal
class Resource:
...
def __exit__(self,
exc_type: Optional[Type[BaseException]],
exc_val: Optional[BaseException],
exc_tb: Optional[TracebackType]) -> Literal[False]:
...
return False
We’ve just uploaded mypy 0.720 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release features a new semantic analyzer and optional warnings about unreachable code. It also includes many other new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
New Semantic Analyzer Used by Default
The new semantic analyzer was introduced in mypy 0.710 and is now the default. The old analyzer is still available under --no-new-semantic-analyzer, but it will be removed in the next mypy release. This release includes many improvements to the new semantic analyzer.
The semantic analyzer is a component of mypy that binds names to definitions. It also keeps track of what sort of thing each name defines, such as a function or a type alias. The new semantic analyzer is a major redesign and achieves several things.
The old semantic analyzer ordinarily processes files in a linear order, from the beginning to the end. This is subtly different in the mypy daemon, which sometimes processes an individual function or a module top level (code not nested within a function) as a single unit. The new semantic analyzer unifies mypy behavior by always analyzing module top-levels separately, before analyzing functions. As a bonus, this more closely reflects Python runtime behavior, since modules are usually fully initialized before any functions are called.
The old semantic analyzer processes each module in an import cycle twice, in an order determined by complicated heuristics. This can cause bogus errors when there are references to another module that hasn’t been processed yet in a dependency cycle. The new analyzer processes modules in a flexible order, and references across an import cycle work significantly more reliably.
The old semantic analyzer can’t always deal with forward references to definitions. The new semantic analyzer improves support for forward references. Example where this helps:
from typing import NamedTuple
f = lambda: A(1) # This forward reference now works
A = NamedTuple('A', [('x', int)])
f()
The new semantic analyzer is stricter than the old semantic analyzer. It catches additional type errors and quality issues. Existing code that passes type checking may require small adjustments.
New Feature: Warn about Unreachable Code
If you run mypy with --warn-unreachable, mypy will warn you about unreachable statements, and expressions that may be never evaluated because of short circuiting. For example:
from typing import Text
def emphasize(text: Text) -> Text:
if isinstance(text, int):
text += 1 # Error: statement is unreachable
return text.upper()
See the documentation for more information. This feature was contributed by Michael Lee.
Plugin API Changes
The new semantic analyzer may require changes to mypy plugins. We have a GitHub issue with more information for maintainers of plugins.
Notable Improvements and Bug Fixes
Have --package usage respect mypy_path (Aaron Batilo, PR 6926)
Support flexible option specification in config files. Now all the flags support invertible forms so that one can specify either strict_optional = False or no_strict_optional = True. This matches how flags work on the command line (PR 7054)
Fix reachability inference with isinstance(Any, Any) (Michael Lee, PR 7048)
Fix mypyc crash with plugins using get_customize_class_mro_hook() (PR 7075)
Fix slicing tuples with non-literal expressions (Ethan Smith, PR 7066)
Fixes to column numbers in error messages (PR 7078, PR 7081)
Make Need type annotation for variable error more consistent (PR 7079, PR 7113)
Fix error reporting context for missing generic type arguments (PR 7100)
Use only directories containing no __init__.py as namespace packages (Arne Welzel, PR 7108)
Support method plugin hooks on union types (PR 6560)
Fix some corner cases in --strict-equality (PR 7156, PR 7164)
We’ve just uploaded mypy 0.711 to the Python Package Index(PyPI). Mypy is a static type checker for Python. This fixes two minor issues with mypy 0.710. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
Details
The following two issues in mypy 0.710 were fixed:
Revert typeshed PR 2878(“Definefunctools.partial as overloaded function instead of its own class”). This caused too many false positive errors in real-world code.(PR3077)
Fix MYPYC_BLACKLIST on Windows. This broke running dmypy on Windows.(PR7032)
We’ve just uploaded mypy 0.710 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release features a new, experimental semantic analyzer and inline configuration options. It also includes many other new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the full documentation for this release on Read the Docs.
New Semantic Analyzer (Experimental)
We invite everyone with more than a passing interest in mypy to try out the new semantic analyzer. Please report any issues you might find, since we will likely make it the default in the next mypy release, and we will drop the old semantic analyzer soon after.
The semantic analyzer is a component of mypy that binds names to definitions. It also keeps track of what sort of thing each name defines, such as a function or a type alias. This release officially supports a revamped implementation of the semantic analyzer, dubbed the new semantic analyzer. In particular, it deals better with import cycles and forward references.
You can enable the new semantic analyzer by passing --new-semantic-analyzer on the command line or by setting new_semantic_analyzer = True in your mypy config file (typically mypy.ini). (docs)
Inline Configuration Options
You can now specify per-file configuration flags in # mypy: comments. For example, you can now put
# mypy: no-strict-optional
at the top of a file to disable strict optional checking in that specific file. Previously you would have to specify this on the command line (e.g., mypy--strict-optional) or in the mypy config file (e.g., strict_optional = True in the [mypy] section or in a file-specific section).
Multiple flags can be separated by commas or placed on separate lines.
Add a get_additional_deps hook for plugins, to support django-stubs. (PR 6598)
Python 3.8 related changes
Support / for positional-only arguments (PEP 570). (Brandt Bucher, PR 6900)
Support importing Protocol, TypedDict, Literal, Final and @final from typing (typing_extensions is still supported). Ironically, this is also supported Python 2.7. These were recently standardized by PEP 544 (Protocol and @runtime_checkable), PEP 586 (Literal), PEP 589 (TypedDict) and PEP 591 (Final/@final).
Fix line numbers assigned to class and def and ignoring for Python 3.8. (Brandt Bucher, PR 6753)
Expression-scoped ignores in Python 3.8. (Brandt Bucher, PR 6648)
Many small improvements were made to typeshed — too many to list. Browse the typeshed commit log here.
Acknowledgments
First of all, we’d like to thank our employer, Dropbox, for funding the mypy core team.
Thanks to all mypy contributors who contributed to this release:
Alexander Bliskovsky
Allison King
Amanda Walker
Anthony Sottile
Bernát Gábor
Brandt Bucher
Brooke
cclauss
Charles-Axel Dein
Christopher Sabater Cordero
crusaderky
Daniel Hahler
duke9509
El Khadiri Yassine
Ekin Dursun
Emil Goldsmith Olesen
Ethan Smith
Felipe de Morais
Ivar
Jan Szopinski
Jared Hance
Jason Michalski
Jelle Zijlstra
Joe Juzl
kcaebe
marco-dd
Matas Kairaitis
Max Mautner
Michael Lee
Nick Gaya
PattenR
Rafael Carício
Robin Chen
Rémi Lapeyre
Samuel Williams
Sanjit Kalapatapu
Seth Yastrov
viourr
Additional thanks to all contributors to typeshed:
We've just uploaded mypy 0.701 to
PyPI. This is a bugfix release
containing no new features, only fixes for some important bugs and
regressions in the (default) mypyc-compiled version of
0.700.
You can install it as follows:
We’ve just uploaded mypy 0.700 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release switches to a compiled version of mypy, which is up to 4x faster than the previous release. It also includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
python3 -m pip install -U mypy
You can read the documentation for this release on Read the Docs.
Much Improved performance
We are now shipping a mypy binary compiled with mypyc by default, which is up to 4x faster than the interpreted version shipped previously. The compiled version has been available as a separate PyPI distribution for a while now, and this is first release where it’s installed by default. Mypyc is a compiler that compiles type-annotated Python modules into efficient CPython C extension modules.
Note that the legacy, optional mypy_mypyc package will not be updated any more. If you used the mypy_mypyc package for an earlier mypy release, you should uninstall it first, before updating to the new package (you can safely ignore this if don’t know what this is):
python3 -m pip uninstall mypy_mypyc
The compiled version is supported on 64-bit macOS, Windows and Linux platforms (Python 3.5 and later). On other platforms an interpreted version is used instead, which doesn’t provide any performance improvement. The compiled version is mostly compatible with the interpreted version, but some mypy plugins may be incompatible with it. You can still install the interpreted version of 0.700 like this:
$ python3 -m pip install --no-binary mypy -U mypy
Stricter Equality Checks
If you run mypy with --strict-equality, mypy will detect equality and in checks that are likely to always result in True or False, since the operand types are not overlapping. For example, it will warn about comparing a str value to a bytes value, which can never be equal. See the documentation for more information.
Python 3.4 Unsupported for Running Mypy
You can no longer run mypy using Python 3.4, since Python 3.4 has reached its end of life. You can still type check code that targets Python 3.4, and we have no plans to drop support for this.
Plugin System Improvements
Add a get_additional_deps() plugin hook to support django-stubs (PR 6598)
Use the get_attribute_hook() plugin hook for dynamic classes (lincolnq, PR 6371)
Other Improvements
Allow # type: ignore comments after type comments (Ethan Smith, PR 6591)
Don't resolve callable NamedTuple fields to their return type (Danny Weinberg, PR 6576)
Support indexing unions containing tuples (PR 6475)
Python is a famously dynamic language. It’s easy to write DSL-like frameworks that are hard to understand by static type checkers. Although with recent mypy features, such as protocols and literal types, and with basic metaclass and descriptor support, we can more often provide precise types, it’s still often hard to avoid false positives and negatives. To solve this problem, while avoiding the need for custom type system features for every framework, mypy supports a plugin system. Plugins are Python modules that provide callbacks (plugin hooks) that mypy will call when type checking classes and functions that interact with a library or framework. For example, such hooks can provide a more precise function return type that is otherwise hard to express, or auto-generate some methods in a class, to reflect the effects of a class decorator. To read more about plugin system architecture and for the full list of available hooks, see these docs.
Bundled plugins for standard library
Mypy comes with a default plugin for some builtin functions and classes, and the ctypes, contextlib, and dataclasses modules. It also includes a plugin for attrs (for historical reasons — it was the first third-party plugin written for mypy). These plugins allow mypy to infer more precise types and correctly type check code using these library features. To illustrate this, consider this snippet:
from dataclasses import dataclass
from typing import Generic, TypeVar
@dataclass
class TaggedVector(Generic[T]):
data: List[T]
tag: str
position = TaggedVector([0, 0, 0], 'origin')
Above, get_class_decorator_hook() is called when analyzing the class definition. It adds autogenerated methods, including __init__(), to the class body. Mypy uses this generated constructor to correctly infer TaggedVector[int] as the type of position. As you can see, plugins work even with generic classes.
Here’s another snippet:
from contextlib import contextmanager
@contextmanager
def timer(title: str) -> Iterator[float]:
...
with timer(9000) as tm:
...
The get_function_hook() hook provides a precise return type for the contextmanager decorator, so that calls to the decorated function can be type-checked precisely. Mypy can now detect an error: the argument to timer() must be a string.
Combining plugins and stubs
In addition to relying on dynamic Python features, frameworks often have the challenge of having large APIs. Mypy needs stub files for the libraries to check code that uses those libraries (unless the library contains inline annotations, which is still often not the case). Distributing the stubs for large frameworks via typeshed is not very practical:
Typeshed has a relatively slow release cycle (it’s bundled with mypy).
Incomplete stubs can cause false positives that are hard to avoid.
There’s no easy way to mix and match stubs from different typeshed versions.
Stub packages, introduced in PEP 561, help with these issues:
Maintainers can release a stub package as often as they want.
Users who haven’t opted in to using the package won’t see any false positives.
You can freely install arbitrary versions of multiple different stub packages.
Moreover, pip allows combining stubs for a library and the corresponding mypy plugin into a single distribution. Stubs for a framework and a corresponding mypy plugin can now be easily developed and distributed together, which is often useful since plugins fill in missing or imprecise definitions in the stubs.
A recent example of such a package is SQLAlchemy stubs and plugin, the first public 0.1 release of which was published on PyPI earlier this month. Although the project is still in early alpha, we are already successfully using it at Dropbox to improve type checking precision. Currently the plugin understands basic ORM declarations:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
In the above snippet, the plugin uses the get_dynamic_class_hook() hook to tell mypy that Base is a valid base class, even though it doesn’t look like one. Then get_base_class_hook() is called on the definition of User, and it adds some autogenerated attributes. Next we instantiate the model:
user = User(id=42, name=42)
The get_function_hook() hook is called, so that mypy can spot the error here: an integer is given instead of a user name.
The stubs define Column as a generic descriptor, so that attributes on a model get correct types:
id_col = User.id # Inferred type is "Column[int]"
name = user.name # Inferred type is "Optional[str]"
We welcome PRs that add more precise types to the stubs (the progress for core modules is tracked here).
Here are some gotchas that we found while working on the stubs:
Use a module-level __getattr__() to avoid false positives during the very early stages, when stubs are incomplete (this suppresses mypy errors due to missing module attributes). You can also use it in __init__.py files if some submodules are missing.
Descriptors often help with giving more precise types for customized attribute access (like in the Column example above), and it’s OK to use them even if the actual runtime implementation uses a more complex mechanism involving a metaclass, for example.
Don’t hesitate to declare framework classes as generic in stubs. Although they are not generic at runtime, this often allows us to give much more precise types for certain framework features, while runtime errors can easily be worked around. (We hope that frameworks will gradually add built-in support for generic types by explicitly inheriting relevant classes from typing.Generic.)
Recently released mypy plugins
There are already several plugins available for some popular Python frameworks. Apart from the above mentioned plugin for SQLAlchemy, other notable recent examples of packages with stubs and a bundled mypy plugin include stubs for Django and Zope Interfaces. All these projects are under active development.
Installing and enabling plugin + stub packages
Use pip to install a package with a mypy plugin and/or stubs in the virtual environment where mypy is installed:
$ pip install sqlalchemy-stubs
Mypy should automatically discover the installed stubs. To enable the plugins you’ve installed, explicitly include them in your mypy.ini (or a custom config file):
[mypy]
plugins = sqlmypy, mypy_django_plugin.main
Developing mypy plugins and writing stubs
If you want to develop a stubs + plugin package for a framework you use, you can use the sqlalchemy-stubsrepository as a template. It includes the setup.py file, testing infrastructure using data-driven tests, and an example plugin class with a bunch of plugin hooks. We recommend using stubgen, an automatic stub generator that comes with mypy, to get started with stubs. Stubgen got several improvements in mypy 0.670.
For more details about mypy plugin system, see the docs. You can also just browse the source code of the plugins mentioned above. If you have any questions, feel free to ask them on the Python typing Gitter chat.
We’ve just uploaded mypy 0.670 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes new features, bug fixes and library stub (typeshed) updates. You can install it as follows:
You can read the documentation for this release on Read the Docs.
New Feature: Variable Redefinition
As you might have experienced, PEP 484 and mypy require a variable to keep its initial type throughout a scope. Occasionally this can be annoying, as the traditional workaround is to rename the second (independent) use of the variable. We have introduced a new flag, --allow-redefinition, which relaxes this behavior in certain contexts. Here’s an example where this may be useful:
def process(items: List[str]) -> None:
# 'items' has type List[str]
items = [item.split() for item in items]
# 'items' now has type List[List[str]]
...
You can also enable this per module in the mypy.ini file using allow_redefinition = True. See the docs for the command-line flag and the config option.
Stubgen Improvements
We’ve done a fairly large overhaul of the stubgen utility, which can automatically generate draft stub files for almost any module or package:
Streamline the command-line interface; in particular, the source discovery/collection options now more closely resemble those of mypy itself: -m <module>, -p <package>, <file> or <directory>
Perform a lightweight semantic analysis to generate somewhat better stubs
When parsing types out of docstrings (this happens for C modules only), stubgen now handles overloaded signatures generated by pybind11 (Wiktor Niesiobędzki, PR 5975)
Always infer in operator as returning bool (Joel Croteau, PR 5688)
Allow star args in ctypes.Array constructor (Alan Du, PR 6213)
Fix plugin invocation for __call__ methods (oremanj, PR 6334)
Implement the XDG directory spec for config files: $XDG_CONFIG_HOME/mypy/config is now included in the search path for config files (Ryan Delaney, PR 6304)
When using the --junit-xmlflag, the Python version and platform in the junit.xml file are now formatted as mypy-py3_6-windows — previously this was mypy-py3.6-windows but the dot was misinterpreted by some tools (PR 6222)
Update the typed_ast dependency to version 1.3.1; this means we now officially support Python 3.7
Temporarily delete pyproject.toml from the repo in order to work around a pip bug (PR 6342)
Include mypy_bootstrap.ini in PyPI packages (John Reese, PR 6252)
Internal Improvements and Code Cleanup
Fix daemon overlapped I/O when more data needs to be read (Ethan Smith, PR 6272)
Move most message constants to mypy.message_registry (Chad Dombrova, PR 6194)
Fix all DeprecationWarning: invalid escape sequence (Mickaël Schoentgen, PR 6195)
Add strictness flags (for mypy itself) that can be added with no source code changes (Michael Lee, PR 6237)
Some daemon performance improvements for large code bases
