Look into using mypy's daemon

https://mypy.readthedocs.io/en/latest/mypy_daemon.html

Looks like as of June 2018 work has begun for Windows daemon support.
python/mypy#5019

FYI, mypy master now has a working implementation of the mypy daemon for Windows if you want to play with it.

How would we be able to use dmypy within vscode? Do you mean just for testing via the command line? Or is there a proper way to integrate this with vscode currently? If not are there plans to use the daemon or give the option?

Eric Hiller (@erichiller)

Eric Hiller (@erichiller) Until there is more official support, as a work around you can use something like the following:

{
    "python.linting.mypyEnabled": true,
    "python.linting.mypyPath": "C:\\Python37\\Scripts\\dmypy.exe",
    "python.linting.mypyArgs": [
        "run",
        "--",
        "--ignore-missing-imports",
        "--follow-imports=skip"
    ]
}

Though this is rather hackish. (It won't kill the daemon when VS Code closes, which likely is not the behavior you want). Also this is rather abusing the whole "set an executable and args" settings system...

Do note you must change the argument for following imports to either skip or error for now.

Eric Hiller (@erichiller)

Eric Hiller (@erichiller) we would like to add support, but we don't have any ETA on when this will get prioritized high enough for us to get around to it.

ethanhs thanks for the work-around!

Ivan Levkivskyi (@ilevkivskyi)

Ivan Levkivskyi (@ilevkivskyi)

Thanks for connecting with us at PyCon, looking forward to landing this in the extension.

Lets get the ball rolling.
A few questions (there'll be a few more, but lets start with a few simple ones):

How do we start multiple daemons for different repositories? The docs don't cover that.
- Assume user opens two instances of VS Code, both instances of VSC will now run dmypy start.
- My assumption is, this will result in two instances of the daemon.
Will shutting down the parent process kill the daemon automatically or is it a detached sub process?
- I.e. if we were to kill vscode (non-graceful shutdown), will this result in the daemon evetually killing itself, or will it stay alive and never die?
Silyy question:
- Assume we have two instances of vscode, one using Python 2.7 another using python 3.7. When Python2.7 sends requests to the daemon, will it send the messages to the daemon that it started.
- I guess the answer is obviously yes, just don't want to make assumptions here.
Cancellation,
- Assume we send a command to lint a large file, and immediately after that we send the same file (but with some changes). Will mypy cancel the previous request or will it simply queue the request. I.e. wait for previous request to complete.
Multiple requests
- Assume we send a command to lint a large file
- While we're waiting for that request to be processed we send another request to the same daemon to process a large file, will the daemon block on the previous request or somehow process this in parallel?

Don Jayamanne (@DonJayamanne)

Don Jayamanne (@DonJayamanne)

How do we start multiple daemons for different repositories?

I think the simplest way is to set current working directory to the repo root when calling dmypy start command.

[...] I.e. if we were to kill vscode (non-graceful shutdown), will this result in the daemon evetually killing itself, or will it stay alive and never die?

No it will not die. Process created by dmypy start (client) exits immediately, but it leaves the daemon (server) "detached". The daemon however will shut down after a timeout if it receives no requests (configurable, default is 24 hours). Also calling dmypy kill unconditionally kills the server.

Assume we have two instances of vscode, one using Python 2.7 another using python 3.7. When Python2.7 sends requests to the daemon, will it send the messages to the daemon that it started.

Mypy only supports Python 3.5+, it can type check older versions (including 2.7), but you need to have Python 3 installed to run it. If you want to type check a repo in Python 2 mode and then in Python 3 mode, I think the best way is to start two daemons each one configured with --python-version=x.y (you will also need to pass --status-file XXX to each daemon when starting the server, and when communicating with them using dmypy client to distinguish them).

Assume we send a command to lint a large file, and immediately after that we send the same file (but with some changes). Will mypy cancel the previous request or will it simply queue the request. I.e. wait for previous request to complete

Currently this is not possible. dmypy check and dmypy recheck are blocking calls, i.e. they don't exit before the server responds with the checking, and then they print checking results to stdout and exit. From user point of view dmypy check behaves essentially the same as mypy. You can of course issuer another dmypy check (i.e. create another client process) before previous exits, but I am not sure this is going to end well currently. We discussed the possibility to add dmypy cancel to gracefully abort last type checking request, and it may be not so easy to implement. But potentially we can do this if it is important.

Assume we send a command to lint a large file.

It is not how mypy normally works (not only the daemon), see #5587. I actually believe we should figure out that issue first before continuing here.

Eric Hiller (@erichiller)

Eric Hiller (@erichiller) Until there is more official support, as a work around you can use something like the following:
{
    "python.linting.mypyEnabled": true,
    "python.linting.mypyPath": "C:\\Python37\\Scripts\\dmypy.exe",
    "python.linting.mypyArgs": [
        "run",
        "--",
        "--ignore-missing-imports",
        "--follow-imports=skip"
    ]
}

This doesn't work for me. Here's the Python extension's output:

> ~/Library/Caches/pypoetry/virtualenvs/env-name-py3.8/bin/python ~/.vscode/extensions/ms-python.python-2021.2.576481509/pythonFiles/pyvsc-run-isolated.py -c "import dmypy"

Linter 'mypy' is not installed. Please install it or select another linter".
Error: Module 'dmypy' not installed.

Looks like it's changed how it does mypy linting so that you can't just pass it arbitrary command line arguments.

ethanhs

At the risk of asking a dumb question, if I can get mypy linting using the right json settings per ethanhs ... why do I need to run the daemon? What added benefit does the daemon give me that I'm not already getting by linting?

ethanhs

At the risk of asking a dumb question, if I can get mypy linting using the right json settings per ethanhs ... why do I need to run the daemon? What added benefit does the daemon give me that I'm not already getting by linting?

I believe that (non daemon) mypy must start up and re-read any cached data before beginning to work on the files you pass it, which takes some time. In contrast the daemon remains in memory and has no startup cost for runs after the first, so it should return results much faster.

Robert (@rben01)

Robert (@rben01) Your not even getting that far.

Linter 'mypy' is not installed. Please install it or select another linter".
Error: Module 'dmypy' not installed.

This means mypy is not being found at all. Like because it isn't installed in the python virtual environment you are running in. Ran into the same thing myself under different circumstances.

Ian Stokes-Rees (@ijstokes)

Ian Stokes-Rees (@ijstokes) FWIW I implemented most of what you asked for in my mypy extension a few months ago. It now runs the mypy daemon directly for each workspace folder. I also added the runUsingActiveInterpreter option which I think you were asking for.

I opened a ticket to upgrade dmypy to a language server a while back. I guess that could be relevant for this as well:

Upgrade dmypy to a language server (LSP) python/mypy#10463

Any updates?

This dev container template uses the daemon:

https://github.com/JamesHutchison/python-dev-container-template

I agree that the daemon makes more sense in general, however the daemon has also been somewhat "semi-experimental" as it runs a different code path so I can understand being hesitant to use it by default.

If your repo is very large and has a very large number of errors, both mypy and dmypy are atrociously slow. This is because the daemon uses errors as a signal to reprocess a file, even if it wasn't touched.

We now have a preview version of mypy extension that uses dmypy for linting. Please give it a try:
https://marketplace.visualstudio.com/items?itemName=ms-python.mypy-type-checker

Karthik Nadig (@karthiknadig)

Karthik Nadig (@karthiknadig) just a heads up, using dmypy by default for vscode may cause bugs like this to surface, causing users problems: python/mypy#12249

Tushar Sadhwani (@tusharsadhwani)

Tushar Sadhwani (@tusharsadhwani) The extension allows you to configure it to run using mypy as well using mypy-type-checker.path setting. You can point it to the mypy executable to use that. You can use variable resolution support to configure the mypy-type-checker.path like "${workspaceFolder}.venv\bin\mypy", or if you have mypy globally or from an activated environment, you could even just say "mypy-type-checker.path": ["mypy"]

Closing this against https://marketplace.visualstudio.com/items?itemName=ms-python.mypy-type-checker

FYI, that extension allows you to switch between dmypy or mypy using the mypy-type-checker.preferDaemon setting. Default is dmypy.

Navigation Menu

Look into using mypy's daemon #1577

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Look into using mypy's daemon #1577

Description

Activity

erichiller commented on Jul 26, 2018

emmatyping commented on Nov 27, 2018

erichiller commented on Dec 30, 2018

emmatyping commented on Dec 30, 2018

brettcannon commented on Jan 3, 2019

DonJayamanne commented on May 9, 2019

ilevkivskyi commented on May 9, 2019

13 remaining items

rben01 commented on Feb 19, 2021

adam-grant-hendry commented on Mar 4, 2021

rben01 commented on Mar 5, 2021

reporter123 commented on Aug 15, 2021

matangover commented on Oct 27, 2021

lithammer commented on Oct 28, 2021

KholdStare commented on Jan 13, 2023

JamesHutchison commented on Apr 11, 2023

karthiknadig commented on Jun 14, 2023

tusharsadhwani commented on Jun 14, 2023

karthiknadig commented on Jun 14, 2023

karthiknadig commented on Oct 19, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions