Unittest framework : Automatic test discovery not working if test has a relative import

[Dr-Irv]

This is with Python Language Server version 0.3.43.0 on Windows 10, Anaconda python 3.7.3.

Have a folder with the following structure:

mypkg
  --> foo.py
  --> __init__.py
mypkg_test
  --> __init__.py
  --> base.py
  --> test_foo.py

The files base.py, and both __init__.py are empty.

Here is foo.py:

def goo():
    return 5

Here is test_foo.py:

import unittest
from . import base
from mypkg.foo import goo


class FooTest2(unittest.TestCase):
    def test_foo(self):
        x = goo()
        self.assertTrue(5 == x)
        
if __name__ == '__main__':
    unittest.main()

Here is settings.json in the .vscode folder in that directory:

{
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        "./mypkg_test",
        "-p",
        "test*.py"
    ],
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": true
}

If you comment out from . import base in test_foo.py, then the test is discovered. Commenting and uncommenting that line makes the test discovered versus not discovered, as evidenced by seeing the buttons Run Test|Debug Test in the editor.

If you rename mypkg_test to test, update the test settings to point to the newly named directory, and leave the source with the line from . import base present, then the test is discovered.

So the name of the test folder seems to affect test discovery when doing a relative import inside one of the tests.

We have a big source code with this naming structure (and I don't control the naming) and base.py has a lot more stuff in it, and we can run tests just fine from the command line, but not from VS Code.

[MikhailArkhipov]

LS does not have any test discovery functionality. This looks like extension issue.

[karrtikr]

If you rename mypkg_test to test, update the test settings to point to the newly named directory, and leave the source with the line from . import base present, then the test is discovered.

I am not able to confirm this. For me, tests are still not discovered if I rename mypkg_test to test. But having a relative import breaking discovery is still a bug.

As a workaround, I would ask you to try the pytest framework instead of unittest. Use Python: Configure Tests from command palette to configure pytest, I can confirm it works.

settings.json would look something like this

{
    "python.testing.pytestEnabled": true,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": false,
    "python.testing.pytestArgs": [
        "<Test_directory_name>"
    ]
}

[Dr-Irv]

Kartik Raj (@karrtikr) We are working with an existing large code base that already uses unittest, so switching to pytest really isn't an option. Hopefully, this bug will get fixed