Clean up pending tasks when event_loop fixture leaves scope

[sureshjoshi]

I'm not sure if this would be considered a bug or a feature :)

I noticed that in some of my tests, my long running tasks (e.g. while loops) were throwing warnings about tasks being destroyed but were pending. I guess this is happening because the loop is closed at the end of a test, but my while loops are still while'ing

In my projects, I maintain a cleanup function at the end of my main function to keep track of what tasks are still alive, then I cancel them before shutting down the loop. The tasks themselves are running forever intentionally, and they should only shut down when the app shuts down (and I shut down gracefully for cleaner logs).

I was wondering if this functionality should be added to the event_loop fixture (maybe optionally) to silence warnings where they don't really belong? Or, is this the responsibility of the developer via a custom fixture?

[diefans]

IMHO you are responsible to cancel your long running tasks before the test exits

[seifertm]

There are very few things that need to be solved for the corresponding PR to get merged. Hence I'm marking this as good first issue.

[priyanshu-panwar]

Can I take this issue and solve it? @seifertm

[seifertm]

@priyanshu-panwar Of course, give it a try.

The existing PR is generally good. My main concerns are the use of non-public CPython APIs (i.e. the use of asyncio.runners._cancel_all_tasks) and the explicit call to gc.collect.

[samypr100]

@seifertm One thought I had is that an alternative, at least on Python 3.11 and above is to use the new asyncio.Runner.

@pytest.fixture(scope="session")
def event_loop() -> Iterator[asyncio.AbstractEventLoop]:
    with asyncio.Runner() as runner:
        yield runner.get_loop()

That would at least help with the cleanup and avoid using non-public API's. A loop factory could be provided for use-cases like using uvloop.

[seifertm]

@samypr100 I fully agree that using asyncio.Runner is the way to go for pytest-asyncio in the future.
This is tightly connected to #127.

As we speak, pytest-asyncio v0.23 is in the making which contains substantial changes, such as the deprecation of event_loop fixture overrides. I don't want to include any additional functionality, because the release is big enough as it is. In fact, the v0.22 had to be retracted due to problems connected to the deprecation.

Once the deprecation is in place, we can come back to this issue and investigate replacing existing calls to loop.run_until_complete() with calls to asyncio.Runner. Note that pytest-asyncio still supports Python 3.8, where asyncio.Runner isn't available. So we also need to find ways to backport that functionality.

[samypr100]

@seifertm In order to attempt help, I took a stab at a working asyncio.Runner backport that works all the way down to Python 3.8 and that its also compatible with uvloop. It preserves contextvars and passes CPython's existing unit tests for asyncio.Runner. If you're interested, here's the package https://pypi.org/project/backports.asyncio.runner/