asyncio.TaskGroup does not close unawaited coroutines

[arthur-tacca]

Bug report

Bug description:

Consider the following code:

import asyncio

async def wait_and_raise():
    await asyncio.sleep(0.5)
    raise RuntimeError(1)

async def wait_and_start(tg):
    try:
        await asyncio.sleep(1)
    finally:
        try:
            tg.create_task(asyncio.sleep(1))
        except RuntimeError as e:
            print(f"wait_and_start() caught {e!r}")

async def main():
    try:
        async with asyncio.TaskGroup() as tg:
            tg.create_task(wait_and_start(tg))
            tg.create_task(wait_and_raise())
    except Exception as e:
        print(f"main() caught {e!r}")

    try:
        tg.create_task(asyncio.sleep(1))
    except RuntimeError as e:
        print(f"main() caught {e!r}")

asyncio.run(main())

This gives the following output

wait_and_start() caught RuntimeError('TaskGroup <TaskGroup tasks=1 errors=1 cancelling> is shutting down')
C:\code\taskgrouptest.py:16: RuntimeWarning: coroutine 'sleep' was never awaited
  print(f"wait_and_start() caught {e!r}")
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
main() caught ExceptionGroup('unhandled errors in a TaskGroup', [RuntimeError(1)])
main() caught RuntimeError('TaskGroup <TaskGroup cancelling> is finished')
C:\code\taskgrouptest.py:29: RuntimeWarning: coroutine 'sleep' was never awaited
  print(f"main() caught {e!r}")
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Arguably, when you call tg.create_task() on a task group that is shutting down or has finished, the calling code "knows" about the error because it gets a RuntimeError exception (as you can see above), so there is no need to get a warning about a coroutine that was not awaited. So, when a TaskGroup encounters this situation, it should close the coroutine before raising the error.

The other argument would be that this still represents a design mistake so should still get the warning. I can see both points of view but I'm raising this issue so a conscious decision can be made.

For comparison, when you do this on a Trio Nursery or AnyIO TaskGroup that has already closed, a coroutine never even gets created in the first place, because you use a different syntax (nursery.start_soon(foo, 1, 2) rather than tg.create_task(foo(1, 2))), so it's a lot like if asyncio were to close the coroutine. The situation is a bit different for a nursery that is shutting down: then it runs till the first (unshielded) await and is cancelled at that point, which is possible because they use level-based cancellation rather than edge-based cancellation.

CPython versions tested on:

3.12

Operating systems tested on:

Windows

Linked PRs

• gh-115957: Close coroutine if the TaskGroup is inactive  #116009

[arthur-tacca]

I'm told loop.create_task() has the same problem if the loop is closed.

[gvanrossum]

I see. Reproduced and understood. The issue is in this code at the top of TaskGroup.create_task():

cpython/Lib/asyncio/taskgroups.py

Lines 156 to 161 in 6087315

	if not self._entered:
	raise RuntimeError(f"TaskGroup {self!r} has not been entered")
	if self._exiting and not self._tasks:
	raise RuntimeError(f"TaskGroup {self!r} is finished")
	if self._aborting:
	raise RuntimeError(f"TaskGroup {self!r} is shutting down")

If we inserted coro.close() before each of the three raise statements there we would no longer get the warning. When using the common pattern, the coroutine would be closed without executing a single line of code.

If the coroutine argument is something else, who knows, but the create_task() argument is expected to conform to the complete protocol of coroutines, so if someone is duck typing this, they should already be prepared for this.

It's also the case that if someone squirrels away a reference to the coroutine object, they can tell that it hasn't been closed (it's tricky, you basically have to try to call its send() or throw() method), but that also feels like a very remote possibility where someone is already breaking the contract for create_task().

So I think this is a reasonable fix, but I wouldn't want to backport it to 3.12 or before.

Would you be interested in submitting a PR? The fix itself is trivial, but it would be nice if there was a test for the new behavior (this can be adapted from your example, using coro.send(None) to test whether the coroutine has already been closed). A "versionchanged" directive in the docs for TaskGroup.create_task() would also be appreciated, as well as a brief mention in "What's new in 3.13". Let me know if you need more pointers.

Regarding loop.create_task(), the logic there is a bit trickier, and the use case feels weaker, so I'd leave that alone for now.

[Jason-Y-Z]

Sorry for jumping in, but this looks quite relevant to what I'm working on, so I had a go at implementing the simple change, in #116009, and modified the tests accordingly.