AsyncIO strange behaviour

[greckov]

BPO	42538
Nosy	@ncoghlan, @asvetlov, @1st1, @nanjekyejoannah, @greckov
Superseder	bpo-44665: asyncio.create_task() documentation should mention user needs to keep reference to the task
Files	output.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-08-24.02:30:11.718>
created_at = <Date 2020-12-02.12:17:27.668>
labels = ['type-bug', '3.7', 'expert-asyncio']
title = 'AsyncIO strange behaviour'
updated_at = <Date 2021-10-22.14:42:50.369>
user = 'https://github.com/greckov'

bugs.python.org fields:

activity = <Date 2021-10-22.14:42:50.369>
actor = 'nanjekyejoannah'
assignee = 'none'
closed = True
closed_date = <Date 2021-08-24.02:30:11.718>
closer = 'ncoghlan'
components = ['asyncio']
creation = <Date 2020-12-02.12:17:27.668>
creator = 'greckov'
dependencies = []
files = ['49644']
hgrepos = []
issue_num = 42538
keywords = []
message_count = 6.0
messages = ['382307', '382309', '382310', '400194', '404777', '404778']
nosy_count = 5.0
nosy_names = ['ncoghlan', 'asvetlov', 'yselivanov', 'nanjekyejoannah', 'greckov']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '44665'
type = 'behavior'
url = 'https://bugs.python.org/issue42538'
versions = ['Python 3.7']

[greckov]

Hi! I've met strange behaviour related to the coroutine execution. Probably it's somehow related to the asyncio.Queue get() method. I have the following lines of code:
class WeightSource:
def __init__(self):
self._queue = asyncio.Queue(maxsize=1)
def __aiter__(self):
return self
async def __anext__(self):
return await self._queue.get()

async def _preconfigure(app: web.Application) -> None:
    setup_db()
    # asyncio.create_task(WeightSource.listen_scales('/dev/ttyACM0', 9600)
    asyncio.create_task(handle_weight_event_creation()) # coroutine prints WEIGHT <int>
    await create_track_tasks()

async def create_track_tasks():
    asyncio.create_task(track_scales_availability())
    asyncio.create_task(track_crm_availability())

async def track_scales_availability():
    async for weight in WeightSource():
        print(weight)

When I'm trying to run _preconfigure coroutine (automatically started by aiohttp), i see the message: ERROR:asyncio:Task was destroyed but it is pending. The strange things two:
The process and loop remains alive (so by the logic error should be not shown) and the second thing is when I export PYTHONASYNCIODEBUG=1 everything works well without any error.
On unset this variable the error returns. When I don't use asyncio.Queue and just place asyncio.sleep(), coroutine doesn't fall. Error happens inside the async for weight in WeightSource() statement. Why PYTHONASYNCIODEBUG changes behaviour?

[asvetlov]

I understand the issue as the following:

1. You don't save the result of asyncio.create_task() somewhere.
   The task object is dereferenced and thus you see a warning.

2. With PYTHONASYNCIODEBUG on the task is referenced also by internal debug structure, that's why you don't see the warning.

Perhaps we should store a weak reference to task in _source_traceback

Minor thing but nice to have it fixed.

[greckov]

Thanks! That's resolved my problem, but this thing wasn't really obvious as for me. Probably it would be better to write about this explicitly in docs for the create_task. All in all this issue can be closed.

[ncoghlan]

While filed later, bpo-44665 correctly describes the problem reported here: asyncio is behaving as expected, but the docs don't make it clear that you need to save a reference to the result of ensure_future()/create_task() if you don't want to risk the task unexpectedly disappearing mid execution.

So marking this as a duplicate of that docs issue.

[nanjekyejoannah]

I opened a documentation PR agreeing with @ncoghlan, that adds this behavior. See https://bugs.python.org/issue44665

[nanjekyejoannah]

I meant, "that describes the behaviour"