asyncio: recv_into() must not return b'' if the socket/pipe is closed

[vstinner]

BPO	41467
Nosy	@db3l, @vstinner, @asvetlov, @1st1, @miss-islington
PRs	bpo-41467: Fix asyncio recv_into() on Windows #21720 [3.9] bpo-41467: Fix asyncio recv_into() on Windows (GH-21720) #21724 [3.8] bpo-41467: Fix asyncio recv_into() on Windows (GH-21720) #21726

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-08-05.00:09:54.697>
created_at = <Date 2020-08-03.22:19:49.785>
labels = ['3.10', 'expert-asyncio']
title = "asyncio: recv_into() must not return b'' if the socket/pipe is closed"
updated_at = <Date 2020-08-05.18:49:30.857>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-08-05.18:49:30.857>
actor = 'db3l'
assignee = 'none'
closed = True
closed_date = <Date 2020-08-05.00:09:54.697>
closer = 'vstinner'
components = ['asyncio']
creation = <Date 2020-08-03.22:19:49.785>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41467
keywords = ['patch']
message_count = 7.0
messages = ['374760', '374762', '374764', '374778', '374780', '374782', '374895']
nosy_count = 5.0
nosy_names = ['db3l', 'vstinner', 'asvetlov', 'yselivanov', 'miss-islington']
pr_nums = ['21720', '21724', '21726']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue41467'
versions = ['Python 3.10']

[vstinner]

The proactor event loop of asyncio returns b'' on recv_into() if the socket/pipe is closed:

    def recv_into(self, conn, buf, flags=0):
        ...
        try:
            ...
        except BrokenPipeError:
            return self._result(b'')
        ...

But a socket recv_into() must always return a number: the number of written bytes. Example with socket.socket.recv_into() method:
https://docs.python.org/3/library/socket.html#socket.socket.recv_into

IMHO zero must be returned here. This bug may be the root cause of bpo-38912 bug.

Attached PR fix the issue.

[vstinner]

The function was added in 2017 in bpo-31819, PR 4051:

commit 525f40d
Author: Antoine Pitrou <pitrou@free.fr>
Date: Thu Oct 19 21:46:40 2017 +0200

bpo-31819: Add AbstractEventLoop.sock_recv_into() (bpo-4051)

[vstinner]

This bug may be the root cause of bpo-38912 bug.

Yeah, very likely. This bug makes asyncio inconsistent. A transport is "not closed" and "closed" at the same time...

C:\vstinner\python\master\lib\asyncio\proactor_events.py:121: ResourceWarning: unclosed transport <_ProactorReadPipeTransport>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Warning -- Unraisable exception
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F30C938EB0>
Traceback (most recent call last):
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 122, in __del__
    self.close()
  File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 114, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

[vstinner]

New changeset 602a971 by Victor Stinner in branch 'master':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
602a971

[miss-islington]

New changeset b934d83 by Miss Islington (bot) in branch '3.8':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
b934d83

[miss-islington]

New changeset 1d16229 by Miss Islington (bot) in branch '3.9':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
1d16229

[db3l]

Just for the record, this fix also appears to have resolved the issue with the Win10 buildbot from bpo-41273, which was related to the same unraisable exceptions mentioned in bpo-38912.