asyncio open_connection fails when a socket is explicitly closed

[danielen1337]

BPO	43253
Nosy	@pfmoore, @tjguk, @asvetlov, @zware, @1st1, @zooba, @mhils, @eamanu, @miss-islington
PRs	bpo-43253: don't shutdown invalid socket handles #31892 [3.9] bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892) #31905 [3.10] bpo-43253: Don't call shutdown() for invalid socket handles (GH-31892) #31906

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-15.15:49:39.190>
created_at = <Date 2021-02-18.13:46:25.959>
labels = ['3.10', '3.11', '3.9', 'OS-windows', 'expert-asyncio']
title = 'asyncio open_connection fails when a socket is explicitly closed'
updated_at = <Date 2022-03-16.12:19:30.989>
user = 'https://bugs.python.org/danielen1337'

bugs.python.org fields:

activity = <Date 2022-03-16.12:19:30.989>
actor = 'mhils'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-15.15:49:39.190>
closer = 'asvetlov'
components = ['Windows', 'asyncio']
creation = <Date 2021-02-18.13:46:25.959>
creator = 'danielen1337'
dependencies = []
files = []
hgrepos = []
issue_num = 43253
keywords = []
message_count = 12.0
messages = ['387228', '387235', '387238', '414620', '415071', '415128', '415205', '415250', '415254', '415255', '415257', '415338']
nosy_count = 11.0
nosy_names = ['paul.moore', 'tim.golden', 'asvetlov', 'zach.ware', 'yselivanov', 'steve.dower', 'mhils', 'eamanu', 'miss-islington', 'danielen1337', 'adhika.setyap']
pr_nums = ['31892', '31905', '31906']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue43253'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[danielen1337]

Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import socket
   ...: s1, s2 = socket.socketpair()
   ...: import asyncio
   ...: async def test():
   ...:     r1, w1 = await asyncio.open_connection(sock=s1)
   ...:     r2, w2 = await asyncio.open_connection(sock=s2)
   ...:     s1.close()
   ...: asyncio.run(test())
Exception in callback _ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None))
handle: <Handle _ProactorBasePipeTransport._call_connection_lost(ConnectionAbo...e, 1236, None))>
Traceback (most recent call last):
  File "c:\python39\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "c:\python39\lib\asyncio\proactor_events.py", line 162, in _call_connection_lost
    self._sock.shutdown(socket.SHUT_RDWR)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

[eamanu]

I cannot reproduce on a Debian machine. Seems to be a Windows component issue?

[danielen1337]

It reproduced on a windows machine

[asvetlov]

Please consider passing 'sock' argument as the ownership transfer.

You should not perform any action on 'sock' object directly anymore.
This is true for all asyncio API.

[mhils]

We are hitting the same traceback with mitmproxy on Windows without ever passing a socket object to open_connection. In other words, this can be triggered without performing any action on 'sock' due to some race conditions in the Windows network stack. I unfortunately don't have a better repro for that race other than what Daniel provided.