test_ftplib is failing with TLS 1.3

[tiran]

BPO	34391
Nosy	@terryjreedy, @tiran
PRs	bpo-34391: Fix ftplib test for TLS 1.3 #8787 [3.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) #8789 [3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) #8790 [2.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) #8791

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2021-04-19.20:13:29.306>
created_at = <Date 2018-08-13.09:50:35.343>
labels = ['expert-SSL', '3.8', 'type-bug', '3.7']
title = 'test_ftplib is failing with TLS 1.3'
updated_at = <Date 2021-04-19.20:13:29.306>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2021-04-19.20:13:29.306>
actor = 'christian.heimes'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2021-04-19.20:13:29.306>
closer = 'christian.heimes'
components = ['SSL']
creation = <Date 2018-08-13.09:50:35.343>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 34391
keywords = ['patch']
message_count = 8.0
messages = ['323472', '323608', '323612', '323615', '323616', '323617', '323678', '391404']
nosy_count = 2.0
nosy_names = ['terry.reedy', 'christian.heimes']
pr_nums = ['8787', '8789', '8790', '8791']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34391'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[tiran]

Related to bpo-32947

Four ftplib tests are failing with OpenSSL 1.1.1-pre8 and TLS 1.3 enabled. All failing tests use a separate data connection and transfer data on the server. For store operations, the client never calls recv(). This breaks bidirectional shutdown. I assume there are session tickets stuck on the wire.

======================================================================
ERROR: test_storbinary (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 591, in test_storbinary
    self.client.storbinary('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storbinary_rest (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 603, in test_storbinary_rest
    self.client.storbinary('stor', f, rest=r)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 514, in storbinary
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_storlines (test.test_ftplib.TestTLS_FTPClassMixin)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 608, in test_storlines
    self.client.storlines('stor', f)
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 545, in storlines
    conn.unwrap()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1091, in unwrap
    s = self._sslobj.shutdown()
OSError: [Errno 0] Error

======================================================================
ERROR: test_data_connection (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ftplib.py", line 889, in test_data_connection
    self.assertEqual(self.client.voidresp(), "226 transfer complete")
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 251, in voidresp
    resp = self.getresp()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 236, in getresp
    resp = self.getmultiline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 222, in getmultiline
    line = self.getline()
  File "/home/heimes/dev/python/cpython/Lib/ftplib.py", line 204, in getline
    line = self.file.readline(self.maxline + 1)
  File "/home/heimes/dev/python/cpython/Lib/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

---

Ran 88 tests in 9.402s

FAILED (errors=4, skipped=1)

[tiran]

The "OSError: [Errno 0] Error" failures are caused by session ticket handling in two way shutdown. I reported the issue a while ago openssl/openssl#6262 and it's fixed in OpenSSL git master (to be released as 1.1.1-pre9).

The error in test_data_connection is actually "[SSL] shutdown while in init". The dummy server code is failing, because the client starts a two way shutdown before the full handshake has been established. A simple recv() call is good enough to finalize the handshake.

[tiran]

New changeset 1590c39 by Christian Heimes in branch 'master':
bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787)
1590c39

[tiran]

New changeset 870247a by Christian Heimes (Miss Islington (bot)) in branch '3.7':
[3.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (GH-8789)
870247a

[tiran]

New changeset cabe916 by Christian Heimes (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (bpo-8790)
cabe916

[tiran]

New changeset 2ec530c by Christian Heimes in branch '2.7':
[2.7] bpo-34391: Fix ftplib test for TLS 1.3 (GH-8787) (GH-8791)
2ec530c

[terryjreedy]

Is there more to do on this? Tests pass on Windows with 1 skip, but I don't know if the failure condition applies here.
test_check_hostname (test.test_ftplib.TestTLS_FTPClass) ... skipped 'FIXME: bpo-32706'

[tiran]

I don't think there is anything left to do.