ftplib, urllib2, poplib, httplib, urllib2_localnet use ssl.PROTOCOL_TLSv1 unconditionally

[doko42]

BPO	31518
Nosy	@doko42, @tiran
PRs	bpo-31518: Change TLS protocol for Debian #3660 bpo-31518: Change TLS protocol for Debian #3661

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2018-02-24.23:46:23.101>
created_at = <Date 2017-09-19.13:35:27.367>
labels = ['expert-SSL', 'type-bug', '3.7']
title = 'ftplib, urllib2, poplib, httplib, urllib2_localnet use ssl.PROTOCOL_TLSv1 unconditionally'
updated_at = <Date 2018-02-24.23:46:23.101>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2018-02-24.23:46:23.101>
actor = 'christian.heimes'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2018-02-24.23:46:23.101>
closer = 'christian.heimes'
components = ['SSL']
creation = <Date 2017-09-19.13:35:27.367>
creator = 'doko'
dependencies = []
files = []
hgrepos = []
issue_num = 31518
keywords = ['patch']
message_count = 8.0
messages = ['302524', '302526', '302527', '302531', '302558', '302562', '312766', '312767']
nosy_count = 2.0
nosy_names = ['doko', 'christian.heimes']
pr_nums = ['3660', '3661']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31518'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[doko42]

Debian's OpenSSL now disables TLS 1.0 and 1.1, letting some of the python tests fail. Please make them use a newer protocol version, or make the use of the older versions conditional.

[doko42]

failing tests:

https://ci.debian.net/data/packages/unstable/amd64/p/python3.6/latest-autopkgtest/log.gz

[tiran]

3.7 does no longer use PROTOCOL_TLSv1 except for test_ssl.py, see bpo-31346

By the way Debian Sid/Buster broke support for SSL and TLS < 1.2. see bpo-31453. There is no way to enable the protocols from Python.

[tiran]

Two tests are failing in 3.7 branch:

======================================================================
ERROR: test_PROTOCOL_TLS (test.test_ssl.ThreadedTests)
Connecting to an SSLv23 server with various client options
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2660, in test_PROTOCOL_TLS
    try_protocol_combo(ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLSv1, 'TLSv1')
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2343, in try_protocol_combo
    chatty=False, connectionchatty=False)
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2268, in server_params_test
    s.connect((HOST, server.port))
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1108, in connect
    self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1099, in _real_connect
    self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1076, in do_handshake
    self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 697, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:864)

======================================================================
ERROR: test_protocol_tlsv1_1 (test.test_ssl.ThreadedTests)
Connecting to a TLSv1.1 server with various client options.
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2734, in test_protocol_tlsv1_1
    try_protocol_combo(ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLSv1_1, 'TLSv1.1')
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2343, in try_protocol_combo
    chatty=False, connectionchatty=False)
  File "/home/heimes/dev/python/cpython/Lib/test/test_ssl.py", line 2268, in server_params_test
    s.connect((HOST, server.port))
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1108, in connect
    self._real_connect(addr, False)
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1099, in _real_connect
    self.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 1076, in do_handshake
    self._sslobj.do_handshake()
  File "/home/heimes/dev/python/cpython/Lib/ssl.py", line 697, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:864)

---

Matthias,
is there any way to detect Debian's modifications of OpenSSL from header files or with an API call? Otherwise we have no way to reliable detect and correctly skip the test. At the moment there is no way to retrieve the minimum protocol from OpenSSL SSL_CTX. I landed an OpenSSL patch just a couple of days ago to add SSL_CTX_get_min_proto_version(), openssl/openssl#4364

[doko42]

Christian, I assume you'd like to see a test which can be done at *runtime*, not *buildtime*. Assuming you have that openssl upstream patch available in your build dependency, would that help with the detection? If yes, I'll talk to Debian's and Ubuntu's openssl maintainers to backport it, so the _ssl module could use it depending on a configure check.

[tiran]

PR 3660 and PR 3661 address most of the failing tests. The two failures in msg302531 are discussed in issue bpo-31453.

[tiran]

New changeset aab2258 by Christian Heimes in branch '3.6':
bpo-31518: Change TLS protocol for Debian (bpo-3660)
aab2258

[tiran]

New changeset 8d4d173 by Christian Heimes in branch '2.7':
bpo-31518: Change TLS protocol for Debian (bpo-3661)
8d4d173