test_ssl.test_get_server_certificate() should use PROTOCOL_SSLv23, not PROTOCOL_SSLv3

[vstinner]

BPO	20896
Nosy	@pitrou, @vstinner, @tiran, @koobs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-04-16.16:58:56.374>
created_at = <Date 2014-03-12.11:20:39.932>
labels = ['type-bug', 'library']
title = 'test_ssl.test_get_server_certificate() should use PROTOCOL_SSLv23, not PROTOCOL_SSLv3'
updated_at = <Date 2015-01-06.11:24:05.743>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2015-01-06.11:24:05.743>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2014-04-16.16:58:56.374>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2014-03-12.11:20:39.932>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 20896
keywords = []
message_count = 13.0
messages = ['213248', '213249', '213250', '213251', '213294', '213316', '213318', '213323', '213777', '213819', '216502', '216532', '233521']
nosy_count = 7.0
nosy_names = ['pitrou', 'vstinner', 'christian.heimes', 'python-dev', 'koobs', 'ddvento@ucar.edu', 'GreenKey']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20896'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.5']

[vstinner]

======================================================================
ERROR: test_get_server_certificate (test.test_ssl.NetworkedTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/haypo/prog/python/default/Lib/test/test_ssl.py", line 1373, in test_get_server_certificate
    _test_get_server_certificate('svn.python.org', 443, SVN_PYTHON_ORG_ROOT_CERT)
  File "/home/haypo/prog/python/default/Lib/test/test_ssl.py", line 1354, in _test_get_server_certificate
    pem = ssl.get_server_certificate((host, port))
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 902, in get_server_certificate
    with context.wrap_socket(sock) as sslsock:
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 344, in wrap_socket
    _context=self)
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 540, in __init__
    self.do_handshake()
  File "/home/haypo/prog/python/default/Lib/ssl.py", line 767, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:598)

---

Extract of the current CA cert of svn.python.org:
---

$ openssl x509 -in Lib/test/https_svn_python_org_root.pem -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 0 (0x0)
    Signature Algorithm: md5WithRSAEncryption
        Issuer: O=Root CA, OU=http://www.cacert.org, CN=CA Cert Signing Authority/emailAddress=support@cacert.org
        Validity
            Not Before: Mar 30 12:29:49 2003 GMT
            Not After : Mar 29 12:29:49 2033 GMT
        Subject: O=Root CA, OU=http://www.cacert.org, CN=CA Cert Signing Authority/emailAddress=support@cacert.org
...

---

Lib/test/https_svn_python_org_root.pem is identical to http://www.cacert.org/certs/root.crt: root certificate of cacert.org authority.

[vstinner]

Debug with OpenSSL command line:

$ openssl s_client -connect svn.python.org:443 -CAfile Lib/test/https_svn_python_org_root.pem 
CONNECTED(00000003)
depth=1 O = Root CA, OU = http://www.cacert.org, CN = CA Cert Signing Authority, emailAddress = support@cacert.org
verify return:1
depth=0 CN = svn.python.org
verify return:1

---

Certificate chain
0 s:/CN=svn.python.org
i:/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEzzCCAregAwIBAgIDDkGJMA0GCSqGSIb3DQEBBQUAMHkxEDAOBgNVBAoTB1Jv
b3QgQ0ExHjAcBgNVBAsTFWh0dHA6Ly93d3cuY2FjZXJ0Lm9yZzEiMCAGA1UEAxMZ
Q0EgQ2VydCBTaWduaW5nIEF1dGhvcml0eTEhMB8GCSqGSIb3DQEJARYSc3VwcG9y
dEBjYWNlcnQub3JnMB4XDTEzMTIyNDIwMjgzMloXDTE1MTIyNDIwMjgzMlowGTEX
MBUGA1UEAxMOc3ZuLnB5dGhvbi5vcmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDM6un3wTW9+HVJ7KC+/GwL0KAxehug0tw2YoSSX+TGxLyr9AUtBHQk
hCNWhRLewa0WMOY6hxIIQY1Hp6vreDiCbBehjVkEAydlKBzaAsgYCEbCC/ZaMzhv
aaFAiLVeaxAKJsBGUJNz5hGgzd67A6SGz+XK7qDWig4NR5eFrsr3DvjyEM7txMiG
gftGWLkadOuqUQsI20AykBGi+RxmrQIwqO2svGmje89DsWVILdP37PssM2zqRonh
4fUKooei3L43tXbTdHayXc9NtFS7q8T4eUlyWaD+BtP80QQOQFFvi+qZpme9bmYI
7YPX+e86lZtxAKM9nWrP93qc+2nS0MsHAgMBAAGjgb8wgbwwDAYDVR0TAQH/BAIw
ADAOBgNVHQ8BAf8EBAMCA6gwNAYDVR0lBC0wKwYIKwYBBQUHAwIGCCsGAQUFBwMB
BglghkgBhvhCBAEGCisGAQQBgjcKAwMwMwYIKwYBBQUHAQEEJzAlMCMGCCsGAQUF
BzABhhdodHRwOi8vb2NzcC5jYWNlcnQub3JnLzAxBgNVHR8EKjAoMCagJKAihiBo
dHRwOi8vY3JsLmNhY2VydC5vcmcvcmV2b2tlLmNybDANBgkqhkiG9w0BAQUFAAOC
AgEAvq2TlM5voqqwlfbEIwdNM3RlqHattH/h2Pqkr5FV6nynhcNyP9wBJyS5mdlt
+tj1Fy6oV+iN0s8VglrwhsmYN2pbJkCJZcNgrhFTiOvZ3HJ0jxkQ8TaROfcH1RnZ
q45DcbSnryiIDBQNYxmUf/bE6Ce+48fiOJpTRtrC8iWfz4J/JHYZ5FVKx+SsC67o
E1iCT2/r1PSvzTg/bohL1kX6Aj3H3UqlXDuDybyHRrTn9kUwCgXK2h4x/qae6xjo
Gy++gtzg4XlJEAh65znJ2RlAYi3lvhCls+viR5vSIXBEXmBPFNonGnPtGRyx6Tii
5ncSUP+bp6aaRZb0qEi0k1R5XYPt5Hm7T/h4IOAKlx7gPMMLOXqbrlWpjgwhfysI
f/KkgGBq7nTsC6WeaW/QUQtdXqWULrK+nINY+s/CRX5UxdAWpqMxqgNkP3zef9yO
etgfpR+6NArjiLNcfJO2yGZTcXn1H2gewTcxwmc+QVWday9HcS3paMpot8BwWUpI
LzOLGTeNH+rQCUAKJHAiK3Ogee+hka4icIN7cKOIIVfZ/XBj1Ex7zjc3j72axNba
S1buqvrm12YCE5+xAjYwU/Nrl4HNFKCoPW7qfEaEuwp49pieAIOnXa82rCh/UdAv
dSj5JpEvxjTuZdVfbk2VUc/z2OeLzFlrQRJsOt1MisY0Aoc=
-----END CERTIFICATE-----
subject=/CN=svn.python.org
issuer=/O=Root CA/OU=http://www.cacert.org/CN=CA Cert Signing Authority/emailAddress=support@cacert.org
---
No client certificate CA names sent
---
SSL handshake has read 2112 bytes and written 439 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : DHE-RSA-AES256-GCM-SHA384
Session-ID: A8A1C5EC36ACD8FF0120271C1F16BDE7720FD0DC69871D1BE394A22309C09FE5
Session-ID-ctx:
Master-Key: EF899D1961B522E380366F8832E7DF72AB56B9D76388B80A907637E2948D94514CADE5885CA3AF11B40F43E14F42ED92
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 7f 00 da de 6a a5 79 fa-9e 83 e8 20 1c 75 ef 54 ....j.y.... .u.T
0010 - 34 43 3a 0a 50 0c f7 00-31 79 02 38 9f 8e 49 d6 4C:.P...1y.8..I.
0020 - f1 25 57 c7 4f 97 f3 3a-a3 fa 8b 1b 8a 3b 5f e9 .%W.O..:.....;_.
0030 - 6b ba 89 e1 db ba a4 e2-1d 3c f6 e4 cf d6 2f 54 k........<..../T
0040 - 82 4f 24 1c 93 44 c3 ff-79 3d 67 34 27 48 34 db .O$..D..y=g4'H4.
0050 - 5b b4 a2 30 5b 16 e7 b5-ba ee 89 0a c5 89 a9 9a [..0[...........
0060 - fe 32 77 23 b3 b5 b4 fb-63 b5 87 d4 20 b2 18 7f .2w#....c... ...
0070 - 45 4e e6 f5 6f bd f4 24-80 b3 37 fd b5 83 2e 87 EN..o..$..7.....
0080 - a1 b2 bb 4f b0 e3 7a 28-26 4a 71 3a 92 5e d1 aa ...O..z(&Jq:.^..
0090 - be 77 67 79 ad ea d0 c8-d4 d2 8a 44 f0 f5 ec c5 .wgy.......D....
00a0 - 00 0b 5b 82 c1 51 45 ef-d0 6c fb 03 46 3f b1 e2 ..[..QE..l..F?..
00b0 - 54 f8 27 4b 8f a0 e0 2e-7b 4f d8 42 29 76 74 b3 T.'K....{O.B)vt.

Start Time: 1394623442
Timeout   : 300 (sec)
Verify return code: 0 (ok)

---

[vstinner]

Script to reproduce the issue:
---

import ssl
pem = ssl.get_server_certificate(('svn.python.org', 443), ca_certs="Lib/test/https_svn_python_org_root.pem")
print("PEM: %r" % pem)

---

It looks the handshake fails if like with PROTOCOL_SSLv3 which is the default protocol, but works with PROTOCOL_SSLv23.

_create_stdlib_context(), SSLContext and wrap_socket use PROTOCOL_SSLv23 which is said to be the "the most compatibility with other versions" protocol. Why get_server_certificate() uses PROTOCOL_SSLv3?

get_server_certificate() was added in 2007 by changeset 9041965a92f2 and it uses PROTOCOL_SSLv3 since this version.

"openssl s_client" says that the server speaks TLSv1.2 which is the most recent TLS version and probably the most secure. Is it possible somehow to try TLSv1.2, and then fallback to other versions if the latest version is not supported?

For the initial issue, it looks like a change at server side (svn.python.org), I don't think that ssl module, the unit test or the certificate of the authority changed recently. The python.org website has been changed recently.

[python-dev]

New changeset c13398566409 by Victor Stinner in branch 'default':
Issue bpo-20896: Workaround the bug temporarely to fix buildbots
http://hg.python.org/cpython/rev/c13398566409

[vstinner]

The bug is not fixed, I commited a workaround. Please repon it.

Le mercredi 12 mars 2014, Benjamin Peterson <report@bugs.python.org> a
écrit :

Changes by Benjamin Peterson <bp+pybugs@benjamin-peterson.org<javascript:;>
>:

----------
resolution: -> fixed
status: open -> closed

---

Python tracker <report@bugs.python.org <javascript:;>>
<http://bugs.python.org/issue20896\>

---

[pitrou]

Ok, apparently the change was backed out and it was not needed (I can't make the test fail here, either). It would have been better to state it on the issue, though :-)

[pitrou]

That said, I agree it would be better to use "SSLv23" in get_server_certificate().

[python-dev]

New changeset 23add5382fb3 by Benjamin Peterson in branch '3.1':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes bpo-20896)
http://hg.python.org/cpython/rev/23add5382fb3

New changeset 789ca594960f by Benjamin Peterson in branch '3.2':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes bpo-20896)
http://hg.python.org/cpython/rev/789ca594960f

New changeset de97d0334314 by Benjamin Peterson in branch '2.7':
use ssl.PROTOCOL_SSLv23 for maximum compatibility (closes bpo-20896)
http://hg.python.org/cpython/rev/de97d0334314

New changeset 12df02358137 by Benjamin Peterson in branch '3.3':
merge 3.2 (bpo-20896)
http://hg.python.org/cpython/rev/12df02358137

New changeset 0cba79667c7d by Benjamin Peterson in branch 'default':
merge 3.3 (bpo-20896)
http://hg.python.org/cpython/rev/0cba79667c7d

[vstinner]

Benjamin: Could you please mention your change in Misc/NEWS?

Is it ok to change that in Python 3.1 & 3.2? Should the change be mentionned in the doc (:versionchanged:)?