bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2.#11017

serhiy-storchaka · 2018-12-07T11:10:19Z

encoding='latin1' should be used for successful decoding.

(This is a copy of #794).

https://bugs.python.org/issue22005

…Python 2. errors='surrogateescape' should be used for successful decoding.

miss-islington · 2018-12-07T11:42:13Z

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7.
🐍🍒⛏🤖

miss-islington · 2018-12-07T11:42:22Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8452ca15f41061c8a6297d7956df22ab476d4df4 3.7

miss-islington · 2018-12-07T11:42:28Z

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8452ca15f41061c8a6297d7956df22ab476d4df4 3.6

…ed by Python 2. (pythonGH-11017) encoding='latin1' should be used for successful decoding.. (cherry picked from commit 8452ca1) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-bot · 2018-12-07T12:16:22Z

GH-11022 is a backport of this pull request to the 3.7 branch.

…ed by Python 2. (GH-11017) (GH-11022) encoding='latin1' should be used for successful decoding. (cherry picked from commit 8452ca1)

…ed by Python 2. (pythonGH-11017) (pythonGH-11022) encoding='latin1' should be used for successful decoding. (cherry picked from commit 8452ca1) (cherry picked from commit 0d5730e) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-bot · 2018-12-07T12:56:51Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 3.7 has failed when building commit 0d5730e.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/122/builds/876) and take a look at the build logs.
Check if the failure is related to this commit (0d5730e) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/122/builds/876

Click to see traceback logs

fatal: unable to access 'https://github.com/python/cpython.git/': Unknown SSL protocol error in connection to github.com:443 

make: *** No rule to make target 'distclean'.  Stop.

…Python 2. (GH-11017) (GH-11022) (GH-11024) encoding='latin1' should be used for successful decoding. (cherry picked from commit 8452ca1) (cherry picked from commit 0d5730e)

bedevere-bot · 2018-12-07T14:09:47Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL 3.6 has failed when building commit 19f6e83.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/29/builds/758) and take a look at the build logs.
Check if the failure is related to this commit (19f6e83) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/29/builds/758

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_robotparser.py", line 355, in test_read_404
    self.assertTrue(parser.allow_all)
AssertionError: False is not true

----------------------------------------------------------------------

Ran 40 tests in 5.037s

FAILED (failures=1)


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1605, in test_networked_good_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_noverification (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1577, in test_networked_noverification
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_trusted_by_default_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/support/__init__.py", line 767, in dec
    f(*args, **kwargs)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1589, in test_networked_trusted_by_default_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
FAIL: test_networked (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1567, in test_networked
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

======================================================================
FAIL: test_networked_bad_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1623, in test_networked_bad_cert
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

----------------------------------------------------------------------

Ran 103 tests in 5.298s

FAILED (failures=2, errors=3)


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1605, in test_networked_good_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_noverification (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1577, in test_networked_noverification
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_trusted_by_default_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/support/__init__.py", line 767, in dec
    f(*args, **kwargs)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1589, in test_networked_trusted_by_default_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
FAIL: test_networked (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1567, in test_networked
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

======================================================================
FAIL: test_networked_bad_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1623, in test_networked_bad_cert
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

----------------------------------------------------------------------

Ran 103 tests in 0.231s

FAILED (failures=2, errors=3)

serhiy-storchaka added 13 commits March 23, 2017 23:37

bpo-22005: Fixed unpickling instances of datetime classes pickled by …

caf031e

…Python 2. errors='surrogateescape' should be used for successful decoding.

Merge branch 'master' into datetime-compat-unpickle

6bd9b2f

Merge branch 'master' into datetime-compat-unpickle

531c5f6

Update datetime.py

cb55d65

Update _datetimemodule.c

f4c80ff

Merge branch 'master' into datetime-compat-unpickle

447f137

Move the NEWS entry to NEWS.d/.

985c923

Merge branch 'master' into datetime-compat-unpickle

66bf2c2

Merge branch 'master' into datetime-compat-unpickle

46b28a0

Use encoding='latin1' instead of errors='surrogateescape'.

7d44e3a

Merge branch 'master' into datetime-compat-unpickle

1f45085

Merge branch 'master' into datetime-compat-unpickle

4e890c0

Merge branch 'master' into datetime-compat-unpickle

93b65b0

serhiy-storchaka added type-bug An unexpected behavior, bug, or error needs backport to 3.6 labels Dec 7, 2018

the-knights-who-say-ni added the CLA signed label Dec 7, 2018

bedevere-bot added the awaiting merge label Dec 7, 2018

serhiy-storchaka merged commit 8452ca1 into python:master Dec 7, 2018

bedevere-bot removed the awaiting merge label Dec 7, 2018

miss-islington assigned serhiy-storchaka Dec 7, 2018

serhiy-storchaka mentioned this pull request Dec 7, 2018

bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2. #794

Closed

serhiy-storchaka deleted the datetime-compat-unpickle2 branch December 7, 2018 11:43

bedevere-bot removed the needs backport to 3.7 label Dec 7, 2018

serhiy-storchaka removed the needs backport to 3.6 label Dec 29, 2020

serhiy-storchaka removed their assignment Dec 29, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2.#11017