pickle fails with SystemError

[alex]

BPO	23914
Nosy	@alex, @bitdancer, @zware, @serhiy-storchaka
Files	unpickle_bad_stack.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-11-23.15:54:09.017>
created_at = <Date 2015-04-11.12:52:42.679>
labels = ['extension-modules', 'type-crash']
title = 'pickle fails with SystemError'
updated_at = <Date 2015-11-27.07:57:42.549>
user = 'https://github.com/alex'

bugs.python.org fields:

activity = <Date 2015-11-27.07:57:42.549>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-11-23.15:54:09.017>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2015-04-11.12:52:42.679>
creator = 'alex'
dependencies = []
files = ['41045']
hgrepos = []
issue_num = 23914
keywords = ['patch']
message_count = 9.0
messages = ['240475', '240476', '240477', '240482', '254672', '255154', '255452', '255453', '255454']
nosy_count = 5.0
nosy_names = ['alex', 'r.david.murray', 'python-dev', 'zach.ware', 'serhiy.storchaka']
pr_nums = []
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue23914'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

[alex]

>>> pickle.loads(b'(o.\x7f.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Objects/tupleobject.c:71: bad argument to internal function

(Or the equivalent using cPickle on Python 2)

Found using http://lcamtuf.coredump.cx/afl/

[serhiy-storchaka]

Warning: The pickle module is not intended to be secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.

pickle.loads(b'cos\nsystem\n(Vrm -rf /\ntR.')

[alex]

Yes, it can execute arbitrary code, but I think we should prefer raising "specific" error messages, instead of failing inside tuple details.

[bitdancer]

I dob't see a strong motivation to do that. What's the use case?

[serhiy-storchaka]

Here is a patch that makes broken OBJ opcode to raise UnpicklingError instead of SystemError, improves some UnpicklingError messages, and adds tests for unpickling broken data.

[python-dev]

New changeset bac3f63ea747 by Serhiy Storchaka in branch '3.4':
Issue bpo-23914: Fixed SystemError raised by unpickler on broken pickle data.
https://hg.python.org/cpython/rev/bac3f63ea747

New changeset 531e2674f003 by Serhiy Storchaka in branch '3.5':
Issue bpo-23914: Fixed SystemError raised by unpickler on broken pickle data.
https://hg.python.org/cpython/rev/531e2674f003

New changeset b08c3a733fda by Serhiy Storchaka in branch 'default':
Issue bpo-23914: Fixed SystemError raised by unpickler on broken pickle data.
https://hg.python.org/cpython/rev/b08c3a733fda

New changeset 686fa9439d38 by Serhiy Storchaka in branch '2.7':
Issue bpo-23914: Fixed SystemError raised by unpickler on broken pickle data.
https://hg.python.org/cpython/rev/686fa9439d38

[python-dev]

New changeset 322060740b64 by Zachary Ware in branch '2.7':
Issue bpo-23914: Fix test_xpickle with python 2.4 and 2.5
https://hg.python.org/cpython/rev/322060740b64

[zware]

This broke test_xpickle with python 2.4 and 2.5 due to 'b' prefixes and an import that's not actually used in the 2.7 patch; now fixed. The ware-gentoo-x86 buildslave now has pythons 2.4, 2.5, and 2.6 installed for test_xpickle to use.

[serhiy-storchaka]

Great! I afraid there were no buildbots with 2.4 and 2.5 for testing test_xpickle. Thank you Zachary!