Type confusion in json encoding

[pkt]

BPO	24683
Nosy	@rhettinger, @ronaldoussoren, @vstinner, @serhiy-storchaka
Files	json_markers.py json_make_encoder_typecheck.patch eip.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-07-26.06:10:26.343>
created_at = <Date 2015-07-22.07:01:58.710>
labels = ['extension-modules', 'type-crash']
title = 'Type confusion in json encoding'
updated_at = <Date 2016-02-06.07:10:47.147>
user = 'https://bugs.python.org/pkt'

bugs.python.org fields:

activity = <Date 2016-02-06.07:10:47.147>
actor = 'pkt'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-07-26.06:10:26.343>
closer = 'serhiy.storchaka'
components = ['Extension Modules']
creation = <Date 2015-07-22.07:01:58.710>
creator = 'pkt'
dependencies = []
files = ['39975', '40025', '41719']
hgrepos = []
issue_num = 24683
keywords = ['patch']
message_count = 18.0
messages = ['247093', '247095', '247100', '247111', '247406', '247408', '247410', '247412', '247413', '247454', '247455', '247456', '258950', '258951', '259679', '259681', '259684', '259719']
nosy_count = 7.0
nosy_names = ['rhettinger', 'ronaldoussoren', 'vstinner', 'Arfrever', 'python-dev', 'serhiy.storchaka', 'pkt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue24683'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[pkt]

on-35dm-i386-linux-gnu.so`encoder_listencode_list(s=0xb6f90394, acc=0xbfc42c28, seq=0xb6f2361c, indent_level=1) + 655 at _json.c:1800

frame #2: 0xb6e4366d _json.cpython-35dm-i386-linux-gnu.so`encoder_listencode_obj(s=0xb6f90394, acc=0xbfc42c28, obj=0xb6f2361c, indent_level=1) + 733 at _json.c:1554

frame #3: 0xb6e3fc4f _json.cpython-35dm-i386-linux-gnu.so`encoder_call(self=0xb6f90394, args=0xb7049304, kwds=0x00000000) + 319 at _json.c:1386

frame #4: 0x080c5758 python`PyObject_Call(func=0xb6f90394, arg=0xb7049304, kw=0x00000000) + 264 at abstract.c:2149

This is a type confusion bug. encoder->markers can be initialized to an

arbitrary object (string in this POC). PyDict_Contains trusts the caller that

"op" is a dictionary without checking. Some callers can't be trusted :)

[vstinner]

I don't understand the issue. Can you elaborate?

What is your code? What is the current result? What is the expected result? What is your platform? What is your Python version? etc.

[pkt]

Sorry, I uploaded a test case.

[ronaldoussoren]

In encoder_init (the __init__ for _json.Encoder) s->marker is set to an argument of __init__, without any kind of type check, it can therefore be an arbitrary object.

encoder_listencode_obj (and other functions) then use s->markers with the concrete API for dicts (such as PyDict_Contains).

PyDict_Contains does not perform a type check, but casts its first argument to a PyDictObject and access fields. That causes problems when the marker isn't actually a dict.

I don't know the module good enough to be 100% sure about a fix, but I think it would be best to add a type check to encoder_init.

BTW. As far as I know _json.make_encoder is a private API and shouldn't be used directly, when you use the public API the argument will always be a dict.

[serhiy-storchaka]

There is similar issue with key_separator and item_separator in 3.x. They are used with _PyAccu_Accumulate that performs a type check only in assert().

Here is a patch.

[rhettinger]

Patch LGTM.

[python-dev]

New changeset b3d0bf112f70 by Serhiy Storchaka in branch '3.4':
Issue bpo-24683: Fixed crashes in _json functions called with arguments of
https://hg.python.org/cpython/rev/b3d0bf112f70

New changeset ef4d09399b99 by Serhiy Storchaka in branch '3.5':
Issue bpo-24683: Fixed crashes in _json functions called with arguments of
https://hg.python.org/cpython/rev/ef4d09399b99

New changeset 7de4abf4eed2 by Serhiy Storchaka in branch 'default':
Issue bpo-24683: Fixed crashes in _json functions called with arguments of
https://hg.python.org/cpython/rev/7de4abf4eed2

[python-dev]

New changeset 0a1266ef1b5d by Serhiy Storchaka in branch '2.7':
Issue bpo-24683: Fixed a crash in _json.make_encoder() called with non-dict 1st argument.
https://hg.python.org/cpython/rev/0a1266ef1b5d

[serhiy-storchaka]

Thank you for your report paul. Thanks for review Raymond.

[pkt]

resolution: not a bug

^ because of private API?