integer overflow in the _csv module's join_append_data function

[benjaminp]

BPO	27758
Nosy	@benjaminp

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-08-14.00:22:42.353>
created_at = <Date 2016-08-14.00:16:14.509>
labels = ['type-security', 'library']
title = "integer overflow in the _csv module's join_append_data function"
updated_at = <Date 2016-08-14.09:43:36.842>
user = 'https://github.com/benjaminp'

bugs.python.org fields:

activity = <Date 2016-08-14.09:43:36.842>
actor = 'tehybel'
assignee = 'none'
closed = True
closed_date = <Date 2016-08-14.00:22:42.353>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2016-08-14.00:16:14.509>
creator = 'benjamin.peterson'
dependencies = []
files = []
hgrepos = []
issue_num = 27758
keywords = []
message_count = 3.0
messages = ['272624', '272625', '272660']
nosy_count = 3.0
nosy_names = ['benjamin.peterson', 'python-dev', 'tehybel']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue27758'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[benjaminp]

Thomas E Hybel on PSRT reports:

This vulnerability is an integer overflow leading to a heap buffer overflow. I
have attached a proof-of-concept script below.

The vulnerability resides in the Modules/_csv.c file, in the join_append and
join_append_data functions.

join_append initially calls join_append_data with copy_phase=0 to compute the
new length of its internal "rec" buffer. Then it grows the buffer. Finally it
calls join_append_data with copy_phase=1 to perform the actual writing.

The root issue is that join_append_data does not check for overflow when
computing the field rec_len which it returns. By having join_append_data called
on a few fields of appropriate length, we can make rec_len roll around and
become a small integer.

Note that there is already a check in join_append for whether (rec_len < 0). But
this check is insufficient as we can cause rec_len to grow sufficiently in a
single call to never let join_append see a negative size.

After the overflow happens, rec_len is a small integer, and thus when
join_append calls join_check_rec_size to potentially grow the rec buffer, no
enlargement happens. After this, join_append_data is called again, now with
copy_phase=1, and with a giant field_len.

Thus join_append_data writes the remaining data out-of-bounds of the self->rec
buffer which is located on the heap. Such a complete heap corruption should
definitely be exploitable to gain remote code execution.

Further details:

Tested version: Python-3.5.2, 32 bits

Proof-of-concept reproducer script (32-bits only):

--- begin script ---

import _csv

class MockFile:
    def write(self, _):
        pass

writer = _csv.writer(MockFile())
writer.writerow(["A"*0x10000, '"'*0x7fffff00]) 

--- end script ---

Python (configured with --with-pydebug) segfaults when the script is run. A
backtrace can be seen below. Note that the script only crashes on 32-bit
versions of Python. That's because the rec_len variable is an ssize_t, which is
4 bytes wide on 32-bit architectures, but 8 bytes wide on 64-bit arches.

(gdb) r
Starting program: /home/ubuntu32/python3/Python-3.5.2/python ../poc1.py
...
Program received signal SIGSEGV, Segmentation fault.
PyType_IsSubtype (a=0x0, b=b@entry=0x82d9aa0 <PyModule_Type>) at Objects/typeobject.c:1343
1343 mro = a->tp_mro;
(gdb) bt
#0 PyType_IsSubtype (a=0x0, b=b@entry=0x82d9aa0 <PyModule_Type>) at Objects/typeobject.c:1343
#1 0x080e29d9 in PyModule_GetState (m=0xb7c377f4) at Objects/moduleobject.c:532
#2 0xb7fd1a33 in join_append_data (self=self@entry=0xb7c2ffac, field_kind=field_kind@entry=0x1, field_data=field_data@entry=0x37c2f038,
field_len=field_len@entry=0x7fffff00, quoted=quoted@entry=0xbffff710, copy_phase=copy_phase@entry=0x1)
at /home/ubuntu32/python3/Python-3.5.2/Modules/_csv.c:1060
#3 0xb7fd1d6e in join_append (self=self@entry=0xb7c2ffac, field=field@entry=0x37c2f018, quoted=0x1, quoted@entry=0x0)
at /home/ubuntu32/python3/Python-3.5.2/Modules/_csv.c:1138
...

[python-dev]

New changeset fdae903db33a by Benjamin Peterson in branch '2.7':
check for overflow in join_append_data (closes bpo-27758)
https://hg.python.org/cpython/rev/fdae903db33a

New changeset afa356402217 by Benjamin Peterson in branch '3.3':
check for overflow in join_append_data (closes bpo-27758)
https://hg.python.org/cpython/rev/afa356402217

New changeset 10b89df93c58 by Benjamin Peterson in branch '3.4':
merge 3.3 (bpo-27758)
https://hg.python.org/cpython/rev/10b89df93c58

New changeset 55e8d3e542bd by Benjamin Peterson in branch '3.5':
merge 3.4 (closes bpo-27758)
https://hg.python.org/cpython/rev/55e8d3e542bd

New changeset 609b554dd4a2 by Benjamin Peterson in branch 'default':
merge 3.5 (closes bpo-27758)
https://hg.python.org/cpython/rev/609b554dd4a2

[tehybel]

Thanks for fixing this. I looked at the patch and it seems correct.