integer overflow in computing unicode's object representation

[pkt]

BPO	22520
Nosy	@vstinner, @benjaminp
Files	poc_repr_unicode.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-12-10.19:07:11.303>
created_at = <Date 2014-09-29.21:04:19.113>
labels = ['type-security']
title = "integer overflow in computing unicode's object representation"
updated_at = <Date 2014-12-10.19:07:11.302>
user = 'https://bugs.python.org/pkt'

bugs.python.org fields:

activity = <Date 2014-12-10.19:07:11.302>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2014-12-10.19:07:11.303>
closer = 'benjamin.peterson'
components = []
creation = <Date 2014-09-29.21:04:19.113>
creator = 'pkt'
dependencies = []
files = ['36756']
hgrepos = []
issue_num = 22520
keywords = []
message_count = 4.0
messages = ['227839', '227867', '227911', '232439']
nosy_count = 5.0
nosy_names = ['vstinner', 'benjamin.peterson', 'Arfrever', 'python-dev', 'pkt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue22520'
versions = ['Python 3.3', 'Python 3.4', 'Python 3.5']

[pkt]

# unicode_repr(PyObject *unicode)
# {
# ...
# 1 isize = PyUnicode_GET_LENGTH(unicode);
# idata = PyUnicode_DATA(unicode);
#
# /* Compute length of output, quote characters, and
# maximum character */
# osize = 0;
# ...
# for (i = 0; i < isize; i++) {
# Py_UCS4 ch = PyUnicode_READ(ikind, idata, i);
# switch (ch) {
# ...
# default:
# /* Fast-path ASCII */
# if (ch < ' ' || ch == 0x7f)
# 2 osize += 4; /* \xHH */
# ...
# }
# }
#
# ...
# 3 repr = PyUnicode_New(osize, max);
# ...
# for (i = 0, o = 1; i < isize; i++) {
# Py_UCS4 ch = PyUnicode_READ(ikind, idata, i);
# ...
# else {
# 4 PyUnicode_WRITE(okind, odata, o++, ch);
# }
# }
# }
# }
# /* Closing quote already added at the beginning */
# 5 assert(_PyUnicode_CheckConsistency(repr, 1));
# return repr;
# }
#
# 1. isize=2^30+1
# 2. osize=isize*4=4
# 3. allocated buffer is too small
# 4. heap overflow
# 5. this assert will likely fail, since there is a good chance the allocated
# buffer is just before the huge one, so the huge one will overwrite itself.

[python-dev]

New changeset 8ba7e5f43952 by Benjamin Peterson in branch '3.3':
prevent overflow in unicode_repr (closes bpo-22520)
https://hg.python.org/cpython/rev/8ba7e5f43952

New changeset 6f54dfa675eb by Benjamin Peterson in branch '3.4':
merge 3.3 (bpo-22520)
https://hg.python.org/cpython/rev/6f54dfa675eb

New changeset 245d9679cd5b by Benjamin Peterson in branch 'default':
merge 3.4 (bpo-22520)
https://hg.python.org/cpython/rev/245d9679cd5b

[vstinner]

It would be nice to add a bigmem test to check that repr('\x00'*(2**30+1)) doesn't crash anymore.

[benjaminp]

As Serhiy has noted on other bugs, the fact that the tests must be restricted to 32-bits limits their usefulness unfortunately.