Python 3: Segfault instead of MemoryError when bytearray too big

[swanson]

BPO	22335
Nosy	@pitrou, @vstinner, @jwilk, @ezio-melotti, @serhiy-storchaka
Files	issue19087.patch ba_resize.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-11-02.17:43:23.082>
created_at = <Date 2014-09-04.10:38:57.101>
labels = ['interpreter-core', 'type-crash']
title = 'Python 3: Segfault instead of MemoryError when bytearray too big'
updated_at = <Date 2014-11-02.17:43:23.080>
user = 'https://bugs.python.org/swanson'

bugs.python.org fields:

activity = <Date 2014-11-02.17:43:23.080>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-11-02.17:43:23.082>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2014-09-04.10:38:57.101>
creator = 'swanson'
dependencies = []
files = ['36917', '37108']
hgrepos = []
issue_num = 22335
keywords = ['patch']
message_count = 10.0
messages = ['226356', '226358', '229314', '229322', '230496', '230497', '230499', '230505', '230509', '230510']
nosy_count = 8.0
nosy_names = ['pitrou', 'vstinner', 'jwilk', 'ezio.melotti', 'Arfrever', 'python-dev', 'serhiy.storchaka', 'swanson']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue22335'
versions = ['Python 3.4', 'Python 3.5']

[swanson]

On Python 3, but not Python 2, you crash with a Segmentation Fault instead of getting a MemoryError as expected. It seems to only be a problem with bytearray, not with other things like tuple:

$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:18) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> bytearray(0x7FFFFFFF)
Segmentation fault (core dumped)
$ 

compare to:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:38) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> bytearray(0x7FFFFFFF)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> 


$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:18) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> (0,)*0x7FFFFFFF
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>>

[serhiy-storchaka]

Looks as integer overflow introduced in bpo-19087.

+ if (size + logical_offset + 1 < alloc) {

[jwilk]

This patch should fix it.

[jwilk]

On a second thought, "logical_offset + 1" alone could overflow; and there are apparently other possible integer overflows in this function.

[serhiy-storchaka]

"logical_offset + 1" can't overflow because logical_offset is an offset in allocated array not counting final null byte.

[pitrou]

Note that there are two possible crashes in debug mode:

$ ./python -c "bytearray(2**31-1)"
Erreur de segmentation

$ ./python -c "bytearray(2**31-2)"
python: Objects/obmalloc.c:1179: _PyObject_Alloc: Assertion `nelem <= ((Py_ssize_t)(((size_t)-1)>>1)) / elsize' failed.
Abandon

[pitrou]

Here is a patch. It also fixes a bug in the debug allocators, which didn't properly check for Py_ssize_t overflow.

[serhiy-storchaka]

LGTM.

[python-dev]

New changeset 1590c594550e by Antoine Pitrou in branch '3.4':
Issue bpo-22335: Fix crash when trying to enlarge a bytearray to 0x7fffffff bytes on a 32-bit platform.
https://hg.python.org/cpython/rev/1590c594550e

New changeset f0b334ae95c9 by Antoine Pitrou in branch 'default':
Issue bpo-22335: Fix crash when trying to enlarge a bytearray to 0x7fffffff bytes on a 32-bit platform.
https://hg.python.org/cpython/rev/f0b334ae95c9

[pitrou]

Thank you. This is now pushed.