Incorrect assert in str_subtype_new

[KevinModzelewski]

BPO	24731
Nosy	@mdickinson, @serhiy-storchaka
Files	issue24731.patch: Patch for 3.6 issue24731-2.7.patch: Patch for 2.7

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-11-25.13:57:54.009>
created_at = <Date 2015-07-26.23:21:25.535>
labels = ['interpreter-core', 'type-crash']
title = 'Incorrect assert in str_subtype_new'
updated_at = <Date 2015-12-01.23:25:40.843>
user = 'https://bugs.python.org/KevinModzelewski'

bugs.python.org fields:

activity = <Date 2015-12-01.23:25:40.843>
actor = 'Kevin Modzelewski'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-11-25.13:57:54.009>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2015-07-26.23:21:25.535>
creator = 'Kevin Modzelewski'
dependencies = []
files = ['41126', '41127']
hgrepos = []
issue_num = 24731
keywords = ['patch']
message_count = 5.0
messages = ['247451', '255117', '255351', '255583', '255683']
nosy_count = 4.0
nosy_names = ['mark.dickinson', 'python-dev', 'serhiy.storchaka', 'Kevin Modzelewski']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue24731'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[KevinModzelewski]

(Using python 3 terminology) str_subtype_new is the function that creates instances of any subtypes of bytes (ie is called by bytes_new if the requested type is not PyBytes_Type -- looks like this function's name comes from python 2). Its approach is to create a bytes object using the same arguments, and then copy the resulting data into the subclass-instance's memory. It does

    tmp = bytes_new(&PyBytes_Type, args, kwds);
    [error checking]
    assert(PyBytes_CheckExact(tmp));

The problem is that bytes_new can return a subclass of bytes, if the argument provides a __bytes__ method that returns a bytes-subtype. For example

    class MyBytes(bytes):
        pass
    class C(object):
        def __bytes__(self):
            return MyBytes(b"hello world")
    MyBytes(C()) # fails the assert

This doesn't seem to cause any issues other than the failing assert in debug builds; it seems like the assert should just be relaxed from PyBytes_CheckExact to PyBytes_Check since that's enough to guarantee that the upcoming manipulation of the "tmp" variable is going to be valid. Also, this would match how unicode_subtype_new behaves.

This bug also applies to Python 2, since I think the relevant code is the same, though in that case it applies to str instead of bytes.

[serhiy-storchaka]

Thank you for your report Kevin and sorry for the delay. I confirm the bug and agree with your solution. The same bug exists for ints and floats.

Here are patches that fix crashes.

[python-dev]

New changeset d8e0b54ece62 by Serhiy Storchaka in branch '3.4':
Issue bpo-24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/d8e0b54ece62

New changeset 80efe5cc8934 by Serhiy Storchaka in branch '3.5':
Issue bpo-24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/80efe5cc8934

New changeset 1c4d256cc370 by Serhiy Storchaka in branch 'default':
Issue bpo-24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/1c4d256cc370

New changeset 37158c067b25 by Serhiy Storchaka in branch '2.7':
Issue bpo-24731: Fixed crash on converting objects with special methods
https://hg.python.org/cpython/rev/37158c067b25

[python-dev]

New changeset 2ea1a3bf448f by Serhiy Storchaka in branch '2.7':
Fixed Py3k warnings in tests for issue bpo-24731.
https://hg.python.org/cpython/rev/2ea1a3bf448f

[KevinModzelewski]

Awesome, thanks!