Double deallocation on iterator exhausting

[serhiy-storchaka]

BPO	26494
Nosy	@rhettinger, @serhiy-storchaka
Files	free_after_iterating.patch free_after_iterating_2.patch free_after_iterating_3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-03-31.12:17:47.383>
created_at = <Date 2016-03-06.18:26:04.911>
labels = ['interpreter-core', 'type-crash']
title = 'Double deallocation on iterator exhausting'
updated_at = <Date 2016-03-31.12:17:47.382>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-03-31.12:17:47.382>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-03-31.12:17:47.383>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-03-06.18:26:04.911>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42080', '42081', '42092']
hgrepos = []
issue_num = 26494
keywords = ['patch']
message_count = 8.0
messages = ['261263', '261270', '261343', '261348', '261420', '261426', '261734', '262675']
nosy_count = 4.0
nosy_names = ['rhettinger', 'python-dev', 'serhiy.storchaka', 'scorp']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue26494'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

[serhiy-storchaka]

Following example causes double deallocation of a sequence and crashing.

class A(list):
    def __del__(self):
        next(it)

it = iter(A())
next(it)

The same is for subclass of tuple, str, bytes and bytearray.

Proposed patch fixes this issue.

[serhiy-storchaka]

Updated patch fixes also set, dict and os.scandir() iterator.

May be sqlite3 cursor needs a fix, but it is too complicated.

[serhiy-storchaka]

TODO: After resolving bpo-26492 add the test for array.

[serhiy-storchaka]

Fixed test for OrderedDict. Tests for list and tuple are moved up to seq_tests.py.

[scorp]

Hi Serhiy,

I tried the short example you gave, but it doesn't crash. I'm getting:

Exception ignored in: <bound method A.__del__ of []>
Traceback (most recent call last):
  File "./test.py", line 5, in __del__
    next(it)
StopIteration
Exception ignored in: <bound method A.__del__ of []>
Traceback (most recent call last):
  File "./test.py", line 5, in __del__
    next(it)
StopIteration
Traceback (most recent call last):
  File "./test.py", line 8, in <module>
    next(it)
StopIteration

Am I missing something?

[serhiy-storchaka]

It crashes in debug build.

[serhiy-storchaka]

Could anyone please look at the patch? I'm not sure about organizing tests.

[python-dev]

New changeset 905b5944119c by Serhiy Storchaka in branch '3.5':
Issue bpo-26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/905b5944119c

New changeset 73ce47d4a7b2 by Serhiy Storchaka in branch 'default':
Issue bpo-26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/73ce47d4a7b2

New changeset cff06d875678 by Serhiy Storchaka in branch '2.7':
Issue bpo-26494: Fixed crash on iterating exhausting iterators.
https://hg.python.org/cpython/rev/cff06d875678