Repository navigation
Calling "functions" used to implement generators/comps easily cause crash #81137
Description
| BPO | 36956 |
|---|---|
| Nosy | @rhettinger, @mr-nfamous, @websurfer5, @iritkatriel |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-05-18.13:53:21.143>
labels = ['3.10', '3.9', 'type-crash', '3.11']
title = 'Calling "functions" used to implement generators/comps easily cause crash'
updated_at = <Date 2021-09-19.21:54:42.882>
user = 'https://github.com/mr-nfamous'bugs.python.org fields:
activity = <Date 2021-09-19.21:54:42.882>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2019-05-18.13:53:21.143>
creator = 'bup'
dependencies = []
files = []
hgrepos = []
issue_num = 36956
keywords = []
message_count = 5.0
messages = ['342797', '346807', '346808', '346809', '402171']
nosy_count = 4.0
nosy_names = ['rhettinger', 'bup', 'Jeffrey.Kintscher', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue36956'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
Activity
mr-nfamous commentedon May 18, 2019
As far as I know, generators, set comprehensions, list comprehensions, and dict comprehensions, (along with their asynchronous variants) are implemented by first calling the GET_(A)ITER opcode and then building and calling a function that acepts the resulting iterator as its sole argument.
Assigning the code object used to make that function (or using it in the types.FunctionType constructor) and then calling it with a non-iterator argument will obviously cause a crash since the FOR_ITER opcode rightly expects that it will never have to deal with non-iterators and calls tp_iternext without checking if it exists.
The 4-liner demonstrates the crash:
websurfer5 commentedon Jun 28, 2019
What is the preferred behavior? Raise an exception of some kind?
websurfer5 commentedon Jun 28, 2019
Specifically, the crash occurs in Python/ceval.c function _PyEval_EvalFrameDefault() in the TARGET(FOR_ITER) case at line 3198 (master branch):
It segfaults because tp_iternext is NULL.
rhettinger commentedon Jun 28, 2019
ISTM this report is close to that covered by the traditional stance that word code hacks aren't protected from crashes because trying to do can be complicated, can be expensive, and isn't always possible.
I'm not really sure where this falls on the grey scale with "normal, everyday python shouldn't crash" on one end and "with ctypes or code hacks, all bets are off" on the other end.
Offhand, the posted snippet seems closer to "messing with the internals" but perhaps small cheap check can be added when the __code__ variable is updated.
iritkatriel commentedon Sep 19, 2021
Reproduced on 3.11.
pythongh-81137: deprecate assignment of code object to a function of …
gh-81137: deprecate assignment of code object to a function of a mism…
pythongh-81137: deprecate assignment of code object to a function of …
pythongh-81137: deprecate assignment of code object to a function of …
pythongh-81137: deprecate assignment of code object to a function of …
Fix DeprecationWarning while spying on generator functions.