frame.f_locals causes segfault on Python >=3.4.1

[msmhrt]

BPO	21897
Nosy	@pitrou, @benjaminp, @skrah
Files	f_locals_clear.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-07-05.00:31:25.470>
created_at = <Date 2014-07-01.13:40:08.592>
labels = ['interpreter-core', 'type-crash']
title = 'frame.f_locals causes segfault on Python >=3.4.1'
updated_at = <Date 2014-07-05.00:31:25.468>
user = 'https://bugs.python.org/msmhrt'

bugs.python.org fields:

activity = <Date 2014-07-05.00:31:25.468>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2014-07-05.00:31:25.470>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2014-07-01.13:40:08.592>
creator = 'msmhrt'
dependencies = []
files = ['35834']
hgrepos = []
issue_num = 21897
keywords = ['patch']
message_count = 7.0
messages = ['222029', '222045', '222061', '222141', '222142', '222324', '222325']
nosy_count = 5.0
nosy_names = ['pitrou', 'benjamin.peterson', 'skrah', 'python-dev', 'msmhrt']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue21897'
versions = ['Python 3.4', 'Python 3.5']

[msmhrt]

It seems that frame.f_locals causes segfault on Python >=3.4.1

$ uname -a
Linux ashrose 3.2.0-61-generic #93-Ubuntu SMP Fri May 2 21:31:50 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
$ ls
test1.py
$ cat test1.py
import unittest

class TestCallable(unittest.TestCase):
    def test_callable(self):
        try:
            with self.assertRaises(IndexError):
                def raise_error(): raise TypeError
                (lambda: raise_error())()
        except TypeError as exception:
            exception.__traceback__.tb_next.tb_frame.f_locals
$ python3.4.1 -m unittest
Segmentation fault
$ python3.4.0 -m unittest
.

---

Ran 1 test in 0.001s

OK
$ python3.3.5 -m unittest
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
$ gdb /home/python/local/3.4.1mgs/bin/python3.4.1

...

(gdb) run -m unittest
Starting program: /home/python/local/3.4.1mgs/bin/python3.4.1 -m unittest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
map_to_dict (deref=1, values=0x7ffff5e97510, dict=0x7ffff5e8f588, nmap=<optimized out>, map=0x7ffff60e0278)
at Objects/frameobject.c:791
791 value = PyCell_GET(value);
(gdb) bt
#0 map_to_dict (deref=1, values=0x7ffff5e97510, dict=0x7ffff5e8f588, nmap=<optimized out>,
map=0x7ffff60e0278) at Objects/frameobject.c:791
#1 PyFrame_FastToLocalsWithError (f=0x7ffff5e97398) at Objects/frameobject.c:915
#2 0x00007ffff79beaa9 in frame_getlocals (f=0x7ffff5e97398, closure=<optimized out>)
at Objects/frameobject.c:24
#3 0x00007ffff79deff3 in _PyObject_GenericGetAttrWithDict (obj=0x7ffff5e97398, name=0x7ffff7fcdbb0, dict=0x0)
at Objects/object.c:1043
#4 0x00007ffff7a57891 in PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>)
at Python/ceval.c:2411
#5 0x00007ffff7a5df3c in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>,
locals=<optimized out>, args=<optimized out>, argcount=1, kws=0x790f48, kwcount=0, defs=0x0, defcount=0,
kwdefs=0x0, closure=0x0) at Python/ceval.c:3578
#6 0x00007ffff7a5ca61 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>,
pp_stack=0x7fffffffbbf0, func=0x7ffff6115158) at Python/ceval.c:4334
#7 call_function (oparg=<optimized out>, pp_stack=0x7fffffffbbf0) at Python/ceval.c:4252
#8 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2829
#9 0x00007ffff7a5df3c in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>,
locals=<optimized out>, args=<optimized out>, argcount=2, kws=0x7ffff7fad060, kwcount=0,
defs=0x7ffff617cd48, defcount=1, kwdefs=0x0, closure=0x0) at Python/ceval.c:3578
#10 0x00007ffff79bf28c in function_call (func=0x7ffff6192400, arg=0x7ffff6101c08, kw=0x7ffff5e8f248)
at Objects/funcobject.c:632
#11 0x00007ffff799300e in PyObject_Call (func=0x7ffff6192400, arg=<optimized out>, kw=<optimized out>)
at Objects/abstract.c:2067
#12 0x00007ffff7a56871 in ext_do_call (nk=0, na=1, flags=<optimized out>, pp_stack=0x7fffffffbec0,
func=0x7ffff6192400) at Python/ceval.c:4551
#13 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at Python/ceval.c:2869
---Type <return> to continue, or q <return> to quit---

[skrah]

6ab3193e890e exposes the issue.

[pitrou]

Following patch seems to fix it, but I have to cook a proper test:

diff --git a/Objects/frameobject.c b/Objects/frameobject.c
--- a/Objects/frameobject.c
+++ b/Objects/frameobject.c
@@ -786,7 +786,7 @@ map_to_dict(PyObject *map, Py_ssize_t nm
         PyObject *key = PyTuple_GET_ITEM(map, j);
         PyObject *value = values[j];
         assert(PyUnicode_Check(key));
-        if (deref) {
+        if (deref && value != NULL) {
             assert(PyCell_Check(value));
             value = PyCell_GET(value);
         }

[msmhrt]

Thanks, pitrou.

Your patch seems ok to me.

$ python3.4.1_with_patch -m unittest
.

---

Ran 1 test in 0.001s

It seems that this issue is same as https://bitbucket.org/hpk42/pytest/issue/528/test-causes-segfault .

[pitrou]

Here is a patch with tests.

[python-dev]

New changeset 758468cdf72c by Antoine Pitrou in branch '3.4':
Issue bpo-21897: Fix a crash with the f_locals attribute with closure variables when frame.clear() has been called.
http://hg.python.org/cpython/rev/758468cdf72c

New changeset bd6515070f9c by Antoine Pitrou in branch 'default':
Issue bpo-21897: Fix a crash with the f_locals attribute with closure variables when frame.clear() has been called.
http://hg.python.org/cpython/rev/bd6515070f9c

[pitrou]

Patch committed. Thank you for reporting this issue!