Segmentation fault when create _tkinter objects

[serhiy-storchaka]

BPO	23815
Nosy	@loewis, @terryjreedy, @amauryfa, @pitrou, @ned-deily, @asvetlov, @serhiy-storchaka
Files	tp_new_workaround.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-05-08.18:29:33.282>
created_at = <Date 2015-03-30.19:56:54.900>
labels = ['extension-modules', 'expert-tkinter', 'type-crash']
title = 'Segmentation fault when create _tkinter objects'
updated_at = <Date 2016-05-08.18:29:33.280>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-05-08.18:29:33.280>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-05-08.18:29:33.282>
closer = 'serhiy.storchaka'
components = ['Extension Modules', 'Tkinter']
creation = <Date 2015-03-30.19:56:54.900>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42775']
hgrepos = []
issue_num = 23815
keywords = ['patch']
message_count = 11.0
messages = ['239636', '240034', '240275', '242678', '254681', '265094', '265096', '265102', '265106', '265150', '265153']
nosy_count = 9.0
nosy_names = ['loewis', 'terry.reedy', 'amaury.forgeotdarc', 'pitrou', 'ned.deily', 'asvetlov', 'python-dev', 'serhiy.storchaka', 'Robin.Schreiber']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue23815'
versions = ['Python 3.5', 'Python 3.6']

[serhiy-storchaka]

In 2.7 and 3.3:

>>> import _tkinter
>>> _tkinter.Tcl_Obj()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create '_tkinter.Tcl_Obj' instances
>>> _tkinter.TkttType()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create 'tktimertoken' instances

In 3.4+:

>>> import _tkinter
>>> _tkinter.Tcl_Obj()
Segmentation fault (core dumped)

And the same for _tkinter.TkttType.

Looks as this is a result of bpo-15721.

[terryjreedy]

With 3.4.3, I get the equivalent on Windows (a 'python has stopped working' box).

[amauryfa]

Probably a PyType_Ready() missing.

[serhiy-storchaka]

PyType_Ready() is called inside PyType_FromSpec().

[serhiy-storchaka]

Before bpo-15721 the tp_new was NULL, and this prevented from creating instances. After bpo-15721 the tp_new is inherited from object. An instance is created, but all internal pointers are NULLs. Following repr() dereferences NULL.

[serhiy-storchaka]

There is special exception -- tp_new is not inherited for static types whose base class is 'object' (see a comment in inherit_special() in Objects/typeobject.c#4569 for explanation). After converting them to heap types they became inherit tp_new from object.

Proposed patch adds workarounds for types in _tkinter and curses.panel modules.

[terryjreedy]

The news part of the patch did not apply -- best not to include such in posted patches.

After recompiliing 3.5 on Windows 10, the two tkinter test cases give the 3.3 TypeErrors instead of crashing. test_tcl also runs. curses does not run on Windows.

I do not know enough internals to really understand bpo-15721 or why you call the fix a workaround, but I do understand 'exception' (OK) versus 'crash' (forbidden). Unless you think restoring the previous situation (tp_new = NUll) can cause a worse problem, I would apply this soon, within a week, so not forgotten before the June update releases. The tcl test addition is correct. The fix can be improved later if you think of something better.

I initially wondered why _tkinter has classes that cannot be instantiated (from Python). Then I discovered that this is normal, and that the exception is standard.

>>> type(iter([]))()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create 'list_iterator' instances

Is list_iterator.tp_new NULL? Or does it raise instead of crash by other means?

[serhiy-storchaka]

I called the fix a workaround because other solution may be to make PyType_FromSpec() not setting tp_new in such cases. I expect that this is common mistake when convert a static class to a heap class with PyType_FromSpec(). At least we should add warnings in C API documentation and PEP-384. Or change the behavior of PyType_FromSpec() if it is appropriate. But this is different issue. If PyType_FromSpec() will be changed, proposed fix can be reverted.

[terryjreedy]

I noticed that you reported the crash and asked for a fix or reversion on bpo-15721 6 months ago, with no response. Hence, lets not wait longer.

Looking again, I see that there were 3 rounds of patches in 2012/2013. If you want to follow up, I suggest closing that issue and open a fresh issue with with alternatives and your recommendation.

[python-dev]

New changeset cd25508c62fc by Serhiy Storchaka in branch '3.5':
Issue bpo-23815: Fixed crashes related to directly created instances of types in
https://hg.python.org/cpython/rev/cd25508c62fc

New changeset 1c6326e81c33 by Serhiy Storchaka in branch 'default':
Issue bpo-23815: Fixed crashes related to directly created instances of types in
https://hg.python.org/cpython/rev/1c6326e81c33

[serhiy-storchaka]

After looking closer I found that making PyType_FromSpec() not inheriting tp_new for static types whose base class is 'object' is not an option, because this breaks examples in the documentation, and likely break some third-party code. Opened new bpo-26979 for documenting the catch.

Thank you for your review Terry.