IDLE debugger causes crash if not quitted properly before next run

[THRlWiTi]

BPO	24455
Nosy	@terryjreedy, @roseman
Files	fix-mainloop2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2015-11-21.01:38:42.481>
created_at = <Date 2015-06-15.15:57:30.966>
labels = ['expert-IDLE', 'type-crash']
title = 'IDLE debugger causes crash if not quitted properly before next run'
updated_at = <Date 2015-11-21.01:38:42.480>
user = 'https://bugs.python.org/THRlWiTi'

bugs.python.org fields:

activity = <Date 2015-11-21.01:38:42.480>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2015-11-21.01:38:42.481>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2015-06-15.15:57:30.966>
creator = 'THRlWiTi'
dependencies = []
files = ['40553']
hgrepos = []
issue_num = 24455
keywords = ['patch']
message_count = 7.0
messages = ['245384', '245522', '247021', '247221', '251400', '255033', '255036']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'markroseman', 'THRlWiTi', 'python-dev', 'sanad']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue24455'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[THRlWiTi]

# Open a module using IDLE
# Run the module (Press F5)
# Activate the debugger ([DEBUG ON])
# Set a breakpoint in the module
# Run the module again
# Run the module for the third time
# Hit the Quit button in Debug Control window (twice, as the first click appears to do nothing)

Congratulations! You got yourself a crash!
After a while, a window dialogue appears asking "Do you want to end this process?". Click "End Process" (As there doesn't seem to be any other way).

One way to avoid this crash is to press quit before the third run. But even without it, this shouldn't happen. Sometimes the programmer may simply forget that there is an active debugger running...

I'm using
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:43:06) [MSC v.1600 32 bit (Intel)] on win32
(Windows 8.1)

[terryjreedy]

This seems like an extension of bpo-15348, but I will leave it open for now. The additional details may be useful.

[sanad]

I have reviewed the patch(http://bugs.python.org/msg172439) submitted in issue bpo-15348 and works very well for solving this particular issue too. Although I have checked it only on Linux but it works arguably fine.

[terryjreedy]

Thank you for testing this. I am hoping that you left the patch applied for testing in routine use. I an now reviewing the code so I can understand it well enough to apply, and to see what non-debug functions might possibly be affected, and need testing with the patch applied.

[roseman]

Like bpo-15347 and bpo-15348, this was also caused by nested event loops, though the exact problem is slightly different. I've attached fix-mainloop2.patch which has a lengthy comment explaining the problem and how the patch solves it.

This patch also includes the changes from fix-nested-mainloop.patch.

[python-dev]

New changeset f51467273d3b by Terry Jan Reedy in branch '2.7':
Issue bpo-24455: Prevent IDLE from hanging when a) closing the shell while the
https://hg.python.org/cpython/rev/f51467273d3b

New changeset 1ddd77a5e8c8 by Terry Jan Reedy in branch '3.4':
Issue bpo-24455: Prevent IDLE from hanging when a) closing the shell while the
https://hg.python.org/cpython/rev/1ddd77a5e8c8

[terryjreedy]

Patch does two things. On startup, check if debugger is already active and if it is, send 'finish' signal to existing interaction and try again after .1 second. This is the part aimed at the particular problem of this issue.

During active running, use the tcl vwait mechanism https://www.tcl.tk/man/tcl/TclCmd/vwait.htm intended for suspending and resuming a function without blocking the event loop. Using root.mainloop and root.quit for this purpose was a hack that sort of worked but created the shutdown problems in bpo-15347 and bpo-15348.

To further explain the suspend/resume ping-ponging of control: IDLE's bebugger is based on the bdb.Bdb debug engine. User interaction is added by overriding Bdb.user_xyz methods. The overrides tell Bdb what to do after they return by calling Bdb.set_uvw methods. idlelib.Debugger.Idb defines .user_line and .user_exception, which are called when the debugger stops on a line or with an exception. Both end by calling Debugger.Debugger.interaction. This method updates the gui and activates the buttons. It must then pause (now with vwait waitvar) and allow button presses (and other IDLE gui events) to be handled *without* returning. (The stdlib pdb can instead just call "cmd = input('pdb>')" and block waiting for user input.) When a debugger button is pressed, a Bdb.set_uvw method is called and waitvar is set. Then .interaction resumes, disables the buttons, and returns so Bdb can do the action requested. Thanks Mark for working this out and finding the proper pause/resume method.

On my Windows 10 machine, at least, the patch did not completely solve the hang-on-close issues. One or two exceptions are raised instead of hanging. This is improvement. Details are on bpo-15348. Pending a better solution, I added try-except around the gui.interaction call to ignore the exception.

After numerous tests (at least 10) where I started and either stopped the debugger or restarted from the editor, I encountered a hang. But this was not something I could reproduce, and Restart Shell worked, whereas it had not previously.