IDLE custom keymaps don't work anymore (Windows, Python 3.6.0a3)

[vedgar]

BPO	27821
Nosy	@terryjreedy, @serhiy-storchaka, @vedgar

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2016-08-25.02:15:16.363>
created_at = <Date 2016-08-21.10:23:15.706>
labels = ['expert-IDLE', 'type-bug']
title = "IDLE custom keymaps don't work anymore (Windows, Python 3.6.0a3)"
updated_at = <Date 2016-08-25.06:22:13.868>
user = 'https://github.com/vedgar'

bugs.python.org fields:

activity = <Date 2016-08-25.06:22:13.868>
actor = 'veky'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2016-08-25.02:15:16.363>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2016-08-21.10:23:15.706>
creator = 'veky'
dependencies = []
files = []
hgrepos = []
issue_num = 27821
keywords = []
message_count = 14.0
messages = ['273289', '273295', '273297', '273320', '273342', '273343', '273516', '273535', '273546', '273586', '273595', '273618', '273619', '273631']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'python-dev', 'serhiy.storchaka', 'veky']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27821'
versions = ['Python 3.6']

[vedgar]

I had a custom keymap in IDLE. Since I installed 3.6.0a3, it doesn't work anymore. Worse, I cannot switch to anything except "IDLE Classic Windows" - Apply does nothing, but when I click OK, it doesn't switch.

I suppose https://hg.python.org/cpython/rev/e6e6c71776b0 is the suspect, but I can't be sure. I tried editing .idlerc by hand, setting name, name2 and default, but nothing seems to help. :-/

Please resolve this, I'm lost without my favorite Python IDE. :-o

[terryjreedy]

I use a custom key set also and have no problem before or after the Modern Unux patch (else I would not have applied it ;-). Did you have .0a2 installed? Which Windows?

What I have in config-main.cfg is

[Keys]
default = False
name = Terry

[Theme]
default = False
name = Custom Dark

name2 should not be set unless you are using Modern Unix of IDLE Dark respectively. Anything else and IDLE will possibly not work. What did you see before editing? Are you running IDLE from a console, so you can see errors printed?

[vedgar]

Windows 10. No, I jumped from .0a1 to .0a3. Now I installed .0a4 and have the same problem. I'm afraid of going back to .0a2, was taught long ago to never downgrade anything. :-) But I have 3.5.1 installed separately, and its IDLE keeps working without problem.

My config-main.cfg is
-------------------------
[EditorWindow]
font-bold = False
font-size = 12
font = consolas
height = 25

[General]
autosave = 1

[Keys]
default = False
name = Veky2

[Theme]
name = IDLE New
------------------------

(Veky2 is what I've made after Veky stopped working, thinking that it got corrupted or something.) Yes, I know about regular semantics of name2, it's just that I was desperate. :-o

I tried running from the console (py -m idlelib, right?), and I don't see any error messages. It's just that every time I go to Options > Configure IDLE > Keys > Key Set, radiobutton Use a Custom Key Set is selected, and option IDLE Classic Windows is selected beside it. I can select Veky2 and click OK, but it doesn't switch, and the next time I go to the same place, it's IDLE Classic Windows again. :-(

[terryjreedy]

(Don't install .0a2, .0a1 is good enough as comparison.)

Are you saying that you see, in one line,
(*) Use a Custom Key Set [IDLE Classic Windows --]
(where -- is the select mini-button)

This would be wrong and would suggest that 'IDLE Classis Windows' was somehow added to .idlerc/config-keys.cfg and should be removed and your install retested.

Or do you see, on two lines
( ) Use a Built-in Key Set [IDLE Classic Windows --]
(*) Use a Custom Key Set [ Veky --]

This would be normal. If so, please tell me one difference between the two key sets (quote the exact mis-matching lines) and what behavior tells you that the IDLE CW key set is being used in spite of Veky being selected.

I retested on my machine that there is a difference between I.C.W and Terry key sets and that switches the selection, back and forth, in the dialog switches the behavior back in Shell in the expected way.

[vedgar]

Neither. I see _two_ rows, lower one is as you quoted. So:

--Key Set----------------------------------------
( ) Use a Built-in Key Set [IDLE Classic Windows --]
(*) Use a Custom Key Set [IDLE Classic Windows --]
-------------------------------------------------

The first button is grayed out. I can select the first radio button and it works (I can switch to a different builtin set), but as soon as I select the second one, the old behavior returns (select Veky2, OK, doesn't work).

The shortcut I miss the most is Ctrl+Up/Down instead of Alt+p/n for history back/forward. But I guess it doesn't really matter much, unless you're willing to add it to the builtin set. :-D

I also have customized sys.ps1, sys.ps2 and sys.__interactivehook__, but I really think this shouldn't matter for this bug.

[vedgar]

P.S. I tested and no, IDLE Classic Windows (nor anything else except Veky2, I deleted Veky) is not added to config-keys.cfg. My config-keys.cfg is

----------------------------------------------------
[Veky2]
toggle-auto-coloring = <Control-Key-slash>
tabify-region = <Alt-Key-5> <Meta-Key-5>
copy = <Control-Key-c> <Control-Key-C>
del-word-right = <Control-Key-Delete>
save-window-as-file = <Control-Shift-Key-S> <Control-Shift-Key-s>
center-insert = <Control-Key-l> <Control-Key-L>
dedent-region = <Shift-Key-Tab>
indent-region = <Control-Key-bracketright>
untabify-region = <Alt-Key-6> <Meta-Key-6>
select-all = <Control-Key-a> <Control-Key-A>
end-of-file = <Control-Key-d> <Control-Key-D>
open-module = <Alt-Key-m> <Meta-Key-m> <Alt-Key-M> <Meta-Key-M>
plain-newline-and-indent = <Control-Key-j> <Control-Key-J>
close-all-windows = <Control-Key-q> <Control-Key-Q>
paste = <Control-Key-v> <Control-Key-V>
open-window-from-file = <Control-Key-o> <Control-Key-O>
python-docs = <Key-F1>
do-nothing = <Control-Key-F12>
cut = <Control-Key-x> <Control-Key-X>
toggle-tabs = <Alt-Key-t> <Meta-Key-t> <Alt-Key-T> <Meta-Key-T>
uncomment-region = <Alt-Key-4> <Meta-Key-4>
open-new-window = <Control-Key-n> <Control-Key-N>
del-word-left = <Control-Key-BackSpace>
find-selection = <Control-Key-F3>
save-window = <Control-Key-s> <Control-Key-S>
history-previous = <Control-Key-Up>
remove-selection = <Key-Escape>
find = <Control-Key-f> <Control-Key-F>
open-class-browser = <Alt-Key-c> <Meta-Key-c> <Alt-Key-C> <Meta-Key-C>
close-window = <Alt-Key-F4> <Meta-Key-F4>
smart-backspace = <Key-BackSpace>
python-context-help = <Shift-Key-F1>
save-copy-of-window-as-file = <Alt-Shift-Key-S> <Alt-Shift-Key-s>
redo = <Control-Shift-Key-Z> <Control-Shift-Key-z>
restart-shell = <Control-Key-F6>
smart-indent = <Key-Tab>
goto-line = <Alt-Key-g> <Meta-Key-g> <Alt-Key-G> <Meta-Key-G>
comment-region = <Alt-Key-3> <Meta-Key-3>
beginning-of-line = <Key-Home>
interrupt-execution = <Control-Key-c> <Control-Key-C>
find-in-files = <Alt-Key-F3> <Meta-Key-F3>
undo = <Control-Key-z> <Control-Key-Z>
view-restart = <Key-F6>
replace = <Control-Key-h> <Control-Key-H>
print-window = <Control-Key-p> <Control-Key-P>
history-next = <Control-Key-Down>
change-indentwidth = <Alt-Key-u> <Meta-Key-u> <Alt-Key-U> <Meta-Key-U>
find-again = <Control-Key-g> <Key-F3> <Control-Key-G>
newline-and-indent = <Key-Return> <Key-KP_Enter>
----------------------------------------------------

[terryjreedy]

I can't make any sense of this report.

In configdialog.py, the DynOptionMenu self.optMenuKeysCuston is set in LoadKeyConfig (line 1066) by
itemList = idleConf.GetSectionList('user', 'keys')
itemList.sort()
if not itemList:
self.radioKeysCustom.config(state=DISABLED)
self.customKeys.set('- no custom keys -')
else:
self.optMenuKeysCustom.SetMenu(itemList, itemList[0])

or the same without the 'if' clause. This code was not changed by the patch. Nor was config.IdleConf.GetSectionList. So there should be no way for anything to appear on that menu unless in the user keys file.

To test Veky2 as posted, I added it to my user keys files. I opened a IDLE fresh and brought up the config dialog. Veky2 was on the list and when I selected it and closed the dialog, there was no problem. I tested and the Veky2 definition of <<dedent region>> as <Shift-Key-Tab> worked. (I just added that to Terry.) When I opened the dialog again, Veky2 was still there, selected.

In the absence of a failing test case or of any idea how the code could produce the behavior you report, there is not much I can do.

[vedgar]

But there is something _I_ can do now. :-)

I put a pdb.set_trace in that code you quoted, and went exploring. And here is what I found.

It calls
currentOption = idleConf.CurrentKeys()
That one calls
return self.current_colors_and_keys('Keys')
and _that one_ calls
default = self.GetOption('main', 'Theme', 'default',
^^^^^ note this
type='bool', default=True)
and later it uses defaut as default for both theme and keys.

I did have custom keys, but not a custom highlight theme. Sure enough, as soon as I made a trivial copy of IDLE classic, things worked perfectly. :-)

So now I don't have a problem anymore. But still I think it is a bug that should be fixed. Not everyone who hates Alt+p also hates the orange words on white background. :-D

[terryjreedy]

Thanks for tracing further. That is the new bug in .0a3. "'Theme'" should be "section", as in the next two GetOption calls. You should then not need a custom theme. My new tests did not cover all the possible cases, so I should add more along with the fix.

[terryjreedy]

Vedran, I have used IDLE's imperfect visual debugger and was thinking about how I might use it to solve this problem. But I have never used pdb. Perhaps I should learn it. Please explain how you used set_trace to track down this bug.

For bpo-24265, I have similarly analyzed some of the top-level calls and believe I know the problem -- a missing function call -- but where? I think I need to trace both what does not work and something similar that does work and compare execution paths. How could I use pdb for this issue?

[vedgar]

Well, I don't know how good this comment box is for a pdb tutorial. :-) And I learned it a long time ago, so I don't really remember how it is to not know it. :-/ The main reason I love it is purely sentimental: it reminds me of those silly text-based adventure games. If you played Zork, you'll feel right at home. :-D

(If you want a more "ordinary" introduction, yet refreshingly different than the boring stdlib manpage, visit https://pythonconquerstheuniverse.wordpress.com/2009/09/10/debugging-in-python/.)

You put "import pdb; pdb.set_trace()" (or "__import__('pdb').set_trace()" if you hate semicolons more than dunders:) before the line you want to set a breakpoint in. Then you run the program normally. When it hits a breakpoint, it gives you a (Pdb) prompt, and you can look around.

(Alternatively, when you get an exception, just "import pdb; pdb.pm()" for a post-mortem debugger. All the stack frames that lead to exception are preserved with all their local variables, nicely inspectable. Beautiful.:)

u and d (up and down) are the elevator buttons that move you along the frame stack: u towards earlier (outer) calls, d towards later (inner) calls. w (where) lets you see which "floor" you're on.

On every floor, p (print) and pp (prettyprint) give you information about various objects you see, like "examine" in text adventures :-). ! lets you execute an arbitrary Python statement (! is not needed in most cases, but it's good to make a habit of writing it) inside the current stackframe (yes, everything is live:). interact gives you a full Python shell attached on the current frame.

l lists a few lines around your current place, ll lists the whole function (or another meaningful whole). a gives you the arguments for the current function call.

s and r are the "in" and "out" commands, allowing you to step inside and return from currently called function. n is just "next", meaning "execute current line and show me the next one". Unfortunately, n has no inverse (you cannot undo an executed statement), but j is a (limited) goto. If you go too far into a strange land, run restarts the program from the beginning.

pdb always shows the line it's about to execute (it's not yet executed). Just pressing Enter repeats the last given command (in case of l, keeps listing onwards).

c means quit the debugger and continue executing program normally. q means quit the debugger and program altogether. There are many more commands, but those are kinda basic ones to know your way around.

Ok, now what I've done with IDLE. You mentioned configdialog.py, line 1066. I go there and find a function

    def LoadKeyCfg(self):

Ok, that's obviously the function I have to trace. As the first statement in that function, I put "import pdb; pdb.set_trace()" call and save the file. Then I go to command prompt, and start IDLE. I go to Options > Configure IDLE, and sure enough, at my command prompt there is

c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\configdialog.py(1069)LoadKeyCfg()
-> self.keysAreBuiltin.set(idleConf.GetOption(
(Pdb)

a Pdb prompt. I type l and Enter, to see where I am in the code.

(Pdb) l
1064 self.SetHighlightTarget()
1065
1066 def LoadKeyCfg(self):
1067 import pdb; pdb.set_trace()
1068 ##current keys type radiobutton
1069 -> self.keysAreBuiltin.set(idleConf.GetOption(
1070 'main', 'Keys', 'default', type='bool', default=1))
1071 ##currently set keys
1072 currentOption = idleConf.CurrentKeys()
1073 ##load available keyset option menus
1074 if self.keysAreBuiltin.get(): #default theme selected
(Pdb)

Ok, so it's going to set a Tkinter variable (I know by seeing ".set", but there is also a whatis command if you want to see the type of something). I say n to execute it (twice because it's two lines), and then

(Pdb) p self.keysAreBuiltin.get()
False
(Pdb)

So far so good. Keys aren't builtin.

(Pdb) n

c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\configdialog.py(1072)LoadKeyCfg()
-> currentOption = idleConf.CurrentKeys()

Ok, now it's going to set CurrentKeys.

(Pdb) n
...
(Pdb) p idleConf.CurrentKeys()
'IDLE Classic Windows'

Ok, this shouldn't happen. Let's goto it again (hoping there aren't many sideeffects:) and step into it this time.

(Pdb) l
1069 self.keysAreBuiltin.set(idleConf.GetOption(
1070 'main', 'Keys', 'default', type='bool', default=1))
1071 ##currently set keys
1072 currentOption = idleConf.CurrentKeys()
1073 ##load available keyset option menus
1074 -> if self.keysAreBuiltin.get(): #default theme selected
1075 itemList = idleConf.GetSectionList('default', 'keys')
1076 itemList.sort()
1077 self.optMenuKeysBuiltin.SetMenu(itemList, currentOption)
1078 itemList = idleConf.GetSectionList('user', 'keys')
1079 itemList.sort()
(Pdb)

So it's line 1072.

(Pdb) j 1072

c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\configdialog.py(1072)LoadKeyCfg()
-> currentOption = idleConf.CurrentKeys()
(Pdb) s
--Call--
c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\config.py(368)CurrentKeys()
-> def CurrentKeys(self):
(Pdb) l
363
364 def CurrentTheme(self):
365 "Return the name of the currently active text color theme."
366 return self.current_colors_and_keys('Theme')
367
368 -> def CurrentKeys(self):
369 """Return the name of the currently active key set."""
370 return self.current_colors_and_keys('Keys')
371
372 def current_colors_and_keys(self, section):
373 """Return the currently active name for Theme or Keys section.
(Pdb) n
c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\config.py(370)CurrentKeys()
-> return self.current_colors_and_keys('Keys')
(Pdb) s
--Call--
c:\users\veky\appdata\local\programs\python\python36\lib\idlelib\config.py(372)current_colors_and_keys()
-> def current_colors_and_keys(self, section):
(Pdb)

A few ns, and we have a culprit. ll to see a whole function and confirm our suspicion, and a final test: make a new color theme. It works. Victory! :-)

[python-dev]

New changeset f198457d1475 by Terry Jan Reedy in branch 'default':
Issue bpo-27821: Fix bug in idlelib.comfig function and add new tests.
https://hg.python.org/cpython/rev/f198457d1475