tabs don't work correctly in python repl

[1st1]

BPO	25660
Nosy	@ronaldoussoren, @ncoghlan, @larryhastings, @ned-deily, @bitdancer, @berkerpeksag, @vadmium, @serhiy-storchaka, @1st1, @cpitclaudel
Files	rlcompleter.patch completion.py: Emacs' completion code

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-03-02.10:33:05.897>
created_at = <Date 2015-11-18.19:58:50.887>
labels = ['interpreter-core', 'type-bug']
title = "tabs don't work correctly in python repl"
updated_at = <Date 2016-10-29.14:14:48.215>
user = 'https://github.com/1st1'

bugs.python.org fields:

activity = <Date 2016-10-29.14:14:48.215>
actor = 'cpitclaudel'
assignee = 'none'
closed = True
closed_date = <Date 2016-03-02.10:33:05.897>
closer = 'berker.peksag'
components = ['Interpreter Core']
creation = <Date 2015-11-18.19:58:50.887>
creator = 'yselivanov'
dependencies = []
files = ['41085', '45235']
hgrepos = []
issue_num = 25660
keywords = ['patch']
message_count = 32.0
messages = ['254855', '254856', '254859', '254860', '254866', '254868', '254871', '254931', '254998', '255028', '255029', '255030', '255085', '255181', '255205', '255248', '255251', '255268', '255451', '259526', '259531', '259532', '259533', '259535', '259538', '259579', '259580', '259590', '259591', '279525', '279652', '279674']
nosy_count = 12.0
nosy_names = ['ronaldoussoren', 'ncoghlan', 'larry', 'ned.deily', 'r.david.murray', 'Yury.Selivanov', 'python-dev', 'berker.peksag', 'martin.panter', 'serhiy.storchaka', 'yselivanov', 'cpitclaudel']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25660'
versions = ['Python 3.5', 'Python 3.6']

[1st1]

When Python is compiled with readline, repeatedly pressing <TAB> key in repl breaks the repl prompt.

Below is what I see when I hit <TAB> 4 times.

yury@ysmac ~/dev/py/cpython $ ./python.exe
Python 3.5.0+ (3.5:4ae62ddf7bc7+, Nov 18 2015, 14:52:57)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

>>>

>>>

>>>

Reproducible when Python is built from source on latest OS X. Works OK when installed from MacPorts.

Further, if I add

   PyOS_ReadlineFunctionPointer = PyOS_StdioReadline;

in PyOS_Readline in myreadline.c, everything works as expected, so I think this is a readline bug.

[bitdancer]

This was reported (on twitter) by David Beazly as well, as applying to the OSX downloaded from python.org. It works fine on linux for me. Is it readline-version dependent, or a bug in the readline shipped with OSX?

[vadmium]

Is this related to the BSD editline library (libedit), or is the actual Gnu Readline library being used? Apparently you can check for "libedit" in readline.__doc__ to tell the difference.

[1st1]

Is this related to the BSD editline library (libedit), or is the actual Gnu Readline library being used? Apparently you can check for "libedit" in readline.__doc__ to tell the difference.

Great suggestion. So on my machine, macports version (no bug) is built against readline; manual build from source -- libedit.

So apparently, this is a libedit thing.

[vadmium]

David Beazley’s description of the bug: “hitting tab inserts spaces and a carriage return at the same time.”

[vadmium]

Applying my patch at bpo-13501, then building with “autoreconf && ./configure --with-readline=editline”, I can reproduce the funny tab behaviour on Linux.

[vadmium]

I think this might just be a side effect of the way we abuse the tab completer to insert a literal tab (bpo-23441, revision 82ccdf2df5ac). If I change the code to insert the letter T instead of tabs:

if not text.strip('T'):
    if state == 0:
        return text + 'T'
    else:
        return None

I see this behaviour:

• Manually type three Ts
• Press Tab once, a fourth T is added nicely
• Press Tab a second time, it beeps and displays a completion list with a single item, and then completes my line to five Ts
• Pressing Tab again repeats the beep, completion list, and appending a T

Illustration:

>>> TTTT  <== Typed T three times, then Tab twice
TTTTT  <== Completion list from second Tab press
>>> TTTTT
TTTTTT
>>> TTTTTT

[1st1]

Attached is a patch that uses different logic for tabulation. Instead of returning '\t' from the completion callback, it instead calls explicit readline API: "readline.insert_text('\t'); readline.redisplay()"

Please review.

[1st1]

Can someone try the patch? I'm fairly confident it should work, and it'd be great to have it fixed in 3.5.1.

[vadmium]

Sorry to throw a potential spanner in the works, but I think this introduces a dependence on the readline module. Consider what happens when the Completer class is used but Readline is not, or even if the “readline” could not be imported (see bottom of rlcompleter.py).

Maybe there is a way around this, but it might involve more complicated code.