textwrap.dedent() fails when largest common whitespace is a substring of smallest leading whitespace

[robertjli]

BPO	21827
Nosy	@rhettinger, @pitrou, @ezio-melotti, @bitdancer, @zware, @serhiy-storchaka, @jayvdb, @ilevkivskyi
Files	cb18733ce8f1.diff 34e88a05562f.diff issue21827.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-10-28.19:50:48.557>
created_at = <Date 2014-06-22.16:44:47.794>
labels = ['easy', 'type-bug', 'library']
title = 'textwrap.dedent() fails when largest common whitespace is a substring of smallest leading whitespace'
updated_at = <Date 2015-10-28.19:50:48.555>
user = 'https://bugs.python.org/robertjli'

bugs.python.org fields:

activity = <Date 2015-10-28.19:50:48.555>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-10-28.19:50:48.557>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2014-06-22.16:44:47.794>
creator = 'robertjli'
dependencies = []
files = ['35727', '35731', '40873']
hgrepos = ['258']
issue_num = 21827
keywords = ['patch', 'easy']
message_count = 16.0
messages = ['221277', '221281', '221282', '221283', '221287', '221292', '224544', '224547', '253593', '253596', '253600', '253605', '253609', '253612', '253620', '253621']
nosy_count = 12.0
nosy_names = ['rhettinger', 'pitrou', 'ezio.melotti', 'r.david.murray', 'python-dev', 'zach.ware', 'serhiy.storchaka', 'puppet', 'robertjli', 'yjchen', 'jayvdb', 'levkivskyi']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21827'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[robertjli]

Failing test case: " \tboo\n \tghost"

expected: " \tboo\n\tghost"
returns: " \tboo\n \tghost"

[rhettinger]

Confirmed that the test case fails on 2.7, 3.4, and 3.5

[rhettinger]

This one isn't hard. Would you like to make a patch? If not, I get to it this evening.

[yjchen]

Hi Raymond- Rob and I have a patch ready. We are figuring out how to upload/submit it. New to this... :)

[robertjli]

YJ and I are adding a patch and an additional test.

[zware]

Raymond confirmed that the issue exists on 2.7 and 3.4, so we'll keep those in the version list. Whoever makes the commit will take care of backporting the patch, though.

[puppet]

Tested and it works fine on CentOS 6.4 in 2.7, 3.4 and 3.5

[rhettinger]

I've got it from here.

[jayvdb]

Not surprising also occurs on Python 3.2, 3.3, & 3.6.
I'm not sure whether fixes like this are suitable to be merged into 3.2/3.3.(is there a document describing this?)

[serhiy-storchaka]

LGTM. But the implementation can be more efficient. Here is optimized patch.

34e88a05562f.diff:
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*100)' -- 'textwrap.dedent(s)'
10000 loops, best of 3: 87.5 usec per loop
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*1000)' -- 'textwrap.dedent(s)'
1000 loops, best of 3: 780 usec per loop

bpo-21827.patch:
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*100)' -- 'textwrap.dedent(s)'
10000 loops, best of 3: 51 usec per loop
$ ./python -m timeit -s 'import textwrap; s = "{0} x\n{0}\tx\n".format(" "*1000)' -- 'textwrap.dedent(s)'
1000 loops, best of 3: 395 usec per loop

[bitdancer]

It's one of the PEPs (I forget which one). We only apply non-security fixes to the most recent non-development branch...except that for now we also apply bug fixes to 2.7. So, 2.7, 3.5, and 3.6 are the targets for this issue.