Remove deprecated re features

[serhiy-storchaka]

BPO	27030
Nosy	@warsaw, @ned-deily, @ezio-melotti, @bitdancer, @serhiy-storchaka
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	re_remove_deprecated.patch re-sub-allow-unknown-escapes.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-12-06.22:26:24.830>
created_at = <Date 2016-05-15.18:38:16.972>
labels = ['expert-regex', 'type-feature', 'library', '3.7']
title = 'Remove deprecated re features'
updated_at = <Date 2017-03-31.16:36:29.026>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:29.026>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-12-06.22:26:24.830>
closer = 'ned.deily'
components = ['Library (Lib)', 'Regular Expressions']
creation = <Date 2016-05-15.18:38:16.972>
creator = 'serhiy.storchaka'
dependencies = []
files = ['42862', '45774']
hgrepos = []
issue_num = 27030
keywords = ['patch']
message_count = 16.0
messages = ['265641', '268222', '268223', '268224', '268334', '281485', '281487', '281491', '281496', '281498', '281503', '281511', '282514', '282556', '282557', '282572']
nosy_count = 7.0
nosy_names = ['barry', 'ned.deily', 'ezio.melotti', 'mrabarnett', 'r.david.murray', 'python-dev', 'serhiy.storchaka']
pr_nums = ['552']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27030'
versions = ['Python 3.6', 'Python 3.7']

[serhiy-storchaka]

Proposed patch removes following deprecated re features:

• Three unused not documented functions: isident(), isdigit() and isname(). They were deprecated since Python 3.3 (bpo-14462).

• '\' + ASCII character now is error if this combination is not defined. This allows to add new control combinations without breaking compatibility. This was deprecated since Python 3.5 (bpo-23622).

• Support for re.LOCALE with string patterns or with re.ASCII. This was deprecated since Python 3.5 (bpo-22407).

[python-dev]

New changeset 09d1af3fe332 by Serhiy Storchaka in branch 'default':
Issue bpo-27030: Unknown escapes consisting of '\' and ASCII letter in
https://hg.python.org/cpython/rev/09d1af3fe332

[serhiy-storchaka]

Thanks Jim for the review.

[python-dev]

New changeset 8ed3880e94e5 by Serhiy Storchaka in branch 'default':
Issue bpo-27030: The re.LOCALE flag now can be used only with bytes patterns.
https://hg.python.org/cpython/rev/8ed3880e94e5

[python-dev]

New changeset a2482e805dff by Martin Panter in branch '3.5':
Fix buggy RE “\parrot_example.py”, uncovered by Issue bpo-27030
https://hg.python.org/cpython/rev/a2482e805dff

New changeset be193f8dbe4c by Martin Panter in branch 'default':
Issue bpo-27030: Merge RE fix from 3.5
https://hg.python.org/cpython/rev/be193f8dbe4c

New changeset c5411cfa0bd3 by Martin Panter in branch '2.7':
Fix buggy RE “\parrot_example.py”, uncovered by Issue bpo-27030
https://hg.python.org/cpython/rev/c5411cfa0bd3

[warsaw]

FWIW, this breaks Mailman 3.1 (and probably 2.1)

[warsaw]

Specifically, point #2; undefined combinations of \ + ASCII becoming an error.

[serhiy-storchaka]

Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings in 3.5. And now they emit warnings even just in string literals (bpo-27364). If Mailman use undefined escape combinations, it could suffer from bpo-27364 too.

[warsaw]

On Nov 22, 2016, at 04:13 PM, Serhiy Storchaka wrote:

Could Mailman be fixed? Undefined combinations of \ + ASCII emitted warnings
in 3.5. And now they emit warnings even just in string literals
(bpo-27364). If Mailman use undefined escape combinations, it could suffer
from bpo-27364 too.

No, I think this is a valid bug/regression.

The Mailman code is basically trying to do this:

    p = re.compile('%\d*d')
    p.sub(r'\s*\d+\s*', some_string)

And so we get the error:

sre_constants.error: bad escape \s at position 0

But this directly contradicts the documentation for re.sub():

"... if it is a string, any backslash escapes in it are processed. That is, \n
is converted to a single newline character, \r is converted to a carriage
return, and so forth. Unknown escapes such as \& are left alone."

Clearly \s is not being left alone, so this is a real regression.