Improve the warning message for invalid escape sequences

[yan12125]

BPO	28128
Nosy	@rhettinger, @ncoghlan, @ericvsmith, @ned-deily, @ezio-melotti, @bitdancer, @encukou, @vadmium, @serhiy-storchaka, @1st1, @timgraham, @yan12125, @Vgr255
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	28128.diff 28128-2.diff 28128-3.diff 28128-3.diff: Regenerated for review 28128-4.diff 28128-5.diff 28128-6.diff 28128-7.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2016-10-31.19:26:57.914>
created_at = <Date 2016-09-13.15:10:24.165>
labels = ['interpreter-core', 'type-feature', '3.7', 'release-blocker']
title = 'Improve the warning message for invalid escape sequences'
updated_at = <Date 2017-03-31.16:36:18.468>
user = 'https://github.com/yan12125'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:18.468>
actor = 'dstufft'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2016-10-31.19:26:57.914>
closer = 'eric.smith'
components = ['Interpreter Core']
creation = <Date 2016-09-13.15:10:24.165>
creator = 'yan12125'
dependencies = []
files = ['44694', '45284', '45285', '45287', '45288', '45290', '45292', '45293']
hgrepos = []
issue_num = 28128
keywords = ['patch']
message_count = 55.0
messages = ['276285', '276286', '276293', '276326', '276341', '276364', '276548', '276560', '276641', '276646', '276658', '276661', '276662', '276663', '276665', '276666', '276668', '276698', '276715', '276716', '276717', '276720', '276728', '276730', '276731', '276785', '277916', '277967', '278251', '279650', '279654', '279724', '279754', '279757', '279759', '279771', '279776', '279778', '279781', '279782', '279783', '279784', '279785', '279789', '279790', '279791', '279793', '279794', '279795', '279796', '279797', '279798', '279799', '279820', '279821']
nosy_count = 14.0
nosy_names = ['rhettinger', 'ncoghlan', 'eric.smith', 'ned.deily', 'ezio.melotti', 'r.david.murray', 'petr.viktorin', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'yselivanov', 'Tim.Graham', 'yan12125', 'abarry']
pr_nums = ['552']
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28128'
versions = ['Python 3.6', 'Python 3.7']

[yan12125]

In bpo-27364, invalid escape sequences in string literals are deprecated. Currently the deprecation message is not so useful when fixing lots of files in one or more large projects. For example, I have two files foo.py and bar.py:

# foo.py
import bar

# bar.py
print('\d')

It gives:
$ python3.6 -W error foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in <module>
    import bar
DeprecationWarning: invalid escape sequence '\d'

My idea is that the warning message can be improved to provide more information. In http://bugs.python.org/issue27364#msg269373 it's proposed to let a linter check such misuses. It's useful within a single project. For a project that depends on lots of external projects, a linter is not enough. Things are worse when __import__, imp or importlib are involved, or sys.path is modified. I have to either add some codes or use a debugger to show which module is imported.

For above reasons, I propose to add at least the filename and the line number to the warning message. For example:

$ ./python -W error foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in <module>
    import bar
  File "/home/yen/Projects/cpython/build/bar.py", line 1
    print('\d')
         ^
SyntaxError: (deprecated usage) invalid escape sequence '\d'

With that I can know which file or project I should blame.

Added some of reviewers from bpo-27364 to nosy list.

[yan12125]

And I'd like to ask Ned: for me it's an improvement of an existing feature, so I guess it can enter the 3.6 branch?

[ned-deily]

It sounds like a fix but let's see the final patch first. If a core developer wants to apply it to the default branch for 3.7, we can decide whether it should go into 3.6, too.

[bitdancer]

The error can't be a SyntaxError, it must be a DeprecationWarning. If you can improve the deprecation warning text, I'd be in favor of calling that a fix to the original feature and put it in 3.6. The deprecation warning can even say that this will be a SyntaxError some time in the future.

[Vgr255]

Hello, and thanks! I'll work on a patch this week, or at most next week. I will make it so that it's completely uncontroversial to apply it to 3.6 as well (won't change the actual feature, only prettify the error message), so no need to worry about that :)

[vadmium]

See also bpo-28028. Serhiy suggested translating warnings to SyntaxWarning in general. Looks like that may help narrowing down the location of escaping problems.

[Vgr255]

Besides converting the DeprecationWarning to a Syntax{Error,Warning}, I don't see an easy way to include the offending line (or even file). The place in the code where the strings are created has no idea *where* they are being defined. AIUI, either we special-case this, or we resolve bpo-28028 (but I don't think the latter can go in 3.6).

[bitdancer]

Are SyntaxWarnings silent by default? If not it can't even go into 3.7.