IDLE doesn't highlight f-strings properly

[KenzieTogami]

BPO	29071
Nosy	@terryjreedy, @serhiy-storchaka, @ammaraskar
Files	python_idle_3.6_bug.png: Screenshot of f- and r- strings in IDLE idle-f-string-highlight.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2016-12-27.05:20:10.842>
created_at = <Date 2016-12-26.01:16:18.012>
labels = ['expert-IDLE', 'type-bug', '3.7']
title = "IDLE doesn't highlight f-strings properly"
updated_at = <Date 2017-04-07.18:30:17.639>
user = 'https://bugs.python.org/KenzieTogami'

bugs.python.org fields:

activity = <Date 2017-04-07.18:30:17.639>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2016-12-27.05:20:10.842>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2016-12-26.01:16:18.012>
creator = 'Kenzie Togami'
dependencies = []
files = ['46037', '46048']
hgrepos = []
issue_num = 29071
keywords = ['patch']
message_count = 11.0
messages = ['284010', '284051', '284062', '284064', '284065', '284420', '284439', '284440', '284472', '284479', '291284']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'python-dev', 'serhiy.storchaka', 'ammar2', 'Kenzie Togami']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29071'
versions = ['Python 3.6', 'Python 3.7']

[KenzieTogami]

IDLE doesn't highlight f-strings like r-strings or u-strings. See the attached screenshot.

[ammaraskar]

I've attached a patch that colorizes f-strings.

The issue is marked as "test needed" which the dev guide says "... reporter should post a script or instructions to let a triager or developer reproduce the issue."

Steps to re-create are to open an IDLE window and type in the code

f"123"
F"123"
u"123"

and notice the colorization difference.

[python-dev]

New changeset a48cc85e5706 by Terry Jan Reedy in branch '3.6':
Issue bpo-29071: IDLE now colors f-string prefixes (but not invalid ur prefixes).
https://hg.python.org/cpython/rev/a48cc85e5706

[terryjreedy]

There are 10 possible f-string prefixes, including combinations with r/R. See
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals

While adding these, I noticed that IDLE was coloring now invalid ur prefixes and deleted those.

I decided to test by modifying the existing human visual htest.

Ammar: thank you for submitting a patch. It happens that I wrote my patch before I saw your submission. If I had seen yours first, I might have given you a chance to finish it. Feel free to try another IDLE issue.

[ammaraskar]

Not a problem, thank you for the encouragement.

[serhiy-storchaka]

The regular expression can be simplified using local flags:

• stringprefix = (r"(\br|R|u|U|f|F|fr|Fr|fR|FR|rf|rF|Rf|RF"

•                "|b|B|br|Br|bR|BR|rb|rB|Rb|RB)?")
  

+ stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?"

[python-dev]

New changeset 62d3c0336df6 by Terry Jan Reedy in branch '3.6':
Issue bpo-29071: Use local flags for IDLE colorizer string prefix matcher.
https://hg.python.org/cpython/rev/62d3c0336df6

[terryjreedy]

Thank you for the new feature (I presume) and its application. I disliked both writing and reading all the near duplication. I am posting this to python-list as an example of what the new feature is good for.

[serhiy-storchaka]

Thank you for advertising the new feature. Would you like to improve the
documentation? It always was the hardest part to me.

[terryjreedy]

The doc appears to be carefully accurate, but the second sentence especially is a bit awkward. I will open a new issue.