f-string assert is too restrictive

[ericvsmith]

BPO	30682
Nosy	@terryjreedy, @ericvsmith, @ned-deily, @serhiy-storchaka
PRs	bpo-30529: Fix assert failure in with pydebug; add tests. #2232 [3.6] bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (GH-2232) #2242

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = <Date 2017-06-16.16:49:50.463>
created_at = <Date 2017-06-16.09:13:47.503>
labels = ['interpreter-core', '3.7', 'type-crash']
title = 'f-string assert is too restrictive'
updated_at = <Date 2017-06-16.17:13:05.283>
user = 'https://github.com/ericvsmith'

bugs.python.org fields:

activity = <Date 2017-06-16.17:13:05.283>
actor = 'serhiy.storchaka'
assignee = 'eric.smith'
closed = True
closed_date = <Date 2017-06-16.16:49:50.463>
closer = 'eric.smith'
components = ['Interpreter Core']
creation = <Date 2017-06-16.09:13:47.503>
creator = 'eric.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 30682
keywords = []
message_count = 5.0
messages = ['296166', '296174', '296193', '296204', '296212']
nosy_count = 4.0
nosy_names = ['terry.reedy', 'eric.smith', 'ned.deily', 'serhiy.storchaka']
pr_nums = ['2232', '2242']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue30682'
versions = ['Python 3.6', 'Python 3.7']

[ericvsmith]

>>> eval("f'\\\n'")
python: Python/ast.c:4906: FstringParser_ConcatFstring: Assertion `!state->last_str || PyUnicode_GET_LENGTH(state->last_str) != 0' failed.
[2]    12810 abort (core dumped)  ./python

The problem is that some literal strings are zero length. The assert can be deleted.

[serhiy-storchaka]

New changeset 11e97f2 by Serhiy Storchaka (ericvsmith) in branch 'master':
bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (bpo-2232)
11e97f2

[ericvsmith]

New changeset 2eca5b4 by ericvsmith (Serhiy Storchaka) in branch '3.6':
[3.6] bpo-30682: Removed a too-strict assertion that failed for certain f-strings. (GH-2232) (bpo-2242)
2eca5b4

[terryjreedy]

Before the patch, eval("f'\\\n'") gives me the same assertion failure, with an IDLE freeze instead of core dump equivalent, but is seems that the string should be of length 2.

>>> len('\\\n')
2
>>> len(f'\\\n')
2

[ericvsmith]

Terry:

The eval is important. The bug was in evaluating an f-string that consisted of two bytes: a backslash followed by a newline.

And just as:
eval("'\\\n'") == '' # len == 0
so should
eval("f'\\\n'") == '' # len == 0

It's the second one that was throwing the assertion. The parser was seeing these bytes:
f 0x66
' 0x27
\ 0x5c
nl 0xa
' 0x27
and behaving badly (to say the least) by asserting.

Without the eval, I can't think of a way to have a string consisting of those two bytes, but I assume someone who's trickier than I can come up with a way.