SystemError in compile builtin function

[Zac-HD]

BPO	42218
Nosy	@serhiy-storchaka, @lysnikolaou, @Zac-HD, @miss-islington, @isidentical
PRs	bpo-42218: Check for the error indicator when expecting a new token #23058 bpo-42218: Correctly handle errors in left-recursive rules #23065 [3.9] bpo-42218: Correctly handle errors in left-recursive rules (GH-23065) #23066

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-31.19:06:57.352>
created_at = <Date 2020-10-31.11:16:42.448>
labels = ['interpreter-core', '3.9']
title = 'SystemError in compile builtin function'
updated_at = <Date 2020-11-01.04:14:56.509>
user = 'https://github.com/Zac-HD'

bugs.python.org fields:

activity = <Date 2020-11-01.04:14:56.509>
actor = 'Zac Hatfield-Dodds'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-31.19:06:57.352>
closer = 'lys.nikolaou'
components = ['Interpreter Core']
creation = <Date 2020-10-31.11:16:42.448>
creator = 'Zac Hatfield-Dodds'
dependencies = []
files = []
hgrepos = []
issue_num = 42218
keywords = ['patch']
message_count = 5.0
messages = ['380053', '380063', '380088', '380090', '380113']
nosy_count = 5.0
nosy_names = ['serhiy.storchaka', 'lys.nikolaou', 'Zac Hatfield-Dodds', 'miss-islington', 'BTaskaya']
pr_nums = ['23058', '23065', '23066']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue42218'
versions = ['Python 3.9']

[Zac-HD]

In Python 3.9.0, running

    compile('A.\u018a\\ ', '<string>', 'single')

raises

SystemError: <built-in function normalize> returned a result with an error set

This is obviously invalid syntax, but it would still be nice to raise SyntaxError for that and the SystemError may be symptomatic of a bug. Unfortunately I have no insight to offer beyond this hand-minimised reproducer :-/

[serhiy-storchaka]

Nice catch Zac!

[lysnikolaou]

New changeset 02cdfc9 by Lysandros Nikolaou in branch 'master':
bpo-42218: Correctly handle errors in left-recursive rules (GH-23065)
02cdfc9

[miss-islington]

New changeset cfcb952 by Lysandros Nikolaou in branch '3.9':
[3.9] bpo-42218: Correctly handle errors in left-recursive rules (GH-23065) (GH-23066)
cfcb952

[Zac-HD]

Wow! Thanks and congrats on the super-fast fix :-)

FYI Paul Ganssle has recently [1] started to work on adding property-based tests to CPython CI [2]. Once Paul gets the stubs and first set of tests working, I'll start moving the others over from my demo repo [3] to CPython master so we catch this kind of bug earlier in future.

[1] https://bugs.python.org/issue42109; see also https://bugs.python.org/issue38953
[2] https://pyfound.blogspot.com/2020/05/property-based-testing-for-python.html
[3] https://github.com/Zac-HD/stdlib-property-tests