enhance the grammar of invalid_replacement_field

[sunmy2019]

I am struggling with finding the correct SyntaxError location. I add a new tool RAISE_SYNTAX_ERROR_ON_NEXT_TOKEN for this use case.

While it increases the test failure of test_fstring, I think this change is beneficial.

Notable changes:

before

  File "<string>", line 1
    f'{"s"!r{":10"}}'
            ^
SyntaxError: f-string: expecting '}'

now (nicer hint)

  File "<string>", line 1
    f'{"s"!r{":10"}}'
            ^
SyntaxError: f-string: expecting ':' or '}'

---

before

  File "<string>", line 1
    f'{3!:'
        ^ 
SyntaxError: f-string: missing conversion character

now (points to the right token)

  File "<string>", line 1
    f'{3!:'
         ^
SyntaxError: f-string: missing conversion character

---

before:

  File "<string>", line 1
    f'{4:{/5}}'
          ^
SyntaxError: f-string: expecting '}'

now (points to the correct token, nicer hint):

  File "<string>", line 1
    f'{4:{/5}}'
         ^
SyntaxError: f-string: expecting '}', or format specs

---

before

    f'{a $ b}'
         ^
SyntaxError: f-string: expecting '}'

now (nicer hint)

  File "?", line 1
    f'{a $ b}'
         ^
SyntaxError: f-string: expecting '=', or '!', or ':', or '}'

[sunmy2019]

The column number is crucial especially in nested fstring-replacement-fields. e.g.
before

  File "<stdin>", line 1
    f"{123:{234 +}}"
                 ^
SyntaxError: f-string: expecting '}'

now

  File "<stdin>", line 1
    f"{123:{234 +}}"
                ^
SyntaxError: f-string: expecting '=', or '!', or ':', or '}'

The variable names in C was spontaneously picked though. But I do carefully changed all occurence of _PyPegen_raise_error.

Request for comment.

CC: @pablogsal @lysnikolaou

[sunmy2019]

I think we can even make one step further with c9de3e8 for more consistency, while this changes many errors messages.

This will capture many invalid syntax, and empty expression not allowed errors.

  File "<string>", line 1
    f'{}'
       ^
SyntaxError: f-string: expecting expression after '{'

AssertionError: "f-string: empty expression not allowed" does not match "f-string: expecting expression after '{' (<string>, line 1)"


  File "<string>", line 1
    f'{!r}'
       ^
SyntaxError: f-string: expecting expression after '{'
AssertionError: "f-string: expression required before '!'" does not match "f-string: expecting expression after '{' (<string>, line 1)"

  File "<string>", line 2
    :a}'''
    ^
SyntaxError: f-string: expecting expression after '{'
AssertionError: "f-string: expression required before ':'" does not match "f-string: expecting expression after '{' (<string>, line 2)"


  File "<string>", line 1
    f'{,}'
       ^
SyntaxError: f-string: expecting expression after '{'
AssertionError: "invalid syntax" does not match "f-string: expecting expression after '{' (<string>, line 1)"

  File "<string>", line 1
    f'{4:{/5}}'
          ^
SyntaxError: f-string: expecting expression after '{'

AssertionError: "f-string: invalid syntax" does not match "f-string: expecting expression after '{' (<string>, line 1)"

[pablogsal]

Whops, I have merged the wrong PR.

@sunmy2019 Can you reopen this so we can discuss it more? I am going to revert it on the pep branch