consider parsing type comments as expressions

[ddfisher]

Pros:

• Easier to understand API. (This has already caused some user confusion; see Are there any specs for the typed AST? #4.)

Cons:

• Errors in type comments will become syntax errors. (But misplaced type comments are already syntax errors, so maybe this makes sense.)

[ddfisher]

That said, you have to call back into the parser to handle forward references in type annotations (which are given as strings), and type comments are allowed to be forward references (because they aren't executed), so maybe the current behavior actually makes sense.

[mbdevpl]

Yes, it does make sense to keep them as strings, because type checker can run into trouble with forward references... On the other hand I disagree that raising syntax errors is a con for parser - it stops the work early instead of saying "OK" to invalid code.

Depending on the intent of the typed_ast.parse() method user, parsing all type hints as code can be good or bad.

In my use case, I post-process the AST looking for annotations that are string literals, as well as type comments, I parse everything and substitute with original tree entries.

I agree that current method is safer, thus I've become un-confused :)

[gvanrossum]

Reporting multiple syntax errors is better than stopping at the first one.

The ast parser currently stops at the first one because I could never come up with a reasonable parser recovery algorithm and I didn't think it was worth it (25 years ago I expected most Python programs to be very short :-). I note that mypy's original parser does try to recover and it usually does a poor job.

But given that type comments are nicely delineated it makes sense to report syntax errors in each type comment rather than stopping at the first one.

The forward references are a red herring, name resolution is independent from parsing.

[mbdevpl]

I'd say that in Python many things are nicely delineated - that's why it's so easy to read. I agree that accurate error reporting is better. It's just that I think "keep it simple, stupid": if there is a syntax error at line 10, I first look at line 10. And even if at this point I knew that there is another syntax error at line 200, I forget about it until I fix my earlier code.

Putting forward references aside, if there are syntax errors in a given code, I'd think that parser should be the one to report them (be there one or many). Are you saying that leaving type comments as strings and leaving parsing them to some other tool is a good idea, or are you saying that parsing them in a subsequent parser run and accumulating potential errors is a good idea?

[gvanrossum]

I am saying that, pragmatically, I like the status quo.

[ddfisher]

I'm going to leave this as-is for now. Given the existence of forward references, I don't think there's a compelling reason to change the status quo.