Allow lexer to recover from some homoglyphs#62963
Merged
bors merged 2 commits intorust-lang:masterfrom Jul 26, 2019
Merged
Conversation
Contributor
|
r? @eddyb (rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
Centril
reviewed
Jul 25, 2019
Contributor
|
r? @Centril r=me rollup with nits fixed. |
eddyb
reviewed
Jul 25, 2019
eddyb
reviewed
Jul 25, 2019
| | ^ | ||
|
|
||
| error: aborting due to 2 previous errors | ||
| error[E0277]: cannot subtract `{integer}` from `{float}` |
Member
There was a problem hiding this comment.
Actually, I think we should do the substitution earlier, so it works in literals too, not just full tokens.
Contributor
There was a problem hiding this comment.
The Unicode stuff is legal inside string literals, so I'm not sure there's a single earlier point where this can be done.
Contributor
Author
There was a problem hiding this comment.
I think that's something I want to do (precisely for string literals) if and only if there is one unambiguous way of parsing it (like somebody copy-pasted code from a PDF and ends up with println!(‟Hello, world!”);, but doing so would require a more aggressive implementation with backtracking written in such a way that it doesn't affect the common case, and I don't see it would ever handling anything more complex than println!(‟It’s 💖”);. All of that I think should be done, but I'd want it to be part of a separate PR.
Contributor
|
r? @eddyb |
Member
petrochenkov
added
S-waiting-on-author
This comment has been minimized.
This comment has been minimized.
estebank
force-pushed
the
homoglyph-recovery
branch
from
cc5f7ab to
6844976
Compare
Contributor
|
@bors r+ |
Collaborator
|
📌 Commit 6844976 has been approved by |
bors
added
S-waiting-on-bors
Centril
added a commit
to Centril/rust
that referenced
this pull request
Jul 26, 2019
…ochenkov Allow lexer to recover from some homoglyphs
bors
added a commit
that referenced
this pull request
Jul 26, 2019
Rollup of 14 pull requests Successful merges: - #62084 (allow clippy::unreadable_literal in unicode tables) - #62421 (Introduce `as_deref` to Option) - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.) - #62801 (Remove support for -Zlower-128bit-ops) - #62828 (Remove vector fadd/fmul reduction workarounds) - #62862 (code cleanup) - #62897 (Attempt to fix backtrace tests on i686-msvc) - #62904 (Disable d32 on armv6 hf targets) - #62907 (Initialize the MSP430 AsmParser) - #62956 (Implement slow-path for FirstSets::first) - #62963 (Allow lexer to recover from some homoglyphs) - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar) - #62983 (Remove needless indirection through Rc) - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter) Failed merges: r? @ghost
Centril
added a commit
to Centril/rust
that referenced
this pull request
Jul 26, 2019
…ochenkov Allow lexer to recover from some homoglyphs
Centril
added a commit
to Centril/rust
that referenced
this pull request
Jul 26, 2019
…ochenkov Allow lexer to recover from some homoglyphs
bors
added a commit
that referenced
this pull request
Jul 26, 2019
Rollup of 22 pull requests Successful merges: - #62084 (allow clippy::unreadable_literal in unicode tables) - #62120 (Add missing type links in documentation) - #62310 (Add missing doc links in boxed module) - #62421 (Introduce `as_deref` to Option) - #62583 (Implement Unpin for all raw pointers) - #62692 (rustc: precompute the largest Niche and store it in LayoutDetails.) - #62801 (Remove support for -Zlower-128bit-ops) - #62828 (Remove vector fadd/fmul reduction workarounds) - #62862 (code cleanup) - #62904 (Disable d32 on armv6 hf targets) - #62907 (Initialize the MSP430 AsmParser) - #62956 (Implement slow-path for FirstSets::first) - #62963 (Allow lexer to recover from some homoglyphs) - #62964 (clarify and unify some type test names) - #62970 (ci: gate toolstate repo pushes on the TOOLSTATE_PUBLISH envvar) - #62980 (std: Add more accessors for `Metadata` on Windows) - #62983 (Remove needless indirection through Rc) - #62985 (librustc_errors: Support ui-testing flag in annotate-snippet emitter) - #63002 (error_index_generator should output stdout/stderr when it panics.) - #63004 (Add test for issue-54062) - #63007 (ci: debug network failures while downloading awscli from PyPI) - #63009 (Remove redundant `mut` from variable declaration.) Failed merges: r? @ghost
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.