Make some of lexer's API private#59671
Conversation
|
r? @eddyb (rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
The API feels like it predates iterators, in some sense, it should probably be streamlined. |
|
For example, the peeking should ideally be handled with Anyway, this seems like a good start. @bors r+ |
|
📌 Commit 1763aea has been approved by |
bors
added
S-waiting-on-bors
Make some of lexer's API private Lexer is a `pub` type, so it feels wrong that its fields are just pub (I guess it wasn't exported initially), so let's minimize visibility. Context: I am looking into extracting rust-lexer into a library, which can be shared by rust-analyzer and rustc. I hope that a simple interface like `fn next_token(src: &str) -> (TokenKind, usize)` would work, but to try this out I need to understand what is the current API of the lexer.
|
@eddyb yeah, I feel like this could be improved massively by separating the actual lexing from rustc-specific bits like spans and source maps. See https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/Extracting.20the.20lexer/near/162515922 for the rough proposal |
Rollup of 8 pull requests Successful merges: - #59470 (Document std::fs::File close behavior ignoring errors) - #59555 (update miri) - #59556 (update stdsimd) - #59596 (Forward formatter settings to bounds of `Range<T>` in `fmt::Debug` impl) - #59639 (Never return uninhabited values at all) - #59671 (Make some of lexer's API private) - #59685 (Add description for -Os and -Oz in rustc.1) - #59686 (Temporarily disable stack probing for gnux32.) Failed merges: r? @ghost
|
Why is bors testing this and #59556? It is already merged. |
|
@bors r- retry |
bors
added
S-waiting-on-author
8 participants
Lexer is a
pubtype, so it feels wrong that its fields are just pub (I guess it wasn't exported initially), so let's minimize visibility.Context: I am looking into extracting rust-lexer into a library, which can be shared by rust-analyzer and rustc. I hope that a simple interface like
fn next_token(src: &str) -> (TokenKind, usize)would work, but to try this out I need to understand what is the current API of the lexer.