Skip to content

Fix parsing logic in proc_macro::quote - #148209

Open
moatom wants to merge 1 commit into
rust-lang:mainfrom
moatom:stabilize-proc-macro
Open

Fix parsing logic in proc_macro::quote#148209
moatom wants to merge 1 commit into
rust-lang:mainfrom
moatom:stabilize-proc-macro

Conversation

@moatom

@moatom moatom commented Oct 28, 2025 •
edited by rustbot
Loading

Copy link
Copy Markdown
Contributor

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Oct 28, 2025
@rust-log-analyzer

This comment has been minimized.

Sign in to view
bjorn3
bjorn3 reviewed Oct 28, 2025
View reviewed changes
Comment thread library/proc_macro/src/quote.rs Outdated
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@moatom
moatom force-pushed the stabilize-proc-macro branch from 114e740 to 02f04b0 Compare October 30, 2025 14:12
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@moatom
moatom force-pushed the stabilize-proc-macro branch from 02f04b0 to 0e341e2 Compare October 30, 2025 15:13
@moatom moatom changed the title WIP: Fix parseing in proc_macro::quote Fix parsing in proc_macro::quote Oct 30, 2025
@moatom
moatom marked this pull request as ready for review October 30, 2025 15:14
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 30, 2025
@rustbot rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Oct 30, 2025
@rustbot

rustbot commented Oct 30, 2025

Copy link
Copy Markdown
Collaborator

r? @petrochenkov

rustbot has assigned @petrochenkov.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@moatom

moatom commented Oct 30, 2025

Copy link
Copy Markdown
Contributor Author

r? @tgross35

@rustbot rustbot assigned tgross35 and unassigned petrochenkov Oct 30, 2025
@rustbot

rustbot commented Oct 30, 2025

Copy link
Copy Markdown
Collaborator

tgross35 is currently at their maximum review capacity.
They may take a while to respond.

@moatom moatom changed the title Fix parsing in proc_macro::quote Fix parsing logic in proc_macro::quote Oct 30, 2025
@moatom
moatom force-pushed the stabilize-proc-macro branch 2 times, most recently from 8dc03cd to 36c13e9 Compare October 30, 2025 15:27
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@tgross35

tgross35 commented Oct 30, 2025

Copy link
Copy Markdown
Member

David knows this much better than I do

r? @dtolnay

@rustbot rustbot assigned dtolnay and unassigned tgross35 Oct 30, 2025
@moatom

moatom commented Oct 31, 2025

Copy link
Copy Markdown
Contributor Author

@bors r+

@bors

bors commented Oct 31, 2025

Copy link
Copy Markdown
Collaborator

@moatom: 🔑 Insufficient privileges: Not in reviewers

@tgross35

tgross35 commented Oct 31, 2025

Copy link
Copy Markdown
Member

@bors r+

@moatom what are you trying to do here? This is the command to approve PRs.

@moatom

moatom commented Oct 31, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@tgross35 thx. I wanted to restart CI just in case because this PR compiles locally.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 4, 2025
@moatom

moatom commented Nov 6, 2025

Copy link
Copy Markdown
Contributor Author

@dtolnay
Thank you for the review!
I've addressed your comments and left the fix commit separate for traceability.
Please let me know if it's okay to squash.

@moatom
moatom requested a review from dtolnay November 6, 2025 15:31
@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 6, 2025
dtolnay
dtolnay reviewed Nov 10, 2025
View reviewed changes
Comment thread tests/ui/proc-macro/quote/not-quotable-repetition-group-bracket.rs Outdated

fn main() {
let arr = [1, 2, 3];
let _ = quote! { ${$arr}* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]

@dtolnay dtolnay Nov 10, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let _ = quote! { ${$arr}* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]
let _ = quote! { $[$arr]* }; //~ ERROR the trait bound `[{integer}; 3]: ToTokens` is not satisfied [E0277]

dtolnay
dtolnay reviewed Nov 10, 2025
View reviewed changes
Comment thread library/proc_macro/src/quote.rs Outdated
match &tree {
TokenTree::Punct(tt) if tt.as_char() == '$' => {
if let Some(TokenTree::Ident(id)) = iter.peek() {
while let Some(ref tree) = iter.next() {

@dtolnay dtolnay Nov 10, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to miss adjacent metavariables. For example this works:

let a = ['a'];
let b = ['b'];
eprintln!("{}", proc_macro::quote!($($a . $b)*));

but this does not:

eprintln!("{}", proc_macro::quote!($($a $b)*));

@moatom moatom Nov 18, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[note] This was caused by an extra iter.next();: b2ae84c#diff-6ebaa6be050b7cea1d542579c11cde3cba0942a3417c53298001f9db02a8d06dL407

dtolnay
dtolnay reviewed Nov 10, 2025
View reviewed changes
Comment thread library/proc_macro/src/quote.rs
}

/// Consume a `$( CONTENTS ) SEP *` accordingly. It handles repetition by expanding `$( CONTENTS ) SEP *` to `{ REP_EXPANDED }`.
fn consume_dollar_group_sep_star(

@dtolnay dtolnay Nov 10, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is accepting too many tokens in the "separator" in some cases.

let arr = ['a', 'b'];
eprintln!("{}", proc_macro::quote!($($arr) $$ x .. false [] *));
'a' $x .. false [] 'b'

@moatom moatom Nov 18, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced is_valid_sep_prefix to resolve this, but there might be a more elegant solution.

@moatom
moatom requested a review from dtolnay November 18, 2025 17:36
@moatom
moatom force-pushed the stabilize-proc-macro branch from abc4763 to ae0617b Compare November 18, 2025 17:57
@moatom

moatom commented Nov 18, 2025

Copy link
Copy Markdown
Contributor Author

r? @dtolnay

@rustbot

rustbot commented Nov 18, 2025

Copy link
Copy Markdown
Collaborator

Requested reviewer is already assigned to this pull request.

Please choose another assignee.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@moatom
moatom force-pushed the stabilize-proc-macro branch from ae0617b to 93fd70a Compare November 19, 2025 00:18
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@moatom
moatom force-pushed the stabilize-proc-macro branch 3 times, most recently from 88f4fbe to 3613f22 Compare November 19, 2025 00:50
@moatom
moatom force-pushed the stabilize-proc-macro branch from 3613f22 to 62534aa Compare November 26, 2025 16:59
@rustbot

rustbot commented Nov 26, 2025

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@moatom

moatom commented Nov 26, 2025

Copy link
Copy Markdown
Contributor Author

@dtolnay

I've addressed your comments and left the fix commit separate for traceability.
Please let me know if it's okay to squash.

I've now rebased and squashed the commits for smoother communication.
If you still need the detailed history, please take a look at moatom:stabilize-proc-macro-saved.

moatom
moatom commented Nov 26, 2025
View reviewed changes
Comment thread library/proc_macro/src/quote.rs
consume_dollar_group_sep_star(tt.stream(), &mut iter).to_tokens(&mut tokens);
continue;
}
TokenTree::Group(_) => {

@moatom moatom Nov 26, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

output_dollar_group_sep seems available here too.

@apiraino apiraino added T-libs Relevant to the library team, which will review and decide on the PR/issue. and removed T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 14, 2026
@kohsine kohsine mentioned this pull request Sep 10, 2026
Open
@tgross35

tgross35 commented Sep 10, 2026

Copy link
Copy Markdown
Member

Been a little while,

r? libs

@rustbot rustbot assigned JohnTitor and unassigned dtolnay Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bjorn3 bjorn3 bjorn3 left review comments
@dtolnay dtolnay Awaiting requested review from dtolnay

Requested changes must be addressed to merge this pull request.

Assignees

@JohnTitor JohnTitor

Labels

S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add support for repetition to proc_macro::quote

10 participants

@moatom @rust-log-analyzer @rustbot @tgross35 @bors @dtolnay @bjorn3 @petrochenkov @apiraino @JohnTitor