macros: Make metavariables hygienic

[jseyfried]

This PR makes metavariables hygienic. For example, consider:

macro_rules! foo {
    ($x:tt) => { // Suppose that this token tree argument is always a metavariable.
        macro_rules! bar { ($x:expr, $y:expr) => { ($x, $y) } }
    }
}

fn main() {
    foo!($z); // This currently compiles.
    foo!($y); // This is an error today but compiles after this PR.
}

Today, the macro_rules! bar { ... } definition is only valid when the metavariable passed to foo is not $y (since it unhygienically conflicts with the $y in the definition of bar) or $x (c.f. #35450).

After this PR, the definition of bar is always valid (and bar!(a, b) always expands to (a, b) as expected).

This can break code that was allowed in #34925 (landed two weeks ago). For example,

macro_rules! outer {
    ($t:tt) => {
        macro_rules! inner { ($i:item) => { $t } }
    }
}

outer!($i); // This `$i` should not interact with the `$i` in the definition of `inner!`.
inner!(fn main() {}); // After this PR, this is an error ("unknown macro variable `i`").

Due to the severe limitations on nested macro_rules! before #34925, this is not a breaking change for stable/beta.

Fixes #35450.

r? @nrc

[jseyfried]

cc @durka @eddyb

[nrc]

@bors: r+

[bors]

📌 Commit cdbfe9f has been approved by nrc

[bors]

⌛ Testing commit cdbfe9f with merge 26f17ba...

[bors]

💔 Test failed - auto-win-gnu-64-opt

[durka]

LLVM didn't build?

[alexcrichton]

@bors: retry

• not sure...

[bors]

⌛ Testing commit cdbfe9f with merge 70bb60c...

[bors]

💔 Test failed - auto-win-gnu-32-opt-rustbuild

[durka]

 error[E0308]: mismatched types
   --> C:\bot\slave\auto-win-gnu-32-opt-rustbuild\build\src/test\run-pass-fulldeps\auxiliary\procedural_mbe_matching.rs:39:22
    |
 39 |             match (&*map[&str_to_ident("matched").name], &*map[&str_to_ident("pat").name]) {
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `syntax::ast::Ident`, found struct `syntax::ast::Name`
    |
    = note: expected type `&syntax::ast::Ident`
    = note:    found type `&syntax::ast::Name`

 error[E0308]: mismatched types
   --> C:\bot\slave\auto-win-gnu-32-opt-rustbuild\build\src/test\run-pass-fulldeps\auxiliary\procedural_mbe_matching.rs:39:60
    |
 39 |             match (&*map[&str_to_ident("matched").name], &*map[&str_to_ident("pat").name]) {
    |                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `syntax::ast::Ident`, found struct `syntax::ast::Name`
    |
    = note: expected type `&syntax::ast::Ident`
    = note:    found type `&syntax::ast::Name`

 error: aborting due to 2 previous errors

[jseyfried]

@durka Thanks!
@bors r+

[bors]

📌 Commit b62ff25 has been approved by jseyfried

[bors]

📌 Commit b62ff25 has been approved by nrc

[bors]

📌 Commit 95b68aa has been approved by nrc

[durka]

So... this is a syntax-breaking change if there is test fallout (sorry).

[eddyb]

cc @Manishearth

[jseyfried]

@durka Good point.
@bors r- (unless @Manishearth's says this isn't actually a syntax-[breaking-change] in practice)

[Manishearth]

I think so. @dtolnay whatsay?

[Manishearth]

We're gettign a pile of PRs that will break aster (and only aster) right now though. So it might be good to do a rollup. OTOH there's a bunch of diagnostics work coming up which I'd like to include.

[dtolnay]

This PR does not affect quasi/aster. For the others - I have time in the next two weeks to deal with fallout so this would be a good time. Thanks for the heads up.

[Manishearth]

Cool. @GuillaumeGomez could you get all libsyntax refactors necessary for diagnostics in now?

[jseyfried]

@bors r=nrc

[bors]

📌 Commit 95b68aa has been approved by nrc

[bors]

⌛ Testing commit 95b68aa with merge f6ba0ad...

[alexcrichton]

@bors: retry force clean

• restarted buildbot

[bors]

⌛ Testing commit 95b68aa with merge eec30ea...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-cargotest, auto-linux-64-cross-freebsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-no-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-no-mir, auto-win-msvc-64-opt-rustbuild
Approved by: nrc
Pushing eec30ea to master...