Rustc hangs on recursive macro expansion on 1.48.0

[bobbobbio]

Code

We have a library that compiles fine on 1.47.0 but after uggrading to 1.48.0 the compilation hangs.

#![allow(unused)]

macro_rules! declare_nats {
    ($prev:ty) => {};
    ($prev:ty, $n:literal$(, $tail:literal)*) => {
        paste::item! {
            pub struct [<N $n>]($prev);
        }
        declare_nats!(Option<$prev>$(, $tail)*);
    };
    (0, $($n:literal),+) => {
        pub struct N0;
        declare_nats!(N0, $($n),+);
    };
}

declare_nats! {
    0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
    16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
    32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
    48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
    64
}

fn main() {}

This code-snippet above is reduced from the original code as much as I can. I'm having problems understanding exactly where the problem is coming from and any further modifications to reduce it haven't worked. It uses the "paste" crate, and I've tried a few different versions including the latest one but it doesn't make a difference .

I've been building it with cargo build at 1.47.0 and 1.48.0 to reproduce the issue which I got just by running rustup default

doesn't work on

rustc 1.48.0 (7eac88abb 2020-11-16)
binary: rustc
commit-hash: 7eac88abb2e57e752f3302f02be5f3ce3d7adfb4
commit-date: 2020-11-16
host: x86_64-unknown-linux-gnu
release: 1.48.0
LLVM version: 11.0

works on

rustc 1.47.0 (18bf6b4f0 2020-10-07)
binary: rustc
commit-hash: 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39
commit-date: 2020-10-07
host: x86_64-unknown-linux-gnu
release: 1.47.0
LLVM version: 11.0

[smmalis37]

Some quick notes as I find them:
This reproduces on just a cargo check, so has nothing to do with codegen (unsurprising)
Compilation time scales by a little more than 2x for each additional number added (going from 0 to 20 took 11 seconds, 21 took 26, 22 took 1 min 5)
Given that I don't think this is a hard hang per se, just a really really long running compilation
Definitely something in macro expansion, compiling the post-cargo expand code is instant
Reproduces on current nightly.
-Zself-profile shows all the time is spent in expand_crate.
Unfortunately I don't have a debug build of rustc handy, but I imagine slapping a profiler on one should make the issue fairly obvious. I'd recommend cutting the provided example down from 64 to 20 though.

[SNCPlay42]

searched nightlies: from nightly-2020-08-24 to nightly-2020-10-03
regressed nightly: nightly-2020-10-02
searched commits: from ef663a8 to 154f1f5
regressed commit: b218b95

bisected with cargo-bisect-rustc v0.6.0

Host triple: x86_64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --preserve --start 2020-08-24 --end 2020-10-03 --script ./test.sh 

Looks like #77153

[bobbygebert]

If it helps, this crate was exploring some ideas related to the typenum crate

[camelid]

Assigning P-high and removing I-prioritize as discussed in the prioritization working group.

[Aaron1011]

This is caused by some of the extra logic I added in #77153 to ensure that we don't throw out the captured TokenStream unnecessarily.

I'm looking into improving the performance. In the longer term, this will be solved by getting rid of the pretty-print/retokenize hack entirely.