Tracking issue for RFC 1566: Procedural macros

[aturon]

Current Status

This issue has been closed in favor of more fine-grained tracking issues

~~~Updated Description~~~

• FCP request
• FCP check off
• All macros-1.2 related issue

Next steps:

• Stabilize use_extern_macros
  • waiting on crater
• Stabilize the proc_macro feature

Possible Stabilization Showstoppers

• Hygiene is weird
  • proposed solution
• Hygiene is weirder
  • disallow modules/macro definitions in output
• Supporting module system macros with macro_rules! and also having pre-1.29 compat is hard
  • current possible routes

Original Description

RFC.

This RFC proposes an evolution of Rust's procedural macro system (aka syntax
extensions, aka compiler plugins). This RFC specifies syntax for the definition
of procedural macros, a high-level view of their implementation in the compiler,
and outlines how they interact with the compilation process.

At the highest level, macros are defined by implementing functions marked with
a #[macro] attribute. Macros operate on a list of tokens provided by the
compiler and return a list of tokens that the macro use is replaced by. We
provide low-level facilities for operating on these tokens. Higher level
facilities (e.g., for parsing tokens to an AST) should exist as library crates.

Roadmap: #38356 (comment).

---

Tasks

• Implement #[proc_macro_attribute] (PR Implement #[proc_macro_attribute] #38842).
  • Fix feature(proc_macro) breaks attributes on custom derive #39347 (PR macros: fix inert attributes from proc_macro_derives with #![feature(proc_macro)] #39572).
  • Fix macros: don't cfg-process items before applying attribute proc macros #39336 (PR Apply attr proc macros before cfg processing #44528).
• Implement #[proc_macro] (PR Implement function-like procedural macros ( #[proc_macro]) #40129).
• Identify and collect uses of proc_macro_derives in the InvocationCollector (PR [WIP] Expand #[derive(..)]s in the InvocationCollector #39391).
• Support macro-expanded proc_macro_derive imports.
  • For example:

#[derive(Trait, OtherTrait)] struct S; // Both these derives should resolve
macro_rules! m { () => {
    #[macro_use(Trait)] extern crate derives;
    use derives::OtherTrait; // this kind of import is gated behind `#![feature(proc_macro)]`
} }
m!();

• Expand items before expanding applied proc_macro_derives (PR Expand items before their derives #48465).
• Implement warnings for unused #[macro_use] imports (PR Improve unused extern crate and unused #[macro_use] warnings #39060).
• Refactor the parser to consume token trees (PR Refactor the parser to consume token trees #39118).
• Clean up TokenStream in preparation for further refactoring (PR Refactor TokenStream #39173).
• Remove TokenTree::Sequence (PR Simplify TokenTree and fix macro_rules! bugs #39419).
• Use TokenStreams instead of Vec<TokenTree> in tokenstream::TokenTree's Delimited variant (PR syntax: integrate TokenStream #40202).
• Use Paths and TokenStreams in ast::Attributes (PR TokenStream-based attributes, paths in attribute and derive macro invocations #40346).
  • Support nontrivial paths in attribute/derive macros (e.g. #[foo::bar], #[derive(foo::Bar)]).
• Include hygiene information with all tokens, not just identifiers (PR macros: improve Span's expansion information #40597).
• Implement a minimal API for proc_macro::TokenStream as outlined in the RFC (PR proc_macro: implement TokenTree, TokenKind, hygienic quote!, and other API #40939).
  • Include source TokenStreams for interpolated AST fragments in Token::Interpolated tokens.
  • Include a TokenStream quoter proc_macro::quote! behind the proc_macro feature gate.
• Provide a way for proc_macro authors to create expansions that use items in a predetermined crate foo without requiring the macro user to include extern crate foo; at the crate root (PR proc_macro: implement TokenTree, TokenKind, hygienic quote!, and other API #40939).
  • Improve ergonomics.
• Include source TokenStreams for items in the AST.
• Stability check (proc-)macros (issue able to invoke unstable macro 2.0 #34079).
• Allow proc macro to initialize a private field with a def_site value (issue Allow proc macro to initialize a private field with a def_site value #47311). (PR macros: improve struct constructor field hygiene, fix span bug #48082)
• Inconsistency between accessing field of braced struct vs tuple struct in proc macro (issue Inconsistency between accessing field of braced struct vs tuple struct in proc macro #47312). (PR Improve tuple struct field access hygiene #48083)
• Make std available to proc macro root in phase 1 (issue Make std available to proc macro root in phase 1 #47314).
• Improve error from invalid syntax inside proc_macro::quote! (issue Improve error from invalid syntax inside proc_macro::quote! #47315).
• Inconsistency between Display and IntoIterator for a TokenStream containing a module (issue Inconsistency between Display and IntoIterator for a TokenStream containing a module #47627).
• #[cfg_attr] makes .to_string() and TokenStream disagree (issue Macros 2.0: #[cfg_attr] makes .to_string() and TokenStream disagree #48644).
• Wishlist for libproc_macro (checklist in Wishlist for libproc_macro #47786).

[aturon]

cc @nrc @jseyfried

[abonander]

I'd love for #[proc_macro_attribute] to be implemented soon. I already have a prototype and test usage that I banged out before realizing there's no compiler support yet 😒 :

Prototype: https://github.com/abonander/anterofit/blob/proc_macro/macros/src/lib.rs
Example/Test: https://github.com/abonander/anterofit/blob/proc_macro/examples/post_service_proc_macro.rs

[jseyfried]

Tasks

(dtolnay edit: moved the checklist up to the OP)

cc @nrc @petrochenkov @durka @Ralith

[abonander]

@jseyfried I ran into an issue where if a legacy macro and an attribute with the same name are imported into the same scope, trying to use the attribute throws an error that macros cannot be used as attributes. Can we make this work so that both can be in the same scope and can be used as intended?

[jseyfried]

@abonander All macros (bang, attribute, and derive) share the same namespace, so we can't use two different macros with the same name in the same scope. However, we could improve that error message -- could you open an issue?