What is a trailing expression in a block exactly?

[petrochenkov]

Is it determined syntactically or semantically?
Before or after macro expansion?

Answering these questions is necessary to specify expansion of macros (stable fn-like ones or unstable attribute ones) in expression and statement positions.
The current implementation is sometimes inconsistent.

Below I'll be dumping some code examples expanded using different expansion models in hope to come up with some rules that are both self-consistent and backward compatible.

cc #33953

[petrochenkov]

Macros used in the examples:

macro empty() {}
macro expr() { 0 }
macro stmt() { 0 ; }
macro stmt_expr() { 0 ; 1 }

and equivalent attribute macros producing the same tokens.

Definitions:

"Token-based expansion" - macro invocation tokens are replaced by tokens produced by the macro, without knowing anything about AST-based context of that macro invocation (whether it's a macro in item position, or in expression position, etc).

Having an expansion model like this is necessary if we want to perform eager expansion.
Suppose that we somehow identified the macro invocation stmt!() in the next token stream:

0 . id 2 stmt!() ; bar +

To expand it we replace it with its produced tokens without having any idea about the context

0 . id 2 stmt!() ; bar +

=>

0 . id 2 0 ; ; bar +

"AST-based expansion" - macro invocation knows its AST node kind (whether it's a macro expression, or a macro item, or something else) and its produced tokens (+ perhaps some other neighbouring tokens) are immediately (re-)parsed as one or multiple AST nodes of that kind, then the new nodes replace the old node.
That's how current expansion in the compiler works, more or less.

stmt!(); // Statement kind

=> 

0 ; // Produced tokens, parsed as a statement `0;`

=>

0; // The parsed statement replaces the previous statement `stmt!();`

[petrochenkov]

fn main() {
    { 2 }

    empty!();
}

Token-based expansion:

fn main() {
    { 2 } // <- trailing expression if only "output(empty!())  ;" is re-parsed after the macro expansion
           // (assuming empty statements are ignored when determining trailing-ness)
           // <- not a trailing expression if some larger context is re-parsed after the macro expansion

    ; // An "empty statement" if only "output(empty!())  ;" is re-parsed after the macro expansion
      // A part of "{ 2 };" statement if some larger context is re-parsed after the macro expansion
}

AST-based expansion:

fn main() {
    { 2 } // <- trailing expression

    // Stmt( empty!(); ) -> [/* empty stmt list */]
    /* Nothing */
}

Compiler's behavior:
{ 2 } is a trailing expression.

[petrochenkov]

fn main() {
    { 2 }

    expr!();
}

Token-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    0 ; // A non-empty statement, not a trailing expression
}

AST-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    // Stmt( expr!(); ) -> [Stmt(0;)]
    0; // A non-empty statement, not a trailing expression

    // ^^^ Assuming "output(expr!()) ;" is re-parsed and not only "output(expr!())".
    // In the second case ";" would be a separate empty statement, and "0" would be a trailing expression 
}

Compiler's behavior:
{ 2 } is not a trailing expression, 0 is not a trailing expression.

[petrochenkov]

fn main() {
    { 2 }

    stmt!();
}

Token-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    0 ; ; // A non-empty statement not a trailing expression, followed by empty statement
}

AST-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    // Stmt( stmt!(); ) -> [Stmt(0;)]
    0; // A non-empty statement, not a trailing expression
}

Compiler's behavior:
{ 2 } is not a trailing expression, 0 is not a trailing expression.

[petrochenkov]

fn main() {
    { 2 }

    stmt_expr!();
}

Token-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    0 ; 1 ; // A non-empty statement not a trailing expression, followed by a non-empty statement not a trailing expression
}

AST-based expansion:

fn main() {
    { 2 } // <- not a trailing expression

    // Stmt( stmt_expr!(); ) -> [Stmt(0;), Stmt(1;)]
    0; 1; // A non-empty statement not a trailing expression, followed by a non-empty statement not a trailing expression
}

Compiler's behavior:
{ 2 } is not a trailing expression, 1 is not a trailing expression.

[petrochenkov]

fn main() {
    { 2 }

    empty!()
}

Token-based expansion:

fn main() {
    { 2 } // <- trailing expression

    /* Nothing */
}

AST-based expansion 1 (empty!() has statement kind):

fn main() {
    { 2 } // <- trailing expression

    // Stmt( empty!() ) -> [/* empty stmt list */]
    /* Nothing */
}

AST-based expansion 2 (empty!() has expression kind):

fn main() {
    { 2 } // N/A

    // Expr( empty!() ) -> ParseError (empty token stream is not an expression)
    /* Nothing */
}

Compiler's behavior:
Parse error error: macro expansion ends with an incomplete expression: expected expression.

[petrochenkov]

fn main() {
    { 2 }

    expr!()
}

Token-based expansion:

fn main() {
    { 2 } // not a trailing expression

    0 // trailing expression
}

AST-based expansion 1 (expr!() has statement kind):

fn main() {
    { 2 } // not a trailing expression

    // Stmt( expr!() ) -> [Stmt(0)]
    0 // trailing expression
}

AST-based expansion 2 (expr!() has expression kind):

fn main() {
    { 2 } // not a trailing expression

    // Expr( expr!() ) -> Expr(0)
    0 // trailing expression
}

Compiler's behavior:
{ 2 } is not a trailing expression, 0 is a trailing expression.

[petrochenkov]

fn main() {
    { 2 }

    stmt!()
}

Token-based expansion:

fn main() {
    { 2 } // not a trailing expression

    0 ; // not a trailing expression
}

AST-based expansion 1 (stmt!() has statement kind):

fn main() {
    { 2 } // not a trailing expression

    // Stmt( stmt!() ) -> [Stmt(0;)]
    0; // not a trailing expression
}

AST-based expansion 2 (stmt!() has expression kind):

fn main() {
    { 2 } // not a trailing expression

    // Expr( stmt!() ) -> ParseError (`0;` is not an expression)
    /* nothing */
}

Compiler's behavior:
{ 2 } is not a trailing expression, 0 is a trailing expression.
This doesn't match any model and is clearly a bug (#33953).

[petrochenkov]

fn main() {
    { 2 }

    stmt_expr!()
}

Token-based expansion:

fn main() {
    { 2 } // not a trailing expression

    0 ; 1 // trailing expression
}

AST-based expansion 1 (stmt_expr!() has statement kind):

fn main() {
    { 2 } // not a trailing expression

    // Stmt( stmt_expr!() ) -> [Stmt(0;), Stmt(1)]
    0; 1 // trailing expression
}

AST-based expansion 2 (stmt_expr!() has expression kind):

fn main() {
    { 2 } // not a trailing expression

    // Expr( stmt_expr!() ) -> ParseError (`0; 1` is not an expression)
    /* nothing */
}

Compiler's behavior:
Parse error: error: macro expansion ignores token 1 and any following

[petrochenkov]

For attribute macros token-based and AST-based interpretations are the same as for fn-like macros, so I'm just going to list the actual compiler's behaviors.

cfg executes separate compiler's logic from other attribute macros, so it's tested here separately.

fn main() {
    { () } // trailing expression

    #[cfg(FALSE)]
    0; // not trailing expression
}

fn main() {
    { () } // trailing expression

    #[cfg(FALSE)]
    0 // not trailing expression
}

fn main() {
    { () } // not trailing expression

    #[cfg(TRUE)]
    0; // not trailing expression
}

fn main() {
    { () } // not trailing expression

    #[cfg(TRUE)]
    0 // trailing expression
}