Rustdoc --no-run runs when --edition=2024 is provided

[TroyKomodo]

I tried this code:

// test the behavior of the --no-run flag

//@ check-pass
//@ compile-flags:-Z unstable-options --test --no-run --edition 2024 --test-args=--test-threads=1
//@ normalize-stdout: "tests/rustdoc-ui/doctest" -> "$$DIR"
//@ normalize-stdout: "finished in \d+\.\d+s" -> "finished in $$TIME"

/// ```
/// let a = true;
/// ```
/// ```should_panic
/// panic!()
/// ```
/// ```ignore (incomplete-code)
/// fn foo() {
/// ```
/// ```no_run
/// loop {
///     println!("Hello, world");
/// }
/// ```
/// fails to compile
/// ```compile_fail
/// let x = 5;
/// x += 2; // shouldn't compile!
/// ```
/// Ok the test does not run
/// ```
/// panic!()
/// ```
/// Ok the test does not run
/// ```should_panic
/// loop {
///     println!("Hello, world");
/// panic!()
/// }
/// ```
pub fn f() {}

Meta

rustdoc --version --verbose:

rustdoc 1.88.0 (6b00bc388 2025-06-23)
binary: rustdoc
commit-hash: 6b00bc3880198600130e1cf62b8f8a93494488cc
commit-date: 2025-06-23
host: x86_64-unknown-linux-gnu
release: 1.88.0
LLVM version: 20.1.5

[TroyKomodo]

2024

troy@troy-server ~/g/s/rustdoc-test (master)> RUSTC_BOOTSTRAP=1 rustdoc -Z unstable-options --test --no-run --edition 2024 --crate-type lib test.rs        python314Full 

running 6 tests
test test.rs - f (line 17) - compile ... ok
test test.rs - f (line 8) ... ok
test test.rs - f (line 28) ... FAILED
test test.rs - f (line 32) - should panic ... ok
test test.rs - f (line 11) - should panic ... ok
test test.rs - test_no_run (line 40) ... FAILED

failures:

---- test.rs - f (line 28) stdout ----
Test executable failed (exit status: 101).

stderr:

thread 'main' panicked at /tmp/nix-shell-1123364-0/rustdoctestOcs7PD/doctest_bundle_2024.rs:24:1:
explicit panic
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


---- test.rs - test_no_run (line 40) stdout ----
Test executable failed (exit status: 101).

stderr:

thread 'main' panicked at /tmp/nix-shell-1123364-0/rustdoctestOcs7PD/doctest_bundle_2024.rs:51:1:
assertion failed: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace



failures:
    test.rs - f (line 28)
    test.rs - test_no_run (line 40)

test result: FAILED. 4 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s


running 2 tests
test test.rs - f (line 14) ... ignored
test test.rs - f (line 23) - compile fail ... ok

test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.01s

troy@troy-server ~/g/s/rustdoc-test (master) [101]>           

2021

troy@troy-server ~/g/s/rustdoc-test (master) [101]> RUSTC_BOOTSTRAP=1 rustdoc -Z unstable-options --test --no-run --edition 2021 --crate-type lib test.rs  python314Full 

running 8 tests
test test.rs - f (line 14) ... ignored
test test.rs - f (line 11) - compile ... ok
test test.rs - f (line 28) - compile ... ok
test test.rs - test_no_run (line 40) - compile ... ok
test test.rs - f (line 23) - compile fail ... ok
test test.rs - f (line 8) - compile ... ok
test test.rs - f (line 17) - compile ... ok
test test.rs - f (line 32) - compile ... ok

test result: ok. 7 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.02s

troy@troy-server ~/g/s/rustdoc-test (master)>      

[TroyKomodo]

Weirder still passing any --test-runtool arg makes the test not run adding --test-runtool=/dev/null causes the tests to not run. I am mistaken it still runs however this can be used to by pass it if we just point it to a script that always returns 0

[GuillaumeGomez]

Taking a look.

[GuillaumeGomez]

Opened #143900.