`rustdoc --test --preserve-doctests --no-run` produces empty non-executable files

[jsgf]

I tried this code:

t.rs:

/// Fungle the foople.
/// ```
/// t::foople();
/// ```
pub fn foople() {}

/// Flomble the florp
/// ```
/// t::florp();
/// ```
pub fn florp() {}

mkdir doctests
rustc +nightly --crate-type rlib t.rs
rustdoc +nightly -Zunstable-options --test --persist-doctests doctests --no-run --extern t=libt.rlib t.rs
ls -lR doctests/*/*

Without --no-run, this generates (and runs) executables:

-rwxrwxr-x. 1 jsgf jsgf 3385848 Aug 17 00:50 doctests/t_rs_2_0/rust_out
-rwxrwxr-x. 1 jsgf jsgf 3385840 Aug 17 00:50 doctests/t_rs_8_0/rust_out

I'd expect the same with --no-run.
Instead, with --no-run I get empty non-executable files:

-rw-rw-r--. 1 jsgf jsgf 0 Aug 17 00:51 doctests/t_rs_2_0/rust_out
-rw-rw-r--. 1 jsgf jsgf 0 Aug 17 00:51 doctests/t_rs_8_0/rust_out

Meta

rustc --version --verbose:

rustc 1.56.0-nightly (2d2bc94c8 2021-08-15)
binary: rustc
commit-hash: 2d2bc94c8c3aa778e191f80261c726e4777439f1
commit-date: 2021-08-15
host: x86_64-unknown-linux-gnu
release: 1.56.0-nightly
LLVM version: 12.0.1

[jyn514]

The problem is that no_run has two meanings:

1. "don't run the generated binary"
2. "don't run any tests at all"

You expect it to have the first meaning, but rustdoc uses the second, as if you'd added no_run to the test directly. I do think the first meaning makes more sense - looking at #83857 it looks like that's what @ABouttefeux intended too and the code just happens to be wrong.

[jyn514]

Mentoring instructions:

rust/src/librustdoc/doctest.rs

Line 969 in 6d30039

let res = run_test(

is never being hit. Find out why and fix it (I suspect it's because of some option being passed to libtest).

[prconrad]

Mentoring instructions:

rust/src/librustdoc/doctest.rs

Line 969 in 6d30039

let res = run_test(

is never being hit. Find out why and fix it (I suspect it's because of some option being passed to libtest).

Actually, the test is being run. However, we're asking the compiler to only emit crate metadata.

rust/src/librustdoc/doctest.rs

Line 365 in 6d30039

compiler.arg("--emit=metadata");

Since we set the compiler output path the same way regardless of the --no-run flag, those output files are two totally different things. Without --no-run, it's the actual binary; with --no-run it's an empty metadata text file, not an empty binary.

Why is the file empty? Well, rustc doesn't appear to output any metadata for binary crates.

$ ls -la libtestcrate.rmeta
-rw-rw-r-- 1 user user 0 Aug 23 16:00 libtestcrate.rmeta
$ echo "pub fn main(){}" | rustc --crate-type="lib" --crate-name="testcrate" -o libtestcrate.rmeta --emit="metadata" -
$ ls -la libtestcrate.rmeta
-rw-rw-r-- 1 user user 1659 Aug 23 16:00 libtestcrate.rmeta

The simplest fix, therefore, is imply to remove the --emit argument so it always produces binaries, rather than using the faster metadata-only compilation. The current behavior would be less confusing if the output was set to a *.rmeta file, mirroring the convention for --emit="metadata".

I'm very new here, but I can try to make that PR and figure out how to test it, if there's interest.

[jyn514]

@prconrad can you only do that if --preserve-doctests is passed? Otherwise it will compile the whole binary and then immediately delete it, which isn't helpful.

[prconrad]

@prconrad can you only do that if --preserve-doctests is passed? Otherwise it will compile the whole binary and then immediately delete it, which isn't helpful.

@jyn514 : Yeah, easy enough. See PR: #88269