rustdoc: Add support for --remap-path-prefix

[edward-shen]

Adds --remap-path-prefix as an unstable option. This is implemented to mimic the behavior of rustc's --remap-path-prefix.

This flag similarly takes in two paths, a prefix to replace and a replacement string.

This is useful for build tools (e.g. Buck) other than cargo that can run doc tests.

cc: @dtolnay

[rustbot]

r? @GuillaumeGomez

(rustbot has picked a reviewer for you, use r? to override)

[edward-shen]

@rustbot ready

[GuillaumeGomez]

The code looks good to me, but it comes a bit out of the blue so needs the rustdoc team to agree on it (very likely through FCP).

But first, what do you think @rust-lang/rustdoc ?

[edward-shen]

but it comes a bit out of the blue so needs the rustdoc team to agree on it

My bad! I didn't think this would be a substantial change since rustc already had this flag. For future reference, should I have dropped in the zulip before the implementation?

[GuillaumeGomez]

At least opening an issue to explain your needs so it makes it easier for everyone to reach a solution. If this PR ends up being rejected, you'd have worked for nothing. :-/

[edward-shen]

Changes:

• rebased
• Realized I made a function pub for a previous implementation attempt. It no longer needed to be pub so I removed it
• Fixed copy-paste error in ui test.

[edward-shen]

At least opening an issue to explain your needs so it makes it easier for everyone to reach a solution. If this PR ends up being rejected, you'd have worked for nothing. :-/

Got it--I'll be sure to do so in the future. Sorry again, and thank you for the gentle explanation!

[dtolnay]

@GuillaumeGomez As far as I can tell from the reference to Buck (which I am familiar with) the point of --remap-path-prefix for rustdoc is to get diagnostics inside of doctests to refer to the source directory tree of the build, not the build system's directory tree. For example without --remap-path-prefix:

---- buck-out/v2/gen/fbcode/882bd00a4ffb1e1d/tracing/stats/__tracing_stats__/__srcs/src/lib.rs - StatsLayer (line 70) stdout ----
error[E0433]: failed to resolve: use of undeclared type `StatsLayer`
 --> buck-out/v2/gen/fbcode/882bd00a4ffb1e1d/tracing/stats/__tracing_stats__/__srcs/src/lib.rs:71:13
  |
3 | let stats = StatsLayer::new("myservice", 8, false)
  |             ^^^^^^^^^^ not found in this scope
  |
help: consider importing this struct
  |
2 | use tracing_stats::StatsLayer;
  |

The build system would want to use --remap-path-prefix buck-out/v2/gen/fbcode/882bd00a4ffb1e1d/tracing/stats/__tracing_stats__/__srcs/=tracing/stats/ to get:

---- tracing/stats/src/lib.rs - StatsLayer (line 70) stdout ----
error[E0433]: failed to resolve: use of undeclared type `StatsLayer`
 --> tracing/stats/src/lib.rs:71:13
  |
3 | ...

It's a common technique for mature build systems to separate the files that get fed to build steps apart from the files that get touched by the programmers. There are many reasons for this; not all of them apply to every build system that uses this technique.

• Determinism, no race conditions: if a user, or IDE, is mutating the source files during the time that a build is already running, the build system can still be 100% sure exactly which iteration of the input files the output of the build was performed against, and the exact delta to the subsequent build. It's not subject to a race condition between when the compiler ends up reading a file vs when the editor writes the file, which would be outside the build system's ability to inspect (leaving aside some kind of kernel module or sophisticated filesystem or syscall interceptor).

• Hermeticity: the build system can sandbox build steps to keep them from reaching into arbitrary filesystem state that is not tracked by the build system's understanding of the precise inputs of every step.

• Integration of code generators: for source files which are generated by code generators during the course of the build (such as protoc or thriftc or bindgen), the build system can map them to easily understood siblings of the crate root, for example using Buck's mapped_srcs and genrule. This is a lot nicer than Cargo's include!(concat!(env!("OUT_DIR" …)) thing.

• Remote execution: build steps in a large build would typically not run on the same machine as the source files are being written on. In general the filesystem structure on the remote build environment are not identical to the filesystem paths on the developer's machine, but the build system is responsible for mapping between them.

• Distributed caching: sharing build step output artifacts (including diagnostics/warnings, not just rlibs/rmeta/binaries) across different devs, or with a cache which is kept perpetually warm by CI. Again different machines could have different path structure (even different OS, as long as at least 1 is cross-compiling).

Edward or I would be happy to elaborate on any of the above if needed.

[bors]

☔ The latest upstream changes (presumably #108096) made this pull request unmergeable. Please resolve the merge conflicts.

[KittyBorgX]

@rustbot author
@edward-shen Hey! It looks like there are some merge conflicts in your PR. Can you sort them out?
Once you're done type in @rustbot review here so that the label can be changed to S-waiting-on-review and your PR should be good for a review!

[edward-shen]

Hey @KittyBorgX, thanks for the ping. Work has been busy, so the conflicts message slipped through the cracks.

Rebased and fixed merge conflicts.

@rustbot review

[matthiaskrgr]

@bors r-
#125725 (comment)

[bors]

☔ The latest upstream changes (presumably #125798) made this pull request unmergeable. Please resolve the merge conflicts.

[rustbot]

Some changes occurred in run-make tests.

cc @jieyouxu

[edward-shen]

Changes:

• Rebased on top of the submodule re-org
• Looks like the Windows failure in the rollup is because they use "exit code" instead of "exit status". I chose to normalize the wording code/status in the unit test to fix this, as panicking in the unit test also reveals a remapped location to test for.

[edward-shen]

Oops, @rustbot ready

[GuillaumeGomez]

Let's try again!

@bors r+ rollup

[bors]

📌 Commit d9f78cb has been approved by GuillaumeGomez

It is now in the queue for this repository.

[bors]

⌛ Testing commit d9f78cb with merge 6d94a87...

[bors]

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing 6d94a87 to master...

[rust-timer]

Finished benchmarking commit (6d94a87): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (primary 2.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Cycles

Results (secondary -2.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.3%	[-2.3%, -2.3%]	1
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: missing data
Artifact size: 319.16 MiB -> 319.73 MiB (0.18%)