Skip to content

fix: exclude abspath of path deps from -Cmetadata hash - #17624

Merged
ranger-ross merged 5 commits into
rust-lang:masterfrom
weihanglo:trim-paths-c-metadata
Oct 24, 2026
Merged

ranger-ross merged 5 commits into
rust-lang:masterfrom
weihanglo:trim-paths-c-metadata

Conversation

@weihanglo

@weihanglo weihanglo commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

What does this PR try to resolve?

-Cmetadata is for rustc to uniquely identify different crates in a
crate graph. Without trim-paths, the binaries embeds absolute paths
anyway. Dropping the path from -Cmetadata doesn't make any difference.

Now we trim-paths we can aggressively drop those absolute paths,
so that binaries are now more reproducible.

Notes to reviewers

See #13586 (comment).

And I believe if this got merged it fixes #13586 anyway.

This doesn't really help caching (yet),
as unit_id hash still hold those absolute paths (and may need checksum-freshness to make it work)


🤖 LLM involvement: Tests were initially generated though LLM did pretty bad at parameterizing those so I rewrote quite a bit.

@weihanglo weihanglo added the A-reproducibility Area: reproducible / deterministic builds label Oct 23, 2026
@rustbot rustbot added the A-layout Area: target output directory layout, naming, and organization label Oct 23, 2026
@weihanglo
weihanglo force-pushed the trim-paths-c-metadata branch 2 times, most recently from 8ab48d0 to 043aba3 Compare October 23, 2026 13:10
@weihanglo

Copy link
Copy Markdown
Member Author

r? ranger-ross

What do you think?

@weihanglo
weihanglo marked this pull request as ready for review October 23, 2026 13:11
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 23, 2026
@weihanglo
weihanglo force-pushed the trim-paths-c-metadata branch from 043aba3 to e367eb4 Compare October 23, 2026 14:03
@ranger-ross

Copy link
Copy Markdown
Member

I am trying to understand what the risks of this change are.
Checking the -Cmetadata doc as I understand it, rustc uses this to create a hash so that it can tell two version of the same crate apart.

So the risk is that we no longer track absolute file paths outside of the workspace. So if that file changes we could end up with 2 rlibs with the same symbol mangling and rustc would not be happy. I am assuming it would error.
Is that correct?

I guess we could add a level of safety by hashing the file contents, but that would probably become a rabbit hole of its own. (if that file pulled in more source files or if it contained absolute paths)
So excluding them seems reasonable to me.

@weihanglo

Copy link
Copy Markdown
Member Author

So the risk is that we no longer track absolute file paths outside of the workspace. So if that file changes we could end up with 2 rlibs with the same symbol mangling and rustc would not be happy. I am assuming it would error.

It depends on whether there is a rebuild triggered. If yes, and rustc would error. For transitive deps from -L (if in legacy layout of course), SVH still checks deps' metadata so rustc would reject as well.

But anyway we have absolute path in unit_id so the extra filename is different anyway. In the future, when we make unit_id also absolute-path free, we'll need the content checksum fingerprint anyway to avoid misreuse.

@weihanglo

Copy link
Copy Markdown
Member Author

The future risk if we ever trim abspath from unit_id : two different path deps with the same name and version, at different locations, map to one cache entry.

And here are what may happen:

  • Nothing recompile: as everything looks fresh.
  • Something recompile against the wrong cache, under the same API: rustc links the wrong code with different behavior (I guess)
  • Something recompile against the wrong cache, under a different API: rustc emits a error like "function not found"

If the blob store / unit_id cache key use content checksum it will mitigate the risk.

Comment on lines +2154 to +2162
# See https://github.com/rust-lang/rust/pull/158873
[target.'cfg(target_env = "msvc")']
rustflags = ["-Clink-arg=/Brepro"]
# On Windows GNU the PE format insert a timestamp by default
# so that we need to opt-out.
# See https://sourceware.org/binutils/docs/ld.html#index-_002d_002dinsert_002dtimestamp
[target.'cfg(all(target_os = "windows", target_env = "gnu"))']
rustflags = ["-Clink-arg=-Wl,--no-insert-timestamp"]

@weihanglo weihanglo Oct 23, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am surprised (but not too surprised) that we need to that much to achieve reproducibility on Windows targets

View changes since the review

@weihanglo

Copy link
Copy Markdown
Member Author

`-Cmetadata` is for rustc to uniquely identify different crates in a
crate graph. Without `trim-paths`, the binaries embeds absolute paths
anyway. Dropping the path from `-Cmetadata` doesn't make any difference.

Now we `trim-paths` we can aggressively drop those absolute paths,
so that binaries are now more reproducible.
@weihanglo
weihanglo force-pushed the trim-paths-c-metadata branch from e367eb4 to 20319b9 Compare October 24, 2026 12:17
@rustbot

rustbot commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@ranger-ross ranger-ross left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Makes sense.
Glad to see yet another step towards more reproducible builds :D

View changes since this review

@ranger-ross
ranger-ross added this pull request to the merge queue Oct 24, 2026
Merged via the queue into rust-lang:master with commit 8bdfbfc Oct 24, 2026
30 checks passed
@rustbot rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 24, 2026
@weihanglo
weihanglo deleted the trim-paths-c-metadata branch October 24, 2026 14:43
rust-bors Bot pushed a commit to rust-lang/rust that referenced this pull request Oct 29, 2026
Update cargo submodule

7 commits in cad07cf8e597397fff5d0f6cb79c9f870c0b1dd2..31d3246e0e8d3a80d4a90b54aa48ff180adb408b
2026-10-23 22:04:58 +0000 to 2026-10-27 13:54:39 +0000
- Warn about conflicting proc-macro manifest settings (rust-lang/cargo#17641)
- docs: use latest `resolver` and `edition` in examples (rust-lang/cargo#17644)
- docs(hints): write mostly-unused docs as end-user documentation (rust-lang/cargo#17549)
- refactor(help): embed man pages without a tarball  (rust-lang/cargo#17640)
- refactor(cargo-test-support): remove build.rs (rust-lang/cargo#17638)
- fix: exclude abspath of path deps from `-Cmetadata` hash (rust-lang/cargo#17624)
- use build-rs for buildscripts (rust-lang/cargo#17637)

r? ghost
@rustbot rustbot added this to the 1.101.0 milestone Oct 29, 2026
hashcatHitman pushed a commit to hashcatHitman/miri that referenced this pull request Oct 30, 2026
Update cargo submodule

7 commits in cad07cf8e597397fff5d0f6cb79c9f870c0b1dd2..31d3246e0e8d3a80d4a90b54aa48ff180adb408b
2026-10-23 22:04:58 +0000 to 2026-10-27 13:54:39 +0000
- Warn about conflicting proc-macro manifest settings (rust-lang/cargo#17641)
- docs: use latest `resolver` and `edition` in examples (rust-lang/cargo#17644)
- docs(hints): write mostly-unused docs as end-user documentation (rust-lang/cargo#17549)
- refactor(help): embed man pages without a tarball  (rust-lang/cargo#17640)
- refactor(cargo-test-support): remove build.rs (rust-lang/cargo#17638)
- fix: exclude abspath of path deps from `-Cmetadata` hash (rust-lang/cargo#17624)
- use build-rs for buildscripts (rust-lang/cargo#17637)

r? ghost
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-layout Area: target output directory layout, naming, and organization A-reproducibility Area: reproducible / deterministic builds

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Not reproducible when a project is copied to another directory

3 participants