Tracking Issue for Reproducible Build bugs and challenges

[jieyouxu]

This is a tracking issue for collecting and triaging bugs and challenges that hinder our ability to produce reproducible and deterministic builds.

This tracking issue is used as a hub for connecting to other relevant issues, e.g., bugs or open design questions. This tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the A-reproducibility Area: Reproducible / deterministic builds label, and tag with OS/architecture/target labels as suitable.

Warning

This tracking issue is non-exhaustive. Please add suitable entries and edit as new issues and PRs pop up or if old issues and PRs are rediscovered.

Bugs and issues

• tests/run-make/reproducible-builds
  • Several skipped combinations due to unreproducibility
• [Inquiry] Inconsistent binary size with 'cargo +stable build' vs 'cargo +1.50-x86_64-pc-windows-msvc' ? #82188
  • Needs further investigation
• rustdoc output directory structure leaks home directory structure #18370
  • Status unclear
• Randomize the ordering of struct fields when compiling in debug mode #38550
  • Related to -Z randomize-layout, status unclear
• Non-deterministic perf.rust-lang.org runs (e.g. syn-opt). #69060
  • Perf; unclear what's to do here
• Occasional non-reproducibility when building rustc #69352
  • Possible non-determinism in LLVM
• User type annotations can result in different mir-opt test results in CI vs locally #73501
  • Unsure how to repro/debug
• Formalize SVH stability for a specific release and target #73932
  • Stable Version Hashes
• sysroot spans still not printed, when remap-path-prefix is set with a custom path #74786
  • --remap-path-prefix related
• Accepted convention for incurring dependencies on external files in macro expansion hurts reproducibility #75887
  • Macro expansions with deps on external files
• Reproducible build run-make tests fails if -C debuginfo=2 (on Linux this time) #89911
  • -Cdebuginfo=2
• Windows builds are not deterministic with --crate-type=bin and path-prefix remapping #88982
  • Affects tests/run-make/reproducible-builds, bin, --remap-path-prefix, -Z remap-cwd-prefix
• --crate-type=rlib + -Cdebuginfo=2 + -Zremap-cwd-prefix=. is not reproducible on Windows #129117
  • -Cdebuginfo=2, -Z remap-cwd-prefix=., rlib

Tracking/meta issues:

• CI for deterministic / reproducible builds #75362
• Add an unstable 'extra randomization' compile-time flag for hashers #65042
• Audit all UNTRACKED options #84232
• Tracking Issue for trim-paths RFC 3127 #111540
• Tracking Issue for -Z remap-cwd-prefix #89434

Resolved:

• Intermittent error page on rust-lang.org #6656
  • -Cdebuginfo=2
• OSX compilation with debuginfo isn't deterministic #47086
  • -Cdebuginfo=2
• Rustdoc output slightly nondeterministic #24473
• Bit-for-bit deterministic / reproducible builds #34902
  • Current status unclear
• Detect and report usage of environment variables at build time and run time. #44074
  • Env var info
• debug_loc info is (slightly) non-deterministic #45397
  • Looks LLVM related, seems to have resolved
• reproducible builds broken in rustc 1.56.0 due to LLVM 13 update #90301
  • Upstream fix: https://reviews.llvm.org/D113468
• DWARF info for static vars in lib crates has stopped being produced reliably in LTO builds #90357
• OSX compilation with debuginfo isn't deterministic #47086
• rustc produces indeterministic asm #57041
  • Insufficiently specific
• Cargo should provide --remap-path-prefix or not hash it from RUSTFLAGS. cargo#6914
  • Resolved separately
• rustdoc generation of search-index.js depends on the order crates are documented #61216
  • Resolved in a separate improvement PR but can't find which
• Create a deterministic FxHashMap wrapper #63713
  • Related: Intermittent failure in src/test/ui/reify-intrinsic.stderr #65036
• Intermittent failure in src/test/ui/reify-intrinsic.stderr #65036
• Semi-spurious ui test fluctuation #81296
• reproducible-build-2 fails if debuginfo-level > 0 #66568
• remap-path-prefix will not re-compile properly in dev build #66955
  • --remap-path-prefix
• Source file names are included into a release binary even if abort upon panic is enabled #75263
  • --remap-path-prefix involved
• Binaries end up containing path to the rust-src component despite --remap-path-prefix #73167
  • --remap-path-prefix involved
• Enable --remap-path-prefix for absolute paths by default #40552
  • Closed in favor of more specific issues and RFC-based approaches
• --remap-path-prefix doesn't work in debug mode? #84125
  • --remap-path-prefix involved
• registry paths hard coded in binary cargo#9311
• Tiny difference of the diagnostics for generator on some environment #71222
• Error reporting order on riscv64gc-unknown-linux-gnu #72913
• Reproducibility regression caused by 57edf88 #76496
• --remap-path-prefix not remapping out_dir in debug info on Wasm #80776
• --remap-path-prefix behavior regression resulting in bad paths in 1.50.0 #82074
  • --remap-path-prefix
• remap-path-prefix appears to be applied in a reverse order #82108
  • --remap-path-prefix; paths reverse order from clang/gcc but was intentional
• The build prefix is part of the compiled compiler #82392
  • -Crpath
• Reproducible outputs without build-environment-specific command lines #87325
  • -Z remap-cwd-prefix; closed in favor of tracking issue Tracking Issue for -Z remap-cwd-prefix #89434 for -Z remap-cwd-prefix
• --remap-path-prefix doesn't map paths to .pdb files, even in release mode #87825
  • --remap-path-prefix
• DWARF info for static vars in lib crates has stopped being produced reliably in LTO builds #90357
• nondeterministic ordering in rmeta section #113584
  • Issue disappeared after stable version update but unclear what fixed it
• building the std for aarch64_be-unknown-linux-gnu_ilp32 twice in a row, the binary libcompiler_builtins.rlib is inconsistent #119372
  • compiler_builtins
• Nondeterminism encountered in metadata for doc_link_resolutions #124357
  • Fuchsia build system specific, unclear
• Separate --remap-path-prefix for diagnostics and debug info #73740
  • --remap-path-prefix
• Add flag to modify the working directory assumed by the diagnostics #88754
• --remap-path-prefix does not apply to secondary files in diagnostics #66251
  • --remap-path-prefix; sysroot handling in diagnostics

Related issues, RFCs, MCPs and discussions

• Do not implement HashStable for HashSet (MCP 533) #108312 ( T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. )
• New rustc and Cargo options to allow path sanitisation by default rfcs#3127 ( F-trim-paths Feature: trim-paths )
• Reproducible command line + determinism compiler-team#450

Related PRs

• Print environment variables accessed by rustc as special comments into depinfo files #71858
• Work around invalid DWARF bugs for fat LTO #89041
• Export ZERO_AR_DATE for macos linker invocations #71931
• data_structures: Add deterministic FxHashMap and FxHashSet wrappers #64131
• Make re-export collection deterministic #65043
• Make weak item traversal deterministic #81393
• Use FxHashSet instead of Vec for well formed tys #88771
• Use UnordMap and UnordSet for id collections (DefIdMap, LocalDefIdMap, etc) #106977
• Add TRACKED_NO_CRATE_HASH and use it for --remap-path-prefix #84233
• Implement rustc part of RFC 3127 trim-paths #115214
• Simplify trim-paths feature by merging all debuginfo options together #122450
• feat: implement RFC 3127 -Ztrim-paths cargo#12625
• Fix rustdoc missing handling of remap-path-prefix option #128736
• Ignore remap-path-prefix in metadata hash. cargo#6966
• Do not show DefId in diagnostics #71310
• resolve: Sort E0408 errors by Symbol str #72982
• SessionDiagnostic: Fix non-determinism in generated format string. #76515
• Set path of the compile unit to the source directory #82102
• Apply path remapping to DW_AT_GNU_dwo_name when producing split DWARF #91566
• Remove the install prefix from the rpath set when using -Crpath #86025
• Introduce -Z remap-cwd-prefix switch #87320
• link.exe: Don't embed full path to PDB file in binary. #121297
• Revert "Work around invalid DWARF bugs for fat LTO" #95685
• Generate outlined_atomics.rs in a deterministic order. compiler-builtins#549
• rustdoc: Add support for --remap-path-prefix #107099