Non-reproducible builds when depending on local crates

It looks like meta fingerprints are not particularly stable when it comes to depending on local crates (i.e. with a dependency specifying a "path", a "[patch.crates-io]" section or a "__CARGO_TESTS_ONLY_SRC_ROOT" environment variable); different absolute pathnames seem to produce different build fingerprints. This makes it difficult to reproduce builds when running as a different user or on a different operating system or distribution.

--remap-path-prefix doesn't seem to help at all here.

To reproduce this problem:


for i in a b
do
    mkdir $i
    (   
        cd $i
        TOPDIR=`pwd`
        cargo new --lib repro
        cat > repro/src/lib.rs <<EOF 
pub fn foo() -> usize { 
    ext_crate::add(1, 2) 
} 
EOF 
        cp -r ../ext_crate ext_crate
        cat > repro/Cargo.toml <<EOF 
[package] 
name = "repro" 
version = "0.1.0" 
edition = "2021" 
 
[dependencies] 
ext_crate = { version = "0.1.0", path="../ext_crate" } 
 
EOF 
        (   
            cd repro
            RUSTFLAGS="--remap-path-prefix=${TOPDIR}=/stablepath" cargo build -vv --release --target-dir=build
        )
    )
done
diff -ur [ab]/repro/build

My expected behavior here would be that those two builds would be identical and reproducible.

These are the versions I am using:

$ cargo --version --verbose
cargo 1.63.0-nightly (4d92f07f3 2022-06-09)
release: 1.63.0-nightly
commit-hash: 4d92f07f34ba7fb7d7f207564942508f46c225d3
commit-date: 2022-06-09
host: x86_64-unknown-linux-gnu
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.83.1-DEV (sys:0.4.55+curl-7.83.1 vendored ssl:OpenSSL/1.1.1n)
os: Ubuntu 20.04 (focal) [64-bit]
$ rustc --version --verbose
rustc 1.63.0-nightly (546c826f0 2022-06-12)
binary: rustc
commit-hash: 546c826f0ccaab36e897860205281f490db274e6
commit-date: 2022-06-12
host: x86_64-unknown-linux-gnu
release: 1.63.0-nightly
LLVM version: 14.0.5

@rustbot

@rustbot label +A-reproducibility

I think this bug may be responsible for the nonreproducibility of Hippotat in Debian:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1028254
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/hippotat.html

This bug may also be the same problem that is seen in the Yocto builds:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875

This might be related to #5505

The script here had several minor errors due to being copied from a terminal probably and bash being actually astonishingly whitespace sensitive, which caused all the heredocs to not be recognized upon copy-pasting this back into a script. I had to fix it to make it work, like so:

#!/bin/bash
cargo new --lib ext_crate
for i in a b
do
    mkdir $i
    (   
        cd $i
        TOPDIR=`pwd`
        cargo new --lib repro
        cat > repro/src/lib.rs <<EOF
pub fn foo() -> usize { 
    ext_crate::add(1, 2) as usize
} 
EOF
        cp -r ../ext_crate ext_crate
        cat > repro/Cargo.toml <<EOF
[package] 
name = "repro" 
version = "0.1.0" 
edition = "2021" 
 
[dependencies] 
ext_crate = { version = "0.1.0", path="../ext_crate" } 
 
EOF
        (   
            cd repro
            RUSTFLAGS="--remap-path-prefix=${TOPDIR}=/stablepath" cargo build -vv --release --target-dir=build
        )
    )
done
diff -ur [ab]/repro/build

In the future, I would suggest you upload shell scripts directly rather than copy-pasting from the terminal (or whatever the original source was) to GitHub if you have the slightest doubt over whether a single space or newline from the script might be altered. With a shebang, too, that properly denotes it's bash or what.

Please? So that it's, er, easier to reproduce the reproducer that reproduces the non-reproducibility?

I guess this is as close to an MCVE as we're gonna get.

This is an issue somewhere inside cargo. It passes different -Cmetadata arguments for both builds of ext_crate due to computing a different dep_hash, which then causes a different build output. I don't know know why it computes a different dep_hash though.

Thanks for clarifying! I suppose we can get this moved to rust-lang/cargo then?

@rustbot

@rustbot transfer cargo

feat(trim-paths): stabilize `profile.trim-paths` (rust-lang#17488)

With #17488 and #17624, this kind partially resolved. However, if we are talking about rlib reproducibility, note that rlib contain metadata which may contain -C extra-filename so we need to make them unit_id hash abspath free (the cargo fingerprint part) to actually resolve this. See also #17624 (comment).

Navigation Menu

Repository navigation

Non-reproducible builds when depending on local crates #15122

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Non-reproducible builds when depending on local crates #15122

Description

Activity

eggyal commented on Jun 17, 2022

ijackson commented on Jan 14, 2023

rwmacleod commented on Feb 6, 2023

ijackson commented on Feb 7, 2023

workingjubilee commented on Aug 3, 2024

workingjubilee commented on Aug 3, 2024

bjorn3 commented on Aug 3, 2024

workingjubilee commented on Aug 4, 2024

workingjubilee commented on Jan 29, 2025

weihanglo commented on Oct 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions