Cross-building rust 1.83.0 on & for NetBSD failure

I am doing my best to try to keep the various NetBSD targets for rust working.
As part of that process, I am cross-building binaries from x86_64 to most of the
other targets, and carry out testing of the results, for some of the targets this involves
self-hosting the rust compiler.

I have two new observations for rust 1.83.0 (1.82.0 works fine):

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

   1.288127806s  INFO prepare_target{force=false package_id=sysroot v0.0.0 (/usr
/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot) target="sysroot"}: ca
rgo::core::compiler::fingerprint: fingerprint error for sysroot v0.0.0 (/usr/pkg
src/wip/rust183/work/rustc-1.83.0-src/library/sysroot)/Build/TargetInner { name_inferred: true, ..: lib_target("sysroot", ["lib"], "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot/src/lib.rs", Edition2021) }
   1.288196443s  INFO prepare_target{force=false package_id=sysroot v0.0.0 (/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/library/sysroot) target="sysroot"}: cargo::core::compiler::fingerprint:     err: failed to read `/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot`

Caused by:
    No such file or directory (os error 2)

There are many such instances. Testing for the existence of the quoted file name afterwards does not reproduce the problem(!)
Looking for the file afterwards shows that it exists:

$ find work -type f -name lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1-std/armv7-unknown-netbsd-eabihf/release/.fingerprint/sysroot-76159fffe79d6c08/lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-904060a630664f36/lib-sysroot
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot
$ 
$ cat /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-std/x86_64-unknown-netbsd/release/.fingerprint/sysroot-40f70deaa15a4468/lib-sysroot; echo
07b22e914d1cad45
$

While natively building the rust compiler on x86_64 works fine, and the resulting compiler also works, I get a new (to me) problem while cross- building the rust compiler, in this first instance for our armv7hf target:

using sysroot /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1
running: "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) (created at src/core/builder.rs:1704:33, executed at src/core/builder.rs:1710:26)

Command "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) did not execute successfully.
Expected success, got exit status: 1
Created at: src/core/builder.rs:1704:33
Executed at: src/core/builder.rs:1710:26

STDOUT ----


STDERR ----
/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc: Shared object "librustc_driver-299f82a8112084d1.so" not found

The asked-for file exists in the build tree, ref.:

$ find work -name librustc_driver-299f82a8112084d1.so -type f
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/lib/librustc_driver-299f82a8112084d1.so
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-sysroot/lib/rustlib/x86_64-unknown-netbsd/lib/librustc_driver-299f82a8112084d1.so
work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage0-rustc/x86_64-unknown-netbsd/release/deps/librustc_driver-299f82a8112084d1.so
$

but apparently, this particular rustc instance isn't being pointed in the right direction.
It is also somewhat unclear to me whether this second problem is related to the first issue listed above.

My initial build attempt was done with -j 32 (I have the resources for it), but repeating it with -j 1 to root out whether this is a bug related to parallelism of the build reveals that the answer to that question is "no".

I expected to see this happen: Cross-building the rust compiler ought to succeed.

Instead, this happened:

The build errored out with

    Finished `release` profile [optimized] target(s) in 2m 45s
using sysroot /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1
running: "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) (created at src/core/builder.rs:1704:33, executed at src/core/builder.rs:1710:26)

Command "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc" "--target" "armv7-unknown-netbsd-eabihf" "--print=file-names" "--crate-type=proc-macro" "-" (failure_mode=Exit) did not execute successfully.
Expected success, got exit status: 1
Created at: src/core/builder.rs:1704:33
Executed at: src/core/builder.rs:1710:26

STDOUT ----


STDERR ----
/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/x86_64-unknown-netbsd/stage1/bin/rustc: Shared object "librustc_driver-299f82a8112084d1.so" not found

Traceback (most recent call last):
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/./x.py", line 50, in <module>
    bootstrap.main()
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 1227, in main
    bootstrap(args)
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 1203, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/src/bootstrap/bootstrap.py", line 202, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /usr/pkgsrc/wip/rust183/work/rustc-1.83.0-src/build/bootstrap/debug/bootstrap -v dist -j 1
*** Error code 1

Stop.
make[1]: stopped in /usr/pkgsrc/wip/rust183
*** Error code 1

Meta

This build is obviously using the previous version, rust 1.82.0:

rustc --version --verbose:

$ work/rust-1.82.0-x86_64-unknown-netbsd/rustc/bin/rustc --version --verbose
rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: x86_64-unknown-netbsd
release: 1.82.0
LLVM version: 19.1.1
$

Backtrace

Backtraces and details from the build provided above.
If wanted, I can provide a copy of the entire build log, hints for how to do so appreciated.

Update: The first issue mentioned above is not the same as the second; the first is visible also in a "native build".

rust183: add a package for rust version 1.83.0.

Well, it's about time to post some more information about this issue. This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed. Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed. It is evident that 1.82 did not critically depend on that feature, but 1.83 does. I have tried looking at the diff between 1.82 and 1.83 to see if a setting of LD_LIBRARY_PATH has been removed, but have come up empty (and the diff is HUGE...).

I have also verified that if I run patchelf on the stage1/bin/rustc and add the above path, and restart the build, the cross-build completes. However, I have not found where I might insert such a workaround in the rust build system. I have found the workarounds for the "NixOS" OS/variant in bootstrap.py, but that is evidently to patch a bootstrap compiler, and not the stage1 built compiler, so extending that to set rpath on NetBSD too probably will not achieve much in terms of mending this particular problem. So ... can anyone else please point me in the right direction of where I might tweak RPATH on the stage1 rustc before it is being used?

@he32

@he32 maybe you are looking for

rust/compiler/rustc_codegen_ssa/src/back/rpath.rs

Lines 46 to 49 in f17cf74

    
           // Use relative paths to the libraries. Binaries can be moved 
        
           // as long as they maintain the relative relationship to the 
        
           // crates they depend on. 
        
           let rpaths = get_rpaths_relative_to_output(config);

Could you elaborate what you mean by

This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed.

Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed.

Like, randomly dropping arguments seem like it's prone to produce random failures, no? Or why does non-absolute paths in RPATH arguments get dropped?

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

this is cargo telling you why it is rebuilding instead of using an existing cache. it is not indicative of an error. you presumably have CARGO_LOG set somewhere in your environment; you can silence this message by unsetting that env variable.

Could you elaborate what you mean by

This build is done within the pkgsrc framework, and as such, the C and C++ compilers are "wrapped", and certain arguments are being suppressed.
Among them is "use of non-absolute paths in RPATH arguments", and, yes, this means that arguments like -Wl,-rpath,$ORIGIN/../lib is being suppressed.

Like, randomly dropping arguments seem like it's prone to produce random failures, no? Or why does non-absolute paths in RPATH arguments get dropped?

I'll try, but please understand that I'm sitting in the squeeze between trying to maintain our rust port inside pkgsrc and the pkgsrc development and pkgsrc infrastructure maintenance in general. I asked on tech-pkg@NetBSD.org about this issue, and the start of that thread is at https://mail-index.netbsd.org/tech-pkg/2025/01/01/msg030311.html

As I understand it, the main issue is that pkgsrc tries to ensure that cross-package shared library dependencies are maintained when doing an upgrade, so pkgsrc needs to track which shared libraries a given package uses and is therefore depending on. So it's not just "randomly dropping arguments". The question has come up in the thread "why can't one just use LD_LIBRARY_PATH?", which is a reasonable question.

However, that aside, as I said above, I have found a workaround, but need to find the proper place to implant this in the rust build. In particular, I'm looking for where e.g. this phase of the build finishes (when cross-building from amd64 to armv7):

Building stage1 library artifacts (x86_64-unknown-netbsd -> armv7-unknown-netbsd-eabihf)

and before this step is started:

Building stage1 library artifacts (x86_64-unknown-netbsd)

because that is apparently the first time the "stage1" rust compiler is used. Optimally, the path of the about-to-be-used stage1 rustc compiler should be reasonably easily available at that point. I have found rustc-1.83.0-src/src/bootstrap/src/core/build_steps/compile.rs which looks like a candidate, but have yet to decipher where the appropriate place would be.

A new seemingly-non-fatal error message which is difficult to get a grasp on has started appearing in the build log:

this is cargo telling you why it is rebuilding instead of using an existing cache. it is not indicative of an error. you presumably have CARGO_LOG set somewhere in your environment; you can silence this message by unsetting that env variable.

OK. But as far as I can tell, this is "new noise" which wasn't there in 1.82.0, and looks remarkably like an error message. But I'm reasonably OK with just ignoring this part of this issue and rather move on with the actual build blocker part of this issue.

FWIW, we're seeing the same issue in our cross-compiles, as are folks from Void. We at least aren't doing any sort of flag mangling and I don't think they are either.

Rollup merge of rust-lang#136309 - onur-ozkan:133629, r=jieyouxu

Auto merge of rust-lang#136309 - onur-ozkan:133629, r=<try>

Rollup merge of rust-lang#136309 - onur-ozkan:133629, r=jieyouxu

Unrolled build for rust-lang#136309

dev-lang/rust: add bug reference to cross patch

fix: bump MSRV consistently

rust183: remove use of patchelf, as suggested by riastradh@

rust184: remove use of patchelf, as suggested by riastradh@

lang/rust: remove use of patchelf, as suggested by riastradh@

Navigation Menu

Repository navigation

Cross-building rust 1.83.0 on & for NetBSD failure #133629

Meta

19 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	// Use relative paths to the libraries. Binaries can be moved
	// as long as they maintain the relative relationship to the
	// crates they depend on.
	let rpaths = get_rpaths_relative_to_output(config);

Cross-building rust 1.83.0 on & for NetBSD failure #133629

Description

Meta

Activity

he32 commented on Nov 30, 2024

he32 commented on Jan 4, 2025

jieyouxu commented on Jan 4, 2025

jieyouxu commented on Jan 4, 2025

jyn514 commented on Jan 4, 2025

he32 commented on Jan 4, 2025

he32 commented on Jan 4, 2025

thesamesam commented on Jan 5, 2025

19 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions