error_index_generator stage 0 tests don't work with download-ci-llvm: "error while loading shared libraries: libLLVM-11-rust-1.49.0-nightly.so"

command did not execute successfully: "/home/joshua/rustc/build/x86_64-unknown-linux-gnu/stage0-tools-bin/error_index_generator" "markdown" "/home/joshua/rustc/build/x86_64-unknown-linux-gnu/test/error-index.md"
expected success, got: exit code: 127

stdout ----

stderr ----
/home/joshua/rustc/build/x86_64-unknown-linux-gnu/stage0-tools-bin/error_index_generator: error while loading shared libraries: libLLVM-11-rust-1.49.0-nightly.so: cannot open shared object file: No such file or directory

failed to run: /home/joshua/rustc/build/bootstrap/debug/bootstrap test --stage 0 src/tools/error_index_generator/

I have download-ci-llvm = "if-available" in case that makes a difference.

Miri also runs into this when doing x.py test --stage 0 src/tools/miri (which used to work)

From zulip:

I think the fix is indeed that these tools should not be run without being in the sysroot.

@jyn514

@jyn514 I believe this may have been resolved some time between the issue being created and now. When i tried to reproduce it locally (a4cbb44) the error index tests were able to start and run just fine.

The tests failed with "error[E0658]: const generics are unstable" because some snippets use #![feature(const_generics)] instead of the recently stabilised min_const_generics, but that doesn't have anything to do with being unable to load shared libraries. Similarly I was able to run x.py test --stage 0 src/tools/miri without an issue.

@jyn514 and @RalfJung are you still able to reproduce this?

Similarly I was able to run x.py test --stage 0 src/tools/miri without an issue.

I just tried this and it still fails:

     Running `build/x86_64-unknown-linux-gnu/stage0-tools/x86_64-unknown-linux-gnu/release/cargo-miri miri setup`
thread 'main' panicked at 'failed to determine underlying rustc version of Miri: UnexpectedVersionFormat', src/tools/miri/cargo-miri/bin.rs:132:38
stack backtrace:
   0: rust_begin_unwind
             at ./library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at ./library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at ./library/core/src/option.rs:1266:5
   3: core::result::Result<T,E>::expect
             at ./library/core/src/result.rs:929:23
   4: cargo_miri::version_info
             at ./src/tools/miri/cargo-miri/bin.rs:132:5
   5: cargo_miri::setup
             at ./src/tools/miri/cargo-miri/bin.rs:325:16
   6: cargo_miri::phase_cargo_miri
             at ./src/tools/miri/cargo-miri/bin.rs:418:5
   7: cargo_miri::main
             at ./src/tools/miri/cargo-miri/bin.rs:757:25
   8: core::ops::function::FnOnce::call_once
             at ./library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I haven't checked, but I assume the "incorrect version format" is precisely because there's still some error such as "cannot open shared object file" being reported. (That's what used to happen.)

Note that I have download-ci-llvm = true; maybe that's important to reproduce the problem.

Note that I have download-ci-llvm = true; maybe that's important to reproduce the problem.

It looks like download-ci-llvm has something to do with this because when I switched to download-ci-llvm = true I also got the same "incorrect version format" error.

I haven't checked, but I assume the "incorrect version format" is precisely because there's still some error such as "cannot open shared object file" being reported. (That's what used to happen.)

I printed the output of rustc -vV (here) and apparently rustc doesn't generate any output. I'm not sure if that is relevant.

[/home/michael/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc_version-0.2.3/src/lib.rs:112] str::from_utf8(&out.stdout) = Ok(
    "",
)

I printed the output of rustc -vV (here) and apparently rustc doesn't generate any output. I'm not sure if that is relevant.

That's stdout; what does stderr say?

Ah yep, it's the "unable to load shared libraries" error message:

[/home/michael/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc_version-0.2.3/src/lib.rs:112] str::from_utf8(&out.stderr) = Ok(
    "/home/michael/Documents/forks/rust/build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri: error while loading shared libraries: libLLVM-11-rust-1.51.0-nightly.so: cannot open shared object file: No such file or directory\n",
)

Inspecting the failing miri binary shows it isn't able to find rustc-driver or std:

$ ldd build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri
	linux-vdso.so.1 (0x00007ffeac5f7000)
	librustc_driver-110e2444bfbc1fba.so => not found
	libstd-6593c2e4e017f5fb.so => not found
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f0f3d890000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f0f3d86e000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f0f3d728000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f0f3d55f000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f0f3dced000)

Should bootstrap be fiddling with LD_LIBRARY_PATH or setting rpath when compiling/running? I was able to make things work by setting LD_LIBRARY_PATH to the stage1 directory.

$ build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri --version
build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri: error while loading shared libraries: librustc_driver-110e2444bfbc1fba.so: cannot open shared object file: No such file or directory

$ export LD_LIBRARY_PATH=build/x86_64-unknown-linux-gnu/stage1/lib/
$ build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri --version
rustc 1.51.0-dev

The error is about libLLVM-11-rust-1.51.0-nightly.so though, not rustc_driver or std.

I think x.py already sets LD_LIBRARY_PATH.

I was able to make things work by setting LD_LIBRARY_PATH to the stage1 directory.

The error you got first is a different one though. Possibly it's not loading the codegen backend. Can you try miri -vV instead of miri --version?

I think x.py already sets LD_LIBRARY_PATH.

Ah, good to know.

The error is about libLLVM-11-rust-1.51.0-nightly.so though, not rustc_driver or std.

That's odd... Given the dependency chain miri > rustc_driver > libLLVM, on my machine the first link is broken while it looks like the second link is broken for you.

This is what I see when looking at the dependencies for rustc_driver.

$ ldd build/x86_64-unknown-linux-gnu/stage1/lib/librustc_driver-110e2444bfbc1fba.so
	linux-vdso.so.1 (0x00007fff511fb000)
	libstd-6593c2e4e017f5fb.so => /home/michael/Documents/forks/rust/build/x86_64-unknown-linux-gnu/stage1/lib/../lib/libstd-6593c2e4e017f5fb.so (0x00007fd8e30e2000)
	libLLVM-11-rust-1.51.0-nightly.so => /home/michael/Documents/forks/rust/build/x86_64-unknown-linux-gnu/stage1/lib/../lib/libLLVM-11-rust-1.51.0-nightly.so (0x00007fd8de6a1000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fd8de494000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fd8de47a000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd8de458000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007fd8de312000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd8de30a000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007fd8de141000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007fd8e776f000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007fd8de127000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007fd8de11c000)

Can you try miri -vV instead of miri --version?

Looks like it's picking up LLVM just fine 🤷‍♂️

$ export LD_LIBRARY_PATH=build/x86_64-unknown-linux-gnu/stage1/lib/  
$ build/x86_64-unknown-linux-gnu/stage0-tools-bin/miri -vV
rustc 1.51.0-dev
binary: rustc
commit-hash: unknown
commit-date: unknown
host: x86_64-unknown-linux-gnu
release: 1.51.0-dev
LLVM version: 11.0.1

Okay, I found who's overwriting LD_LIBRARY_PATH, it was add_rustc_lib_path. Here's my hackfix that makes ./x.py test --stage 0 src/tools/miri work with the downloaded LLVM:

diff --git a/src/bootstrap/test.rs b/src/bootstrap/test.rs
index 86d940cd733..96a94d43428 100644
--- a/src/bootstrap/test.rs
+++ b/src/bootstrap/test.rs
@@ -393,6 +392,12 @@ fn run(self, builder: &Builder<'_>) {
             );
             cargo.arg("--").arg("miri").arg("setup");
 
+            // Make sure we can access all the dynamic libraries.
+            let mut dylib_path = dylib_path();
+            let ci_llvm_lib = builder.out.join(&*host.triple).join("ci-llvm/lib");
+            dylib_path.insert(0, ci_llvm_lib);
+            cargo.env(dylib_path_var(), env::join_paths(&dylib_path).unwrap());
+            eprintln!("{} = {:?}", dylib_path_var(), env::join_paths(&dylib_path).unwrap());
             // Tell `cargo miri setup` where to find the sources.
             cargo.env("XARGO_RUST_SRC", builder.src.join("library"));
             // Tell it where to find Miri.
@@ -442,6 +447,7 @@ fn run(self, builder: &Builder<'_>) {
                 SourceType::Submodule,
                 &[],
             );
+            cargo.env(dylib_path_var(), env::join_paths(&dylib_path).unwrap());
 
             // miri tests need to know about the stage sysroot
             cargo.env("MIRI_SYSROOT", miri_sysroot);
@@ -450,7 +456,7 @@ fn run(self, builder: &Builder<'_>) {
 
             cargo.arg("--").args(builder.config.cmd.test_args());
 
-            cargo.add_rustc_lib_path(builder, compiler);
+            //cargo.add_rustc_lib_path(builder, compiler);
 
             if !try_run(builder, &mut cargo.into()) {
                 return;

I hope at some point someone can take this and turn it into something proper. ;)

It is also worth noting that this is a regression -- testing Miri on stage 0 used to work just fine some time in summer/fall last year. I have no idea which rustbuild change broke things, though.

EDIT: ah I remember, it's probably the pre-built LLVM download.

So how can we make progress on this? I produced a working diff above, but it's a crude hack and only fixes the problem for Miri. I don't know nearly enough about rustbuild to figure out the proper fix for this though -- there are lots of moving pieces here, so this is unlikely to be fixed until someone from the rustbuild team can take a look. That would be much appreciated. :)

@RalfJung

@RalfJung there is not really a rustbuild team - there's me and Mark-Simulacrum, and ehuss has a fair bit of knowledge I think. I'm on break for a while and Simulacrum is always busy.

I mean, I guess that makes you two (or three) the rustbuild team. At least this is exactly how I became part of the Miri and const-eval teams. ;)

Regarding the problem at hand, I could probably try patching add_rustc_lib_path so that it sets up dylib_path_var to include ci-llvm/lib (hopefully I can check to only do this when we are actually using CI LLVM). But would that make any sense?

@RalfJung

@RalfJung sure, that seems reasonable.

Auto merge of rust-lang#85959 - RalfJung:miri-stage-0, r=Mark-Simulacrum

Reopening as the issue persists for error_index_generator specifically, which has some custom logic for dylib paths that still needs adjusting:

rust/src/bootstrap/tool.rs

Lines 396 to 400 in 903e369

    
           add_dylib_path( 
        
               vec![ 
        
                   PathBuf::from(&builder.sysroot_libdir(compiler, compiler.host)), 
        
                   PathBuf::from(builder.rustc_libdir(compiler)), 
        
               ],

@RalfJung

@RalfJung the error index is already tracked by #80096

Navigation Menu

Repository navigation

error_index_generator stage 0 tests don't work with download-ci-llvm: "error while loading shared libraries: libLLVM-11-rust-1.49.0-nightly.so" #78778

15 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	add_dylib_path(
	vec![
	PathBuf::from(&builder.sysroot_libdir(compiler, compiler.host)),
	PathBuf::from(builder.rustc_libdir(compiler)),
	],

error_index_generator stage 0 tests don't work with download-ci-llvm: "error while loading shared libraries: libLLVM-11-rust-1.49.0-nightly.so" #78778

Description

Activity

jyn514 commented on Nov 5, 2020

RalfJung commented on Nov 7, 2020

jyn514 commented on Nov 7, 2020

Michael-F-Bryan commented on Jan 21, 2021

RalfJung commented on Jan 21, 2021

Michael-F-Bryan commented on Jan 21, 2021

RalfJung commented on Jan 21, 2021

Michael-F-Bryan commented on Jan 21, 2021

Michael-F-Bryan commented on Jan 21, 2021

RalfJung commented on Jan 21, 2021

RalfJung commented on Jan 21, 2021

Michael-F-Bryan commented on Jan 21, 2021

15 remaining items

RalfJung commented on Mar 27, 2021

RalfJung commented on Jun 2, 2021

RalfJung commented on Jun 2, 2021

jyn514 commented on Jun 2, 2021

RalfJung commented on Jun 2, 2021

RalfJung commented on Jun 2, 2021

jyn514 commented on Jun 2, 2021

RalfJung commented on Jun 4, 2021

jyn514 commented on Jun 4, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions