rust-lld fails with "dyld: Library not loaded: @rpath/libLLVM.dylib" on 1.30.0-nightly (02cb8f2a4 2018-08-29)

Short description: I am building for a no_std environment using cargo-xbuild, and the embedded lld as my linker of choice. The linker fails to load since it depends on a dynamic library that doesn't actually exist.

$ rustc --version
rustc 1.30.0-nightly (02cb8f2a4 2018-08-29)

$ rustc --print sysroot
/Users/parasyte/.rustup/toolchains/nightly-x86_64-apple-darwin

$ /Users/parasyte/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld --help
dyld: Library not loaded: @rpath/libLLVM.dylib
  Referenced from: /Users/parasyte/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld
  Reason: image not found
Abort trap: 6

$ otool -L /Users/parasyte/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld
/Users/parasyte/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld:
	@rpath/libLLVM.dylib (compatibility version 1.0.0, current version 7.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)

Rolling back to nightly-2018-08-27-x86_64-apple-darwin works

$ rustup override set nightly-2018-08-27-x86_64-apple-darwin
info: using existing install for 'nightly-2018-08-27-x86_64-apple-darwin'
info: override toolchain for '/Users/parasyte/projects/technek' set to 'nightly-2018-08-27-x86_64-apple-darwin'

  nightly-2018-08-27-x86_64-apple-darwin unchanged - rustc 1.30.0-nightly (721913067 2018-08-26)

$ rustc --version
rustc 1.30.0-nightly (721913067 2018-08-26)

$ rustc --print sysroot
/Users/parasyte/.rustup/toolchains/nightly-2018-08-27-x86_64-apple-darwin

$ /Users/parasyte/.rustup/toolchains/nightly-2018-08-27-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld --help
lld is a generic driver.
Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-lld (WebAssembly) instead

$ otool -L /Users/parasyte/.rustup/toolchains/nightly-2018-08-27-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld
/Users/parasyte/.rustup/toolchains/nightly-2018-08-27-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld:
	/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)

Searching the source tree for rpath turns up some humorous comments...

rust/src/bootstrap/bin/rustc.rs

Lines 199 to 232 in 4f1e235

    
           // Dealing with rpath here is a little special, so let's go into some 
        
           // detail. First off, `-rpath` is a linker option on Unix platforms 
        
           // which adds to the runtime dynamic loader path when looking for 
        
           // dynamic libraries. We use this by default on Unix platforms to ensure 
        
           // that our nightlies behave the same on Windows, that is they work out 
        
           // of the box. This can be disabled, of course, but basically that's why 
        
           // we're gated on RUSTC_RPATH here. 
        
           // 
        
           // Ok, so the astute might be wondering "why isn't `-C rpath` used 
        
           // here?" and that is indeed a good question to task. This codegen 
        
           // option is the compiler's current interface to generating an rpath. 
        
           // Unfortunately it doesn't quite suffice for us. The flag currently 
        
           // takes no value as an argument, so the compiler calculates what it 
        
           // should pass to the linker as `-rpath`. This unfortunately is based on 
        
           // the **compile time** directory structure which when building with 
        
           // Cargo will be very different than the runtime directory structure. 
        
           // 
        
           // All that's a really long winded way of saying that if we use 
        
           // `-Crpath` then the executables generated have the wrong rpath of 
        
           // something like `$ORIGIN/deps` when in fact the way we distribute 
        
           // rustc requires the rpath to be `$ORIGIN/../lib`. 
        
           // 
        
           // So, all in all, to set up the correct rpath we pass the linker 
        
           // argument manually via `-C link-args=-Wl,-rpath,...`. Plus isn't it 
        
           // fun to pass a flag to a tool to pass a flag to pass a flag to a tool 
        
           // to change a flag in a binary? 
        
           if env::var("RUSTC_RPATH") == Ok("true".to_string()) { 
        
               let rpath = if target.contains("apple") { 
        
                   // Note that we need to take one extra step on macOS to also pass 
        
                   // `-Wl,-instal_name,@rpath/...` to get things to work right. To 
        
                   // do that we pass a weird flag to the compiler to get it to do 
        
                   // so. Note that this is definitely a hack, and we should likely 
        
                   // flesh out rpath support more fully in the future.

rust/src/librustc_codegen_llvm/back/linker.rs

Lines 368 to 371 in 9d69e81

    
           // Note that the `osx_rpath_install_name` option here is a hack 
        
           // purely to support rustbuild right now, we should get a more 
        
           // principled solution at some point to force the compiler to pass 
        
           // the right `-Wl,-install_name` with an `@rpath` in it.

I have this issue too!

I believe this issue is caused by 3cf6f0d: "Link LLVM tools dynamically...".

It only enables dynamic linking libllvm on Linux and Mac OS, and since no Windows users complained so far I can only assume it's the probable cause.

Speaking of which, it might be a good idea to add testing the built LLVM tools / LLD to Rust's CI, to ensure issues like this are caught earlier on.

@alexcrichton

@alexcrichton @michaelwoerister this may have been caused by #53245.

Also, I don't see this issue on Linux:

$ ldd $(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld
        linux-vdso.so.1 (0x00007ffe1cd74000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fab8c935000)
        libLLVM-7.so => /home/japaric/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-7.so (0x00007fab88ecb000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007fab88d46000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fab88d2c000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007fab88b68000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fab8c984000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007fab88b5c000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fab88b57000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fab889c8000)

$ $(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld -flavor gnu -version
LLD 7.0.0 (compatible with GNU linkers)

$ rustc -V
rustc 1.30.0-nightly (02cb8f2a4 2018-08-29)

@japaric

@japaric I keep my toolchain updated with rustup, yet it seems I don't have libllvm.so...?

$ ldd $(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/bin/rust-lld
        linux-vdso.so.1 (0x00007ffe30fa4000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007ff1b4c40000)
        libLLVM-7.so => not found
        libm.so.6 => /usr/lib/libm.so.6 (0x00007ff1b4abb000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ff1b4aa1000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007ff1b48dd000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007ff1b4c9d000)

rustup update output:

info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: checking for self-updates

  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.30.0-nightly (02cb8f2a4 2018-08-29)

@GabrielMajeri

@GabrielMajeri it seems that libLLVM.so is provided by the llvm-tools-preview component which I have installed. Try running rustup component add llvm-tools-preview and see if that fixes LLD.

@alexcrichton @michaelwoerister it sounds like libLLVM.so should always by shipped with the compiler since rust-lld depends on it.

@japaric

@japaric I installed them, but it didn't help. The LLVM tools do seem to work, and I now have the libLLVM.so library in the <rust-sysroot>/lib directory.

However, checking rust-lld's RPATH shows $ORIGIN/../lib,
i.e. it points to <rust-sysroot>/lib/rustlib/x86_64-unknown-linux-gnu/lib/.

Doing

ln -s $(rustc --print sysroot)/lib/libLLVM-7.so $(rustc --print sysroot)/lib/rustlib/x86_64-unknown-linux-gnu/lib/

does (temporarily) fix the problem.

I'm working on a fix for this!

rustbuild: Distribute libLLVM.so with rustc

@japaric

@japaric's diagnosis is spot on and this should be fixed in #53828

To recap, interim fixes include:

rustup component add llvm-tools-preview --toolchain nightly
Downgrading to an older nightly temporarily

@alexcrichton

Thanks, @alexcrichton!

Auto merge of #53828 - alexcrichton:dist-llvm-shared, r=japaric

rustup component add llvm-tools-preview --toolchain nightly

Doesn't work for me on macOS by the way.

Edit I was wrong. It's still nightly

I get this with stable rust too (1.28.0):

192-168-1-4:wasm-game-of-life anton$ wasm-pack init
  
  [1/8] 🔧  Checking crate configuration...
  [2/8] 🎯  Adding WASM target...
\ [3/8] 🌀  Compiling to WASM...
Compilation of your program failed. stderr:

   Compiling wasm-game-of-life v0.1.0 (file:///Users/anton/projects/hacks/rust/wasm-game-of-life)
warning: function is never used: `set_panic_hook`
  --> src/utils.rs:11:9
   |
11 |         pub fn set_panic_hook() {}
   |         ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(dead_code)] on by default

error: linking with `rust-lld` failed: signal: 6
  |
  = note: "rust-lld" "-flavor" "wasm" "-L" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/wasm_game_of_life.wasm_game_of_life.59gcvxac-cgu.0.rcgu.o" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/wasm_game_of_life.wasm_game_of_life.59gcvxac-cgu.1.rcgu.o" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/wasm_game_of_life.wasm_game_of_life.59gcvxac-cgu.2.rcgu.o" "-o" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/wasm_game_of_life.wasm" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/wasm_game_of_life.bxpq0uzydj2zf4e.rcgu.o" "--gc-sections" "-O3" "-L" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps" "-L" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/release/deps" "-L" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/libwasm_bindgen-f9f62e4f9e995bb5.rlib" "/Users/anton/projects/hacks/rust/wasm-game-of-life/target/wasm32-unknown-unknown/release/deps/libcfg_if-954cfacc8a40931a.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libstd-b0efc82f64d03095.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libpanic_abort-4f93017645c0d252.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libunwind-c43a3d8cf78f9d5a.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/liblibc-68373f2b6bc18151.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/liballoc_system-40a64b71ec5a8f18.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libdlmalloc-f6caffaf44cc60a0.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/liballoc-863f2b2485046aef.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libcore-21433d3b7663b55b.rlib" "/Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/wasm32-unknown-unknown/lib/libcompiler_builtins-bf63ca2c3bc4de53.rlib" "--no-threads" "-z" "stack-size=1048576" "--stack-first" "--allow-undefined" "--no-entry" "--export-table"
  = note: dyld: Library not loaded: @rpath/libLLVM.dylib
            Referenced from: /Users/anton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/bin/rust-lld
            Reason: image not found
          

error: aborting due to previous error

error: Could not compile `wasm-game-of-life`.

To learn more, run the command again with --verbose.

rustbuild: Tweak LLVM distribution layout

Auto merge of #53955 - alexcrichton:fix-dist-again, r=<try>

@alexcrichton

Thanks for your super-fast help, @alexcrichton! 🚀

Update for new nightly (hits rust-lang/rust#53813 on macOS though)

Auto merge of #53955 - alexcrichton:fix-dist-again, r=japaric

For anyone wondering, the osx fix is not on the current nightly – rustc 1.30.0-nightly (6e0f1cc15 2018-09-05) – yet.

Why's this issue closed then? 🤔

Why's this issue closed then? 🤔

Is it because the fix will be in tomorrow's nightly build?

Yes this fix missed last night's nightly but it should be in tonight's nightly!

I'm still getting this error with rustc 1.30.0-nightly (a8c11d216 2018-09-06).

Looks like there's at least one more bug to fix -- #54018

Finally, I managed to compile on a mac with the latest nightly build of rust!

Navigation Menu

Repository navigation

rust-lld fails with "dyld: Library not loaded: @rpath/libLLVM.dylib" on 1.30.0-nightly (02cb8f2a4 2018-08-29) #53813

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	// Dealing with rpath here is a little special, so let's go into some
	// detail. First off, `-rpath` is a linker option on Unix platforms
	// which adds to the runtime dynamic loader path when looking for
	// dynamic libraries. We use this by default on Unix platforms to ensure
	// that our nightlies behave the same on Windows, that is they work out
	// of the box. This can be disabled, of course, but basically that's why
	// we're gated on RUSTC_RPATH here.
	//
	// Ok, so the astute might be wondering "why isn't `-C rpath` used
	// here?" and that is indeed a good question to task. This codegen
	// option is the compiler's current interface to generating an rpath.
	// Unfortunately it doesn't quite suffice for us. The flag currently
	// takes no value as an argument, so the compiler calculates what it
	// should pass to the linker as `-rpath`. This unfortunately is based on
	// the compile time directory structure which when building with
	// Cargo will be very different than the runtime directory structure.
	//
	// All that's a really long winded way of saying that if we use
	// `-Crpath` then the executables generated have the wrong rpath of
	// something like `$ORIGIN/deps` when in fact the way we distribute
	// rustc requires the rpath to be `$ORIGIN/../lib`.
	//
	// So, all in all, to set up the correct rpath we pass the linker
	// argument manually via `-C link-args=-Wl,-rpath,...`. Plus isn't it
	// fun to pass a flag to a tool to pass a flag to pass a flag to a tool
	// to change a flag in a binary?
	if env::var("RUSTC_RPATH") == Ok("true".to_string()) {
	let rpath = if target.contains("apple") {

	// Note that we need to take one extra step on macOS to also pass
	// `-Wl,-instal_name,@rpath/...` to get things to work right. To
	// do that we pass a weird flag to the compiler to get it to do
	// so. Note that this is definitely a hack, and we should likely
	// flesh out rpath support more fully in the future.

	// Note that the `osx_rpath_install_name` option here is a hack
	// purely to support rustbuild right now, we should get a more
	// principled solution at some point to force the compiler to pass
	// the right `-Wl,-install_name` with an `@rpath` in it.

rust-lld fails with "dyld: Library not loaded: @rpath/libLLVM.dylib" on 1.30.0-nightly (02cb8f2a4 2018-08-29) #53813

Description

Activity

parasyte commented on Aug 30, 2018

TejasQ commented on Aug 30, 2018

GabrielMajeri commented on Aug 30, 2018

japaric commented on Aug 30, 2018

GabrielMajeri commented on Aug 30, 2018

japaric commented on Aug 30, 2018

GabrielMajeri commented on Aug 30, 2018

alexcrichton commented on Aug 30, 2018

alexcrichton commented on Aug 30, 2018

TejasQ commented on Aug 30, 2018

pepyakin commented on Aug 31, 2018

thehappycoder commented on Aug 31, 2018

11 remaining items

TejasQ commented on Sep 5, 2018

dvdplm commented on Sep 6, 2018

TejasQ commented on Sep 6, 2018

jcaudle commented on Sep 6, 2018

alexcrichton commented on Sep 6, 2018

kzvi commented on Sep 7, 2018

alexcrichton commented on Sep 7, 2018

thehappycoder commented on Sep 8, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions