Link error when using native_link_modifiers_bundle and ld.bfd

[12101111]

This is found in #85600 (comment)

code: https://github.com/12101111/native_link_modifiers_bundle

export RUSTFLAGS="-Ctarget-feature=+crt-static -Clinker=ld.bfd"
cargo run --target=x86_64-unknown-linux-musl

   Compiling cc v1.0.69
   Compiling foo v0.1.0 (/tmp/musl-link)
error: linking with `ld.bfd` failed: exit status: 1
  |
  = note: "ld.bfd" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.15pu1hl0kal84f1y.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1h68etnkmxrn6yzz.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1pe73trdgmny6a8t.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.1zms9s2k5rtx003u.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.2re1n6nelkxha2og.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.36sds9xqyeuceja1.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.3nlkyw0e8uutnyey.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.8q3ae5zw7l4m4j0.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.i3svhsqj32n5bqa.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.j7ech079p6rmvfx.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.wktolicz9f1e7px.rcgu.o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844.46l3v0trxis9yeee.rcgu.o" "--as-needed" "-L" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps" "-L" "/tmp/musl-link/target/debug/deps" "-L" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/build/foo-c6c7efd0b76b3015/out" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Bstatic" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/libfoo-cb4158256382c543.rlib" "--start-group" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-f1cbe3fb941687fb.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-6774d3dce98216e2.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libminiz_oxide-2b70a69428b67501.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libadler-7c0d8eb24f55575d.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libobject-39efbcf3ca573d24.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libaddr2line-65df03b5b3a62814.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libgimli-6c5352065190f68a.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_detect-7a40648f799302f6.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-255ccea4ef40f4d4.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-e6cbd0b0da1406a9.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-6be43c7f9d293971.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-a8a6eb096906f234.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-84f9c9d47d8869d8.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-1d56f1e04d9d51b1.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-b292e64c32db2ac6.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-d144bec3bbe14c26.rlib" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-58e3a51e774ec0f7.rlib" "--end-group" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-5added8ca6387776.rlib" "-lfoo" "-Bdynamic" "--eh-frame-hdr" "-znoexecstack" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib" "-L" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-o" "/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/foo-656e6e56d7971844" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-zrelro" "-znow" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: ld.bfd: /tmp/musl-link/target/x86_64-unknown-linux-musl/debug/build/foo-c6c7efd0b76b3015/out/libfoo.a(foo.o): in function `bar':
          /tmp/musl-link/foo.c:3: undefined reference to `printf'
          
  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `foo` due to previous error

This is the part of link command line:

"/tmp/musl-link/target/x86_64-unknown-linux-musl/debug/deps/libfoo-cb4158256382c543.rlib" 
"--start-group" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-f1cbe3fb941687fb.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-6774d3dce98216e2.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libminiz_oxide-2b70a69428b67501.rlib" 
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libadler-7c0d8eb24f55575d.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libobject-39efbcf3ca573d24.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libaddr2line-65df03b5b3a62814.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libgimli-6c5352065190f68a.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_detect-7a40648f799302f6.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-255ccea4ef40f4d4.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-e6cbd0b0da1406a9.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-6be43c7f9d293971.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-a8a6eb096906f234.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-84f9c9d47d8869d8.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-1d56f1e04d9d51b1.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-b292e64c32db2ac6.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-d144bec3bbe14c26.rlib"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-58e3a51e774ec0f7.rlib"
"--end-group"
"/home/han/.rustup/toolchains/nightly-x86_64-unknown-linux-musl/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-5added8ca6387776.rlib" 
"-lfoo"

libfoo.a need symbols from libc, but libc is include in liblibc-*.rlib, which appear in the command line before libfoo.a. ld.lld is "smart" enough to find those symbols from liblibc-*.rlib, but ld.bfd use the traditional unix linker behavior.

The logic of assemble linker command line is at

rust/compiler/rustc_codegen_ssa/src/back/link.rs

Lines 1797 to 1814 in aea2e44

	// FIXME: Move this below to other native libraries
	// (or alternatively link all native libraries after their respective crates).
	// This change is somewhat breaking in practice due to local static libraries being linked
	// as whole-archive (#85144), so removing whole-archive may be a pre-requisite.
	if sess.opts.debugging_opts.link_native_libraries {
	add_local_native_libraries(cmd, sess, codegen_results);
	}
	// Rust libraries.
	add_upstream_rust_crates::<B>(cmd, sess, codegen_results, crate_type, tmpdir);
	// Native libraries linked with `#[link]` attributes at and `-l` command line options.
	// If -Zlink-native-libraries=false is set, then the assumption is that an
	// external build system already has the native dependencies defined, and it
	// will provide them to the linker itself.
	if sess.opts.debugging_opts.link_native_libraries {
	add_upstream_native_libraries(cmd, sess, codegen_results, crate_type);
	}

Currently libfoo.a is added to linker command line by add_upstream_native_libraries

Possible solution:

1. Add native libraries with kindNativeLibKind::Static { bundle: Some(false), .. } from upstream rlib crate to linker command line right after that crate is added in add_upstream_rust_crates.
2. Move all static native libraries and rlib into one link group
3. (Maybe just a workaround): also link libc with kind = "static", modifiers = "-bundle": Use link modifiers -bundle on musl and wasi target libc#2272

Meta

rustc --version --verbose:

rustc 1.56.0-nightly (2faabf579 2021-07-27)
binary: rustc
commit-hash: 2faabf579323f5252329264cc53ba9ff803429a3
commit-date: 2021-07-27
host: x86_64-unknown-linux-musl
release: 1.56.0-nightly
LLVM version: 12.0.1

[petrochenkov]

cc #85805

I don't think this is a bug.
libunwind does not indicate in any way that it depends on libc, so rustc builds the linker invocation without knowing about this dependency, thus we have linking errors.

We should support specifying all dependencies explicitly (similarly to how Rust crates form a dependency tree with Cargo) instead of making everything automatically depend on everything by using mechanisms like groups (which lld does by default).