Windows: Static libraries can't be linked as files by libtool/MinGW due to .lib extension

[str4d]

Consider a C++ project using common build system tooling (autoconf, automake, gcc, libtool, pkg-conf etc.), that wants to statically link in a Rust library (which it will interact with via a C FFI). The project wants to target both Unix and Windows, so leverages MinGW for compiling C++ on Windows.

libtool follows the MinGW/GCC convention of static libraries ending in .a, not the MSVC convention of ending in .lib. libtool does support linking libraries with libfoo.a and foo.lib naming patterns if the -Lpath/to/foo/ -lfoo option pattern is used. However, if the library file is passed directly as an argument (which is a very common pattern in automake build systems when linking subunits together), libtool only treats .a files as libraries; .lib files are treated as unknown options, and moved to earlier in the eventual linker command. This in turn causes obscure linker errors due to a reordering of the library arguments.

The root cause is that rustc unconditionally generates static libraries with .lib extensions for all Windows compiler environments, instead of using .lib for MSVC environments and .a for MinGW environments. This behaviour was introduced in Rust 1.8.0 (previously .a was used unconditionally for all static libraries - #29508).

Possible workarounds without changes to rustc:

• Use the -lfoo option for linking.
  • This could require significant refactoring of the C++ build system, as the Makefiles could not rely on the built-in support for detecting that the library has been built.
• Rename foo.lib to libfoo.a after Rust compilation.
  • This is awkward and hacky, and the same situation that MSVC users were in before their issue was addressed.

Possible solutions:

• Change the default extensions to match the compilation environment's conventions.
  • This ensures that the output of rustc matches users' default expectations.
  • This could cause regressions for projects that have already integrated workarounds into their build systems, but the fix would be to remove complexity from their projects.
• Enable users to override the output filename.
  • Similar to the --out-dir option for cargo, but changing the name of the artifact instead of the location.
  • There's probably an open issue already for this, but I can't find one.
• Clearly document this non-standard convention, and the recommended workaround(s).
  • Partly why I opened this issue, so future me would have something to find in search results 😄

This issue is effectively a re-opening of #43749, as recommended in #43749 (comment).

[retep998]

If possible, could we emit both libfoo.a and foo.lib for specifically pc-windows-gnu? That way code already hacked up to use foo.lib will continue to work while code that wasn't hacked up yet can just work correctly with libfoo.a.