Implement cross-language ThinLTO

[michaelwoerister]

What is cross-language LTO?

Rust uses LLVM as its code generation backend, as does the Clang C/C++ compiler and many other languages. As a consequence, all of those LLVM-based compilers can produce artifacts that can partake in a common Link-Time-Optimization step, irrespective of the given source language. Thus, in this context, cross-language LTO means that we enable the Rust compiler to produce static libraries that can make use of LLVM-LTO-based linker plugins as exist for newer versions of ld, gold, and in lld.

Why is cross-language LTO a good thing?

In order for Rust to interoperate with code written in other languages, calls have to go through a C interface. This interface poses a boundary for inter-procedural optimizations like inlining. At the same time inter-procedural optimizations are very important for performance. Cross-language LTO makes this boundary transparent to LLVM, effectively allowing for C/C++ code to be inlined into Rust code and vice versa.

How can it be implemented?

There are several options. The basic requirement is that we emit LLVM bitcode into our object files in a format that the LLVM linker plugin can handle. There are two formats that fulfill this requirement:

1. We emit .o files that actually aren't object files but plain LLVM bitcode files.
2. We add the LLVM bitcode of an object file into a special .llvmbc section of the object file.

Given these requirements there are a few ways of implementing the feature:

1. Always emit bitcode into object files instead of storing them as separate files in RLIBs

   • Pros
     • simple for users, it just works
     • this is something that some platforms, like IOS, want to have anyway
   • Cons
     • staticlibs would contain bitcode, even though it might not be needed
     • the Rust compiler would have to be modified to read bitcode out of a section instead of separate obj-file
     • we could not store bitcode compressed anymore

2. Just stabilize -Z embed-bitcode and require users to do the rest

   • Pros
     • simple to implement
   • Cons
     • harder to use (needs user intervention)
     • unclear how to integrate with Cargo
     • RLIBs generated this way will contain bitcode twice

3. Add a flag that makes rustc emit bitcode files instead of object files

   • Pros
     • no redundant codegen
     • no redundant machine code
   • Cons
     • harder to use (needs user intervention)
     • produces libraries that are incompatible with regular compilation, which is weird
     • even more strange special casing Rust compiler backend
     • unclear how to integrate with Cargo

4. Add a -C cross-language-lto flag that (1) makes the compiler embed bitcode into RLIBs and static libraries, and (2) makes the compiler invoke the linker with the LLVMgold.so plugin, if applicable.

   • Pros
     • would make cross-language LTO available for binaries built with rustc
     • rustc can skip the redundant ThinLTO step for binaries and dylibs
     • RLIBs and staticlibs would be bigger but it's on an opt-in basis
   • Cons
     • since LTO is deferred to the linker, it would not be integrated with the Make jobserver
     • harder to use (needs user intervention)
     • unclear how to integrate with Cargo

I think I would opt for option (1) since it's the most straightforward to use. EDIT: Added option (4) which I also like.

cc @rust-lang/compiler @alexcrichton
(@rust-lang/wg-codegen might also be interested in this)

[alexcrichton]

I think I personally like option 4 best, although I might spin it a little differently as -C lto=cross-language or something like that. In that case rustc would do nothing for LTO other than ensuring it's ready to LTO inside the linker, and then then linker could do all the work.

[michaelwoerister]

With -C lto=cross-language, would rustc take care of invoking the linker correctly (e.g. when compiling an executable)? And if so, how would rustc know whether to invoke the linker for thin or for full LTO?

[alexcrichton]

@michaelwoerister yeah I think we could try to pass all the right options by default. I'm not actually sure how you configure full/thin at the linker layer?

One neat thing we could do, though, is that if you're on MSVC, for example, we could switch to lld by default or something like that

[michaelwoerister]

I'm not actually sure how you configure full/thin at the linker layer?

By passing -plugin-opt=thinlto to the linker, I think.

If we did it as -C lto=cross-language, we'd need another way of selecting thin vs full. Or have -C lto=thin-cross-language, which I find aesthetically displeasing :)

I'd love if we could shift all of LTO into the linker completely. But that would mean that we essentially can't use the MSVC linker anymore. And the Make jobserver story would regress too.

[nagisa]

`-Clto=thin,cross`?

…

On Fri, Apr 13, 2018, 11:14 Michael Woerister ***@***.***> wrote: I'm not actually sure how you configure full/thin at the linker layer? By passing -plugin-opt=thinlto to the linker, I think. If we did it as -C lto=cross-language, we'd need another way of selecting thin vs full. Or have -C lto=thin-cross-language, which I find aesthetically displeasing :) I'd love if we could shift *all* of LTO into the linker completely. But that would mean that we essentially can't use the MSVC linker anymore. And the Make jobserver story would regress too. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#49879 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AApc0oZ6AV9XuBLXcI0eQQ2QLHzXH5MHks5toF5rgaJpZM4TQJAM> .

[alexcrichton]

@michaelwoerister ah good point, in that case having a separate -C cross-language-lto seems fine by me (or @nagisa's idea)

[michaelwoerister]

@alexcrichton, do you know how to enable building the LLVMgold.so linker plugin? When I build LLVM from SVN, it gets built automatically but for rustc's LLVM version that doesn't seem to be the case.

[alexcrichton]

Ah I've never built it myself so I'm not sure :(