xLTO: Code from standard library does not partake in linker-based LTO (unless compiling with -Clto)

[michaelwoerister]

xLTO works by emitting LLVM bitcode instead of machine code and then letting the linker run LLVM optimizations (at a point where also code from C/C++ is available for IPO). In the current setup, this means that code from the standard library does not partake in the final LTO step because it is pre-compiled to machine code.

This limitation can be lifted by compiling to a staticlib with -Clto because the fat LTO step that rustc does pulls in the compressed bitcode version of the standard library and then emits the unified bitcode file for further processing by the linker. However, this adds an additional fat LTO step into the compilation pipeline, which is very costly.

In theory it should be possible to make rustc emit libstd LLVM bitcode into the staticlib instead of the object files (if -Clinker-plugin-lto is specified). The bitcode is available and rustc knows how to decompress it. Then the linker could do its LTO step with the standard library included but without the additional cost.

[NobodyXu]

Hey, is this issue still alive?

It will be awesome if the libstd.rlib can contain LLVM bitcode so that LTO can be performed to reduce size of the binary.

Without LTO even a helloworld program would take 200KB or more.

[bjorn3]

Libstd.rlib already contains LLVM bitcode, however it is only suitable for LTO performed by rustc (-Clto) rather than the linker (-Clinker-plugin-lto) which is what this issue is about. LTO alone can't reduce the size of hello world below 200kb as the binary among other things still includes code to print a backtrace on panics. If you want small rust binaries you can follow the instructions at https://github.com/johnthagen/min-sized-rust.

[NobodyXu]

Libstd.rlib already contains LLVM bitcode, however it is only suitable for LTO performed by rustc (-Clto) rather than the linker (-Clinker-plugin-lto) which is what this issue is about. LTO alone can't reduce the size of hello world below 200kb as the binary among other things still includes code to print a backtrace on panics. If you want small rust binaries you can follow the instructions at https://github.com/johnthagen/min-sized-rust.

Thank you for the explanation!

[insilications]

If you want std library to partake in linker-based LTO, do this. I've build the standard libraries with "-Clinker-plugin-lto=on -Ccodegen-units=1", so that all .rlib files contain the LLVM IR bitcode instead of the native object. I placed them all inside /usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib/ and remove the previous ones. Following the guide (https://doc.rust-lang.org/nightly/rustc/linker-plugin-lto.html), you can them have all the benefits of LTO along with cross LTO, for all your program and all dependencies, including over the standard library as well.