[PowerPC64] Rust performs improper function call linkage when using LLVM's linker

[DrChat]

With this code:

#![feature(no_core, lang_items, rustc_attrs, repr_simd)]
#![crate_type = "bin"]
#![no_core]
#![no_main]

#[lang = "sized"]
trait Sized {}
#[lang = "copy"]
trait Copy {}

#[no_mangle]
pub fn my_rad_unmangled_function() {
    loop {}
}

pub fn my_rad_function() {
    loop {}
}

#[no_mangle]
pub extern fn _start() {
    my_rad_unmangled_function();
    my_rad_function();
}

And this compiler invocation:

$ rustc --version
rustc 1.54.0-nightly (3e99439f4 2021-05-17)
$ rustc --target=powerpc64-unknown-linux-gnu -C linker=lld -C relocation-model=static .\min.rs

Rust will emit code with the ELFv1 ABI, which dispatches function calls to a function descriptor table in the .opd section for later fixup by the linker.

As per this LLVM thread, they mention lld does not support the ELFv1 ABI, and will not perform the necessary function call fixups.
As such, it appears that Rust somehow suffers from the same bug that clang suffered from, wherein it is not explicitly marking binaries as using the ELFv1 ABI, and LLVM is not detecting the ELFv1 ABI.

This causes Rust to output an invalid binary:

$ llvm-objdump -d .\min

.\min:  file format ELF64-ppc64


Disassembly of section .text:

0000000010010000 .text:
10010000: 48 00 00 04                   b .+4
10010004: 48 00 00 00                   b .+0
                ...
10010014: 48 00 00 04                   b .+4
10010018: 48 00 00 00                   b .+0
                ...
10010028: 7c 08 02 a6                   mflr 0
1001002c: f8 01 00 10                   std 0, 16(1)
10010030: f8 21 ff 91                   stdu 1, -112(1)
10010034: 48 01 ff cd                   bl .+131020 # <----------------- branch to .opd!
10010038: 60 00 00 00                   nop
1001003c: 48 01 ff dd                   bl .+131036 # <----------------- branch to .opd!
10010040: 60 00 00 00                   nop
10010044: 38 21 00 70                   addi 1, 1, 112
10010048: e8 01 00 10                   ld 0, 16(1)
1001004c: 7c 08 03 a6                   mtlr 0
10010050: 4e 80 00 20                   blr
                ...

$ llvm-nm .\min
0000000010028000 d .TOC.
0000000010030018 d _ZN3min15my_rad_function17h0209550c6e3677ccE
0000000010030030 D _start
0000000010030000 D my_rad_unmangled_function

Testing reveals that linking with GNU's powerpc64-linux-gnu-ld linker yields a correct binary:

$ llvm-objdump -d .\min                                                                                                        
.\min:  file format ELF64-ppc64


Disassembly of section .text:

0000000010000158 .text:
10000158: 48 00 00 04                   b .+4
1000015c: 48 00 00 00                   b .+0
                ...
1000016c: 48 00 00 04                   b .+4
10000170: 48 00 00 00                   b .+0
                ...
10000180: 7c 08 02 a6                   mflr 0
10000184: f8 01 00 10                   std 0, 16(1)
10000188: f8 21 ff 91                   stdu 1, -112(1)
1000018c: 4b ff ff cd                   bl .-52 # <--------------------- branches to code!
10000190: 60 00 00 00                   nop
10000194: 4b ff ff d9                   bl .-40 # <--------------------- branches to code!
10000198: 60 00 00 00                   nop
1000019c: 38 21 00 70                   addi 1, 1, 112
100001a0: e8 01 00 10                   ld 0, 16(1)
100001a4: 7c 08 03 a6                   mtlr 0
100001a8: 4e 80 00 20                   blr
                ...

$ llvm-nm .\min 
0000000010027f00 d .TOC.
000000001001ffd0 d _ZN3min15my_rad_function17h0209550c6e3677ccE
00000000100001b8 r __GNU_EH_FRAME_HDR
0000000010020000 D __bss_start
0000000010020000 D _edata
0000000010020000 D _end
000000001001ffe8 D _start
000000001001ffb8 D my_rad_unmangled_function

[ostylk]

The LLVM thread link is dead (it's also not archived by Wayback Machine :/).

Unfortunately today I have also run into this issue...

[workingjubilee]

Should we just migrate PPC64 on Linux to the ELFv2 ABI?

[ostylk]

The same issue already occurred for the guys at zig (ziglang/zig#5927). They solved it by generating ppc64 binaries only with ELFv2 ABI: https://github.com/ziglang/zig/pull/21310/files .

The real fix would most likely be to implement ELFv1 ABI in lld (especially because other llvm tools support ELFv1, causing this bug). However, since ELFv1 is basically dead it probably won't happen, see e.g. llvm/llvm-project#27630 (or llvm/llvm-project#50927).

I think forcing to use ELFv2 for ppc64 targets is fine. If that is not desired (too hacky?), it would be worth it so set the e_flags of ELFv1 object files correctly (e_flags=1) such that lld errors out instead of generating a broken binary (https://github.com/llvm/llvm-project/blob/main/lld/ELF/Arch/PPC64.cpp#L642). lld seems to interpret e_flags=0 as ELFv2.