The sparc64, riscv64, loongarch64 `extern "C" fn` ABIs are all wrong when aligned/packed structs are involved

[RalfJung]

All of these ABIs have in common that they are looking at layout.fields to compute the ABI for the function, and they are all getting it wrong, in various different ways. (I previously reported bugs against all of these because repr(transparent) is not properly respected; those bugs have the same cause -- relying on layout.fields -- but they are orthogonal.)

For instance, this type

#[repr(packed)]
struct P(i8, f32);

on riscv64 gets a PassMode::Cast of

                           prefix: [
                               Some(
                                   Reg {
                                       kind: Integer,
                                       size: Size(1 bytes),
                                   },
                               ),
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                           ],
                           rest: Uniform {
                               unit: Reg {
                                   kind: Float,
                                   size: Size(4 bytes),
                               },
                               total: Size(4 bytes),
                           },

which gets translated to the LLVM type { i8, f32 }, and that's a non-packed LLVM struct. On a call, P gets transmuted to that LLVM type, which obviously produces garbage since the fields are at different offsets.

On sparc64, the type translates into { i32, f32 } which is wrong as well.

As another example, this type

#[repr(align(8))]
struct Aligned8Int(i32);

#[repr(C)]
struct S2(Aligned8Int, f32);

on loongarch64 gets a PassMode::Cast of

                           prefix: [
                               Some(
                                   Reg {
                                       kind: Integer,
                                       size: Size(4 bytes),
                                   },
                               ),
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                               None,
                           ],
                           rest: Uniform {
                               unit: Reg {
                                   kind: Float,
                                   size: Size(4 bytes),
                               },
                               total: Size(4 bytes),
                           },

which is the LLVM type { i32, f32 }, and that's obviously not the right type for S2.

(Caveat: I can't actually run code on these targets, so there's a small chance that I am completely misinterpreting what these PassMode mean. I hope that's not the case...)

I don't know what the C ABI on those platforms has to say about packed and aligned structs. Currently, we don't even provide ABIs a way to generate a packed LLVM struct for the arguments -- PassMode::Cast always uses an un-packed LLVM struct.

Updates:

• Fixed on RISC-V in Fix RISC-V C function ABI when passing/returning structs containing floats #139340
• SPARC64 PR: Fix passing/returning structs with the 64-bit SPARC ABI #142680

[bjorn3]

Looks like clang will split

struct P {
    int8_t a;
    float b;
} __packed__;

into two arguments on rv64gc with i8 and float respectively as types. Also I think clang will tell LLVM that the struct is packed if necessary on various targets. I also found this function: https://github.com/llvm/llvm-project/blob/d1487670ee1caaf1762c341f9f96cfb07540b5c1/clang/include/clang/CodeGen/CGFunctionInfo.h#L247-L250

[RalfJung]

into two arguments

So that's <{ i8, f32 }>? Or really two separate arguments to the function -- like Rust's ScalarPair? Or does that not even make a difference for the ABI?

[bjorn3]

In this case it seemed like two separate args were emitted. AFAIK LLVM will internally recursively split structs into separate arguments anyway though if byval is not used. At least that is what it does for Webassembly.

[heiher]

The #[repr(packed)] attribute explicitly defines and constrains the memory layout for types, while the extern "C" for functions requires the function's calling convention to be the same as in the C language. On some architectures with only 32-bit and 64-bit register widths, separating the passing of i8 and f32 can improve access performance. So, as long as parameter passing matches the C calling convention of the target architecture, it is correct. Isn't it?

[bjorn3]

The issue is with how we turn the value as represented in memory into an argument that can be passed in registers. Currently we assume that the value as represented in memory and the llvm type we use for argument passing have the same layout, which is not the case for aligned/packed structs as the llvm type has a different amount of padding from the real type.

[RalfJung]

The #[repr(packed)] attribute explicitly defines and constrains the memory layout for types, while the extern "C" for functions requires the function's calling convention to be the same as in the C language.

The issue is: when I declare a packed type in C, and a packed type in Rust, then they should be ABI-compatible. But currently they are not. If you take the type P from my example, declare an equivalent type in C, have a C function that takes a struct P as argument, and call it from Rust, then things will explode in the most amazing ways.

[heiher]

Thanks for your explanation. ❤️ I tried the test cases you mentioned, and indeed there is an issue when accessing packed structure fields through memory addresses. If I didn't make any mistakes, is x86_64 also wrong?

t.rs:

#[repr(packed, C)]
struct P {
    i: i8,
    f: f32,
}

#[no_mangle]
extern "C" fn test(p: P) {
    let ip = std::ptr::addr_of!(p.i);
    let fp = std::ptr::addr_of!(p.f);
    let i = unsafe { std::ptr::read_unaligned(ip) };
    let f = unsafe { std::ptr::read_unaligned(fp) };
    println!("{:p} {:p} {} {}", ip, fp, i, f);
}

t.c:

struct P
{
    char i;
    float f;
} __attribute__((packed));

extern void test(struct P p);

int
main (int argc, char *argv[])
{
    struct P p;
    
    p.i = 16;
    p.f = 128.0;

    test(p);

    return 0;
}

run:

rustc --crate-type=cdylib -o libt.so t.rs; gcc -o t t.c -L . -l t; LD_LIBRARY_PATH=`pwd` ./t

x86_64 with #[repr(packed)] and __attribute__((packed)):

0x7ffc31f02520 0x7ffc31f02521 16 128

x86_64 without #[repr(packed)] and __attribute__((packed)):

0x7ffc5a60eaa0 0x7ffc5a60eaa4 16 128

loongarch64 with #[repr(packed)] and __attribute__((packed)):

0x7ffffbf10720 0x7ffffbf10721 16 0.000000000000000000000000000000000000023509886

loongarch64 without #[repr(packed)] and __attribute__((packed)):

0x7ffffba12f90 0x7ffffba12f94 16 128

[RalfJung]

Interesting. x86_64 uses indirect pass mode for packed structs, so... that can't really be an ABI difference. Not sure what is happening. Do the field offsets match on both sides?