wasm32-unknown-unknown "C" ABI doesn't use proper ABI adjustments

[RalfJung]

rustc has a PassMode enum to declare how arguments should be passed between functions. PassMode::Direct is used for the simple case where we can just generate LLVM IR with an argument type that matches the usual LLVM type for this Rust type. It should only be used for types with Scalar or Vector ABI; for larger types, Rust will generate LLVM struct/union types and those

• are sometimes adjusted to obtain better code generation
• make it very hard to reason about our effective ABI, since it depends on how LLVM lowers those complex struct types into an effective ABI. In particular, this causes a huge risk of violating our ABI compatibility guarantees, e.g. for repr(transparent) types.

Unfortunately the compiler never had an assertion ensuring that PassMode::Direct is only used with Scalar/Vector types, and so one target accidentally broke this rule: wasm32-unknown-unknown. Later all wasm targets got an extern "wasm" ABI that has the same issue. We should find some way to fix this target, as the current situation makes life harder for anyone wanting to tweak our Rust-to-LLVM-type-mapping, and for alternative backends that have to match our effective ABI.

The issue is that last time @bjorn3 tried to fix this, it lead to #81386 and had to be reverted. It will be basically impossible to do this change without breaking the ABI at least for some cases. I'm not sure how much ABI breakage we can get away with. This might have to be blocked on rust-lang/compiler-team#672 which should help to implement the current effective ABI at least for types that are ABI-stable (i.e., repr(C) and repr(transparent)).

Cc #122532

[kjetilkjeka]

I know this issue is related to wasm. But it's also the one that is linked to when hitting the assert failure when using PassMode::Direct with abi::Abi::Aggregate and this is a general question about that.

On the nvptx64 target normal device functions does not hit this assert due to using make_indirect() for aggregates. This is not possible for functions with the Abi::PtxKernel abi due to a call not being a proper call but instead involving the copy of the arguments from CPU (and a copied pointer would usually be invalid). I have a few questions so I'm able to understand and resolve this problem.

What kind of details leaks from llvm to the abi when using PassMode::Direct?

Is PassMode::Cast to an Uniform with the correct alignment and size the correct one to use? It states that a uniform is based on registers, but I assume that can be very loosely translated to copying the data from CPU to GPU.

EDIT: the solution with PassMode::Cast ends up breaking the abi tests for extern "ptx-kernel" fn. This abi is unstable and feature gated so it's luckily not a backward compatibility issue.

[RalfJung]

@kjetilkjeka please open a new issue; this one here is for the wasm ABI problems.