The __m64 not passed according to i386 ABI

[llvmbot]

Bugzilla Link	41029
Version	trunk
OS	Linux
Depends On	#41664
Reporter	LLVM Bugzilla Contributor
CC	@topperc,@hjl-tools,@jyknight,@RKSimon,@zygoloid

Extended Description

$ cat m64.c
#include <immintrin.h>
void callee(__m64 __m1, __m64 __m2);
__m64 caller(__m64 __m1, __m64 __m2)
{
  __m64 a = _mm_set_pi16(1, 2, 3, 4);
  callee(__m2, __m1);
  return a;
}
$ gcc -m32 -O2 -S -o - m64.c
        .file   "m64.c"
        .text
        .p2align 4
        .globl  caller
        .type   caller, @function
caller:
.LFB5121:
        .cfi_startproc
        subl    $12, %esp
        .cfi_def_cfa_offset 16
        movq    %mm0, %mm2
        movq    %mm1, %mm0
        movq    %mm2, %mm1
        call    callee
        movq    .LC0, %mm0
        addl    $12, %esp
        .cfi_def_cfa_offset 4
        ret
        .cfi_endproc
.LFE5121:
        .size   caller, .-caller
        .section        .rodata.cst8,"aM",@progbits,8
        .align 8
.LC0:
        .value  4
        .value  3
        .value  2
        .value  1
        .ident  "GCC: (GNU) 9.0.1 20190131 (experimental)"
        .section        .note.GNU-stack,"",@progbits

$ clang -m32 -O2 -S -o - m64.c
        .text
        .file   "m64.c"
        .globl  caller                  # -- Begin function caller
        .p2align        4, 0x90
        .type   caller,@function
caller:                                 # @caller
# %bb.0:                                # %entry
        subl    $12, %esp
        pushl   20(%esp)
        pushl   20(%esp)
        pushl   36(%esp)
        pushl   36(%esp)
        calll   callee
        addl    $16, %esp
        movl    $196612, %eax           # imm = 0x30004
        movl    $65538, %edx            # imm = 0x10002
        addl    $12, %esp
        retl
.Lfunc_end0:
        .size   caller, .Lfunc_end0-caller
                                        # -- End function

        .ident  "clang version 9.0.0 (http://llvm.org/git/clang.git 59f2009cd157fc96a0d558125405b98586cd83d2) (http://llvm.org/git/llvm.git 6a7719c7965af52f904e16588c1754f65bcb8ff0)"
        .section        ".note.GNU-stack","",@progbits
        .addrsig

According to i386 ABI, __m64 values should be passed by mmx registers.

[jyknight]

Looks like this was originally changed (broken) in 651c183 back in 2013.

https://reviews.llvm.org/D59744 is a recent attempt to fix this bug, but was reverted because it broke (at least) chromium on x86-32 -- in part due to to bug 42319.

[jyknight]

I think that it's likely preferable to continue violating this ABI requirement indefinitely, and not fix this. Clang has already been violating it for 7+ years, and there's not a whole lot of demand to change here.

And, unfortunately, there's a very significant downside to changing, here. Adding any more usage of MMX is a giant foot-gun, due to the x87/mmx mode-switching issues.

After llvm/llvm-bugzilla-archive#42320 is implemented, there will be no use of MMX from clang, aside from inline-assembly. Adding back the hassle of accidental MMX mode-switch when passing or returning an __m64 would be extremely unfortunate -- it's just not worth it.

I do think it's unfortunate that GCC's and clang's ABI when built with -mno-mmx are not compatible.

E.g. given this function:
__m64 mmx() {
return (__m64)55LL;
}

gcc -O2 -mno-mmx -m32 treats it as if the return type was 'struct X { int a, int b}':
mmx():
movl 4(%esp), %eax
movl $55, (%eax)
movl $0, 4(%eax)
ret $4

clang -O2 -mno-mmx -m32 treats it as if the return type were 'long long':
mmx(): # @​mmx()
movl $55, %eax
xorl %edx, %edx
retl

[RKSimon]

mentioned in issue #41664