[AVX512F] Suboptimal code generated for a _mm_mask_store_sd expansion

[ZviRackover]

Bugzilla Link	30563
Resolution	FIXED
Resolved on	Jan 27, 2019 06:40
Version	trunk
OS	All
Blocks	#29972
CC	@topperc,@igor-breger,@RKSimon
Fixed by commit(s)	286229, 331958

Extended Description

We currently do not support the AVX512F _mm_mask_store_sd [https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_mm_mask_store_sd] intrinics, but a reasonable expansion of this intrinsics to LLVM IR may be:

define void @​_mm_mask_store_sd(double* nocapture %__W, i8 zeroext %__U, <2 x double> %__A) local_unnamed_addr {
entry:
%tobool.i = trunc i8 %__U to i1
%mask = insertelement <2 x i1> <i1 undef, i1 false>, i1 %tobool.i, i32 0
%V = bitcast double * %__W to <2 x double>*
call void @​llvm.masked.store.v2f64.p0v2f64(<2 x double> %__A, <2 x double>* %V, i32 16, <2 x i1> %mask)
ret void
}
declare void @​llvm.masked.store.v2f64.p0v2f64(<2 x double>, <2 x double>*, i32, <2 x i1>)

which should be lowered to:

    kmovw     %esi, %k1
    vmovsd    %xmm0, (%rdi){%k1}

==========================================

'llc -mcpu=knl' generates:

    vmovq   %rsi, %xmm1
    vpsllq  $63, %xmm1, %xmm1
    vpsrad  $31, %xmm1, %xmm1
    vpshufd $245, %xmm1, %xmm1      # xmm1 = xmm1[1,1,3,3]
    vmaskmovpd      %xmm0, %xmm1, (%rdi)
    retq

==========================================

'llc -mcpu=skylake-avx512' generates:

    andl    $1, %esi
    kmovw   %esi, %k0
    kxorw   %k0, %k0, %k1
    kshiftrw        $1, %k1, %k1
    kshiftlw        $1, %k1, %k1
    korw    %k0, %k1, %k1
    vmovapd %xmm0, (%rdi) {%k1}
    retq

==========================================

[RKSimon]

Current codegen https://godbolt.org/z/t-_LR3

skx:

_mm_mask_store_sd: # @​_mm_mask_store_sd
kmovd %esi, %k0
kshiftlb $7, %k0, %k0
kshiftrb $7, %k0, %k1
vmovapd %xmm0, (%rdi) {%k1}
retq

knl:

_mm_mask_store_sd: # @​_mm_mask_store_sd
andl $1, %esi
kmovw %esi, %k1
vmovupd %zmm0, (%rdi) {%k1}
retq

[RKSimon]

Resolving, actual support for these intrinsics was added to the headers at rL286229, and tweaked at rL331958 to give the optimal codegen requested in the bug.

The IR we use is:

define @​test_mm_mask_store_sd(double*, i8 zeroext, <2 x double>) {
%4 = bitcast double* %0 to <2 x double>*
%5 = and i8 %1, 1
%6 = bitcast i8 %5 to <8 x i1>
%7 = shufflevector <8 x i1> %6, <8 x i1> undef, <2 x i32> <i32 0, i32 1>
tail call void @​llvm.masked.store.v2f64.p0v2f64(<2 x double> %2, <2 x double>* %4, i32 1, <2 x i1> %7)
ret void
}
declare void @​llvm.masked.store.v2f64.p0v2f64(<2 x double>, <2 x double>*, i32, <2 x i1>)

@​Zvi - please reopen this if you think we have a need to optimise your alternative IR any further than [Comment #​1].

[ZviRackover]

mentioned in issue #29972