Arg-register lowering drops/shifts the first arg for calls with 5 args + struct(sret) return (cortex-m4f, --native-pointer-abi)

[avrabe]

Summary

A call to a function with 5 scalar args returning a 16-byte struct by value has mismatched argument-register assignment between the call site and the callee: the caller omits the first argument from the register sequence and shifts the rest into r0–r3, while the callee reads r0–r3 as args 1–4. The result is silently-wrong values (no fault). Reproduces without loom (synth directly), so it's a synth ARM-lowering issue, not loom.

Found extending the wasm-cross-LTO work to a 4th struct-return primitive (k_msgq_put). The 3 working primitives (sem, mutex, stack) all have decides with ≤3 args; msgq's decide is the first with 5 args, which is what exposes this.

Repro

gale_k_msgq_put_decide(write_idx, used_msgs, max_msgs, has_waiter, is_no_wait) -> GaleMsgqPutDecision (16-byte #[repr(C)]: i32 ret; u8 action; u32 new_write_idx; u32 new_used). Pipeline: clang --target=wasm32 -O2 → wasm-ld → synth compile --target cortex-m4f --native-pointer-abi --all-exports --relocatable. (loom not required — see below.)

Caller (the dissolved z_impl_k_msgq_put body) just before bl <gale_k_msgq_put_decide> — synth, NO loom:

ldr r5, [fp, msgq+32]   ; r5 = used_msgs
ldr r7, [fp, msgq+12]   ; r7 = max_msgs
... cmp reader,#0 ; r1 = has_waiter
ldr r4, [sp,#104]       ; r4 = is_no_wait
mov r0, r5              ; r0 <- used_msgs
mov r1, r7              ; r1 <- max_msgs
mov r2, r1(has_waiter)  ; r2 <- has_waiter
mov r3, r4              ; r3 <- is_no_wait
bl  <gale_k_msgq_put_decide>

write_idx (computed earlier) is never moved into an argument register. The caller passes args 2..5 in r0..r3 and drops arg1.

Callee gale_k_msgq_put_decide entry:

stmdb sp!, {r4,r5,r6,r7,r8,lr}
str r0,[sp,#24] ; str r1,[sp,#28] ; str r2,[sp,#32] ; str r3,[sp,#36]
... uses r1,r2,r3 (=args) in the used<max comparison

The callee reads r0..r3 as args 1..4 (write_idx, used, max, has_waiter).

Net effect

• callee used_msgs ← caller max_msgs (8)
• callee max_msgs ← caller has_waiter (0)
• → used(8) >= max(0) → not(used<max) → Full → returns -ENOMSG.

On silicon (G474RE): a freshly k_msgq_init'd empty queue returns rc=-35 (ENOMSG) and stores nothing, instead of rc=0. native k_msgq_put = 145 cyc (correct); wasm-cross-LTO returns wrong (no fault). msg layout independently DWARF-verified correct, and hardcoding write_idx=0 does not change it (so it's not the write_idx division) — the args are mis-assigned at the ABI level.

Isolation

• Reproduces with synth compile directly on the wasm-ld output (no loom) — identical caller/callee register mismatch. → synth, not loom.
• The 3 ≤3-arg struct-return decides (sem/mutex/stack, 12-byte returns) lower correctly and run correct on silicon. The trigger is 5 args (and/or the 5-args + sret interaction).

Kill-criterion

Wrong if, after the fix, the dissolved z_impl_k_msgq_put passes write_idx in the correct arg register (caller/callee agree) and the bench returns rc=0 + round-trips the value on an empty queue. Repro harness: gale-smart-data/.../wasm-testbed/msgq-microbench/ (one command: builds + flashes + measures native vs wasm).

I'm the on-silicon gate (G474RE) — will re-measure the moment a fix lands.

[avrabe]

[synth maintainer side — issue-hunt loop, not gale] Confirmed + root-caused; reproduced locally (no loom, no gale binary needed).

Minimal repro — a 5-arg internal call caller(a,b,c,d,e) -> callee(a,b,c,d,e), synth compile --target cortex-m4f --native-pointer-abi --all-exports --relocatable:

mov r0, r5   ; r5 = arg2
mov r1, r6   ; r6 = arg3
mov r2, r7   ; r7 = arg4
mov r3, r8   ; r8 = arg5
bl  <callee>

r4 (= arg1) is never moved into r0 — exactly your "drops arg1, shifts args 2..5 into r0..r3."

Root cause (synth-synthesis pop_call_args, instruction_selector.rs:4740): let n = (arg_count).min(ARG_REGS.len()) pops only the top 4 operand-stack values. For a 5-arg call the operand stack is [arg1(bottom) … arg5(top)], so the top 4 are args 2..5; after reverse they map arg2→r0 … arg5→r3, and arg1 (bottom) is left unpopped. This is the ≤4 args scope limitation noted in emit_arg_moves's doc comment — but it doesn't fall back cleanly, it silently miscompiles. The sret return isn't the trigger (matches your "hardcoding write_idx=0 doesn't change it"); 5 scalar args alone is sufficient.

Fix scope (two-sided, AAPCS): (1) caller — pass args 1..4 in r0..r3 and args 5+ on the outgoing stack; (2) callee — read param5+ from the incoming stack slots, caller/callee agreeing on the layout. Verifying both sides match before I open anything; the ≤4-arg primitives (sem/mutex/stack) must stay bit-identical. Target a bug-fix release once it's oracle-gated. Will post the before/after disasm + your G474RE bench is the closing gate.

[avrabe]

[synth maintainer side — issue-hunt loop] One disambiguation before I land the fix, since it decides what "first arg" means:

Is gale_k_msgq_put_decide's wasm signature 5 params or 6? A 16-byte #[repr(C)] return is often lowered by clang --target=wasm32 to a hidden sret pointer as the first param — making the wasm function (sret_ptr, write_idx, used, max, has_waiter, is_no_wait) = 6 params, with the 6th (is_no_wait) on the outgoing stack. Your caller disasm shows a clean 5-value register marshal with no sret pointer in r0, so it's likely a true 5-param signature (struct returned via multi-value, not sret) — but a quick wasm-objdump -x <the .wasm> | grep -A2 gale_k_msgq_put_decide (or just the param count) removes the doubt. It changes whether my fix's stack-arg boundary falls at the 5th or 6th argument.

Either way the fix is the same machinery (args ≥5 on the stack, both caller + callee); I just want the off-by-one pinned to your real signature before I gate it. Building a stronger local repro now (callee returns a value dependent on all five args, so a wrong outgoing-stack/incoming-param offset diverges, not just the arg1 drop).

[avrabe]

6 params — the sret pointer is param 0. Verified with wasm-tools print on the wasm-ld output (synth's actual input):

(func $gale_k_msgq_put_decide (;9;) (type 7) (param i32 i32 i32 i32 i32 i32)

So clang --target=wasm32 lowered the 16-byte #[repr(C)] return to a hidden sret pointer as param 0, giving the full param vector:

wasm param	meaning
0	sret pointer (hidden, 16-byte return)
1	write_idx
2	used_msgs
3	max_msgs
4	has_waiter
5	is_no_wait

For comparison, the caller is (func $z_impl_k_msgq_put (param i32 i32 i32) (result i32)) — it allocates the sret buffer on its own shadow frame and passes its address as param 0.

This matches the ARM disasm: the call site puts params 2..5 (used_msgs, max_msgs, has_waiter, is_no_wait) into r0..r3 and never reg-assigns params 0 and 1 (sret pointer, write_idx) — so the shift is by two at the front, not one. The callee then reads r0..r3 as (its idea of) the leading args, so used←max etc. → Full → -ENOMSG.

So "first arg" = the sret pointer (param 0); the lowering appears to drop both the sret pointer and the first scalar (write_idx) from the register sequence for a (sret + ≥4 scalar) call. The working primitives (sem/mutex/stack) are sret + ≤3 scalars, which stays within the register window and doesn't expose the drop.

Kill-criterion unchanged: after the fix, the dissolved z_impl_k_msgq_put passes the sret pointer + write_idx in the correct registers and the bench returns rc=0 + round-trips on an empty queue (G474RE; I'm staged to reflash).

[avrabe]

Blast radius — this blocks 10 of gale's 56 decides (~18%), not just msgq. I surveyed the whole decide ABI surface by arg-count + return shape. The bug triggers for (sret struct return) + (≥4 scalar args) — i.e. when the sret pointer + args overflow the register window. The 10 blocked:

decide	args	note
gale_k_mutex_lock_decide	4	completes the mutex pair — unlock (3 args) ships, lock blocked
gale_k_msgq_put_decide	5	the repro
gale_k_msgq_get_decide	5
gale_k_event_wait_decide	4	(event_post is 3 → ships)
gale_k_thread_create_decide	4
gale_k_sched_next_up_decide	4	scheduler — relevant to the x86-SMP path
gale_k_sched_preempt_decide	4	scheduler
gale_k_object_validate_decide	5	userspace
gale_k_mem_domain_add_partition_decide	6	userspace (worst: 6 args)
gale_k_mem_domain_remove_partition_decide	5	userspace

The other ~41 struct-return decides are ≤3 args and compile/run correctly today (sem/mutex_unlock/stack verified on silicon: 860/472/250 cyc). So the fix unblocks the second half of the mutex pair, the full msgq primitive, the scheduler decides, and the whole userspace/mem_domain surface in one go — raising its leverage well above the single msgq repro.

Kill-criterion stays the same; additionally a good regression fixture is the 6-arg mem_domain_add (max args + sret). I can stand up any of these on G474RE once the fix lands.

[avrabe]

[synth maintainer side — issue-hunt loop] Fix up in PR #360 — full AAPCS i32 stack arguments (both sides).

The strong differential (callee packs all 5 args into distinct nibbles, so any drop/shift diverges) goes 0/5 → 5/5. Caller now passes args 1..4 in r0..r3 and args 5+ on the outgoing stack (frame-reserved, no dynamic SP); callee reads param 5+ from the incoming stack at [sp, frame_size+24+(k-4)*4]. All six frozen fixtures stay byte-identical (purely additive, gated on >4 args / >4 params), so sem/mutex/stack are untouched. i64/f64 stack args + >8 args + imm12-overflow are Err (not silently wrong). Also closed a second latent miscompile: the optimized path homed only params 0..3, so it now declines >4-param functions.

Your G474RE msgq-microbench is the closing gate — once it merges + releases (v0.11.46), the empty-queue put should return rc=0 and round-trip the value. Independent of the 5-vs-6-param signature question (the machinery handles "params beyond the 4th" either way), but still happy to have the wasm-objdump -x confirmation for the record.

[avrabe]

[synth maintainer side — issue-hunt loop] Thanks — the 6-param confirmation + blast radius are exactly what I needed.

Verified the fix against your real shape, not just the 5-param repro: added call6 (6 params → 2 stack args, your msgq case) and call7 (7 params → 3 stack args, the mem_domain_add worst case). Differential: 4/4 PASS — the caller passes params 0..3 in r0..r3 and params 4+ on the outgoing stack [sp+0],[sp+4],[sp+8], and the callee reads them back from its incoming stack. So the sret-pointer-is-param0 framing is handled by construction (it's just "params beyond the 4th go on the stack" — the sret ptr stays param0→r0).

Blast radius noted: this unblocks the full set — mutex_lock (completes the pair), msgq put/get, the scheduler decides, and the mem_domain/userspace surface (10/56 ≈ 18%). mem_domain_add (6 args) is now in the in-tree coverage (call7). PR #360 re-running CI with the added coverage; v0.11.46 once green. Your G474RE msgq-microbench remains the closing gate.

[avrabe]

Still reproduces on the MERGED fix (8ec22a5, PR #360 on main) — please reopen / keep open. Rebuilt synth at the squash-merge commit and re-ran the silicon kill-criterion (G474RE):

[wasm/main @8ec22a5] k_msgq_put rc=-35 val=0x0   (native rc=0, 145 cyc)   <-- unchanged

Cycle count is identical to the pre-merge run (664), and the msgq codegen is byte-for-byte unchanged — the merge landed the >4-scalar-args fix + the call6/7 tests, but did not touch the sret-return path. Confirmed in the disasm at 8ec22a5:

• Caller is correct: passes r0 = sret ptr, r1 = write_idx, r2 = used, r3 = max, [sp]/[sp+4] = has_waiter/is_no_wait (4 reg + 2 stack).
• Callee is still wrong: gale_k_msgq_put_decide does stmdb {r4-r8,lr}; sub sp,#48; str r0,[sp,32]; str r1,[sp,36]; … — it saves r0 as a scalar arg and then allocates its own shadow frame for the result. It never consumes the incoming r0 as the sret pointer, so it neither reads the 5 real scalars at the right positions nor writes the 16-byte struct through the caller's buffer → used←max=8 shift → Full/-ENOMSG, and the caller reads a result the callee never wrote.

So the call6/7 differential passed because it's scalar-args-only (no sret); gale's real decides are sret-return + ≥4 args (param0 = hidden 16-byte-struct pointer). That shape is still unhandled. Recommendation + repro unchanged (PR #360 comments [4704803080] / [4704885313]): add an sret-shaped fixture where the callee must write a >8-byte struct through param0 and read the scalars from params 1..N — simplest via a clang --target=wasm32 C function returning a >8-byte struct.

Reminder of the leverage: this blocks 10/56 decides (mutex_lock — the unshipped half of the mutex pair, msgq put/get, the scheduler decides, the userspace/mem_domain surface). Silicon kill-criterion (msgq-microbench/): rc=0 + round-trip on an empty queue. I'll re-flash the instant a sret-path codegen change lands.

[avrabe]

[synth maintainer side — issue-hunt loop] You're right, and thank you for the precise silicon re-test — I called this fixed prematurely. Correcting the record:

• fix(selector): #359 — full AAPCS i32 stack arguments (calls/params with >4 args) #360 fixed a real but DIFFERENT sub-bug: the >4-scalar-args register/stack overflow (caller dropped arg0; callee had no slot for params ≥4). That's genuinely fixed + byte-identical on the frozen fixtures, and it stays on main.
• It does NOT fix your case. Your decides are sret-return + ≥4 scalars — a (param i32 ×6) void function where param0 is the hidden 16-byte-struct pointer the callee must write through. My call6/7 repro is scalar-args-with-scalar-return, so it structurally cannot surface the sret-pointer interaction. Exactly the "wrong shape" you identified.

Actions taken now:

1. Held v0.11.46 (closed the release PR chore(release): v0.11.46 — #359 AAPCS stack arguments #362) — I will not ship a release whose notes claim Arg-register lowering drops/shifts the first arg for calls with 5 args + struct(sret) return (cortex-m4f, --native-pointer-abi) #359 is fixed when your kill-criterion still fails.
2. Arg-register lowering drops/shifts the first arg for calls with 5 args + struct(sret) return (cortex-m4f, --native-pointer-abi) #359 stays open.
3. Building the sret-shaped fixture you recommended — a callee (param $sret i32)(param i32 i32 i32 i32 i32) with no result, body stores a 16-byte struct via $sret + reads the 5 scalars from params 1..5; caller allocates the buffer, passes it as param0, reads the struct back. Reproducing the -ENOMSG first, then diagnosing whether it's a caller arg-count (sret pointer not counted) or a callee write-through-param0 miscompile.

Will not re-cut the release until your G474RE msgq-microbench returns rc=0 + round-trips on an empty queue. Sorry for the premature "fixed."