Skip to content

fix(service): reassemble streamed assistant replies - #1107

Merged
edwin-zvs merged 1 commit into
mainfrom
fix-service-reply-streaming
Aug 1, 2026
Merged

fix(service): reassemble streamed assistant replies#1107
edwin-zvs merged 1 commit into
mainfrom
fix-service-reply-streaming

Conversation

@edwin-zvs

Copy link
Copy Markdown
Contributor

The bug

GET /svc/<name>/sessions/<id> read the reply as the last assistant Message event. But harnesses stream a turn as one Message event per token delta, so the "reply" was the final fragment rather than the answer.

Verified against a live daemon with an openai:gpt-5 service session:

model actually said:  'Red, green, blue.'
API returned:         '.'

The transcript for "what is 2 plus 2" arrives as eight separate events: "2", " plus", " ", "2", " is", " ", "4", ".".

This survived testing because single-token answers mask it — a service asked "17×3" returns 51 correctly, since the last delta is the whole reply. Every real multi-word answer was silently truncated.

The fix

Concatenate the trailing assistant run instead. Walking backwards, transcript bookkeeping (status, cost, usage, reasoning) is skipped, and collection stops at the first boundary that ends the final answer — the user's message, or a tool call whose narration precedes it. A tool-using turn therefore reports its answer rather than the commentary leading up to it.

Extracted into a testable helper; there was previously no coverage of reply extraction.

Verification

Same live-daemon setup, after the fix:

API reply:      'Red, green, blue.'
transcript:     'Red, green, blue.'
delta count:    6

Also re-checked: keyed routing still reuses sessions with context intact, a turn parked on tool approval correctly reports ready:false / reply:null.

Tests: 4 new unit tests (streaming deltas, turn boundary, tool-use narration, no-reply-yet). construct-daemon --lib 483 pass, construct-cli --bins 1314 pass, workspace builds clean.

This PR only touches crates/daemon → the relevant binary is construct.

A service session's reply was read as the last assistant Message event,
but harnesses stream a turn as one Message per token delta. Every
multi-token answer was therefore truncated to its final fragment: a
session that replied "Red, green, blue." returned "." over HTTP.
Single-token answers masked this, so it survived manual testing.

Concatenate the trailing assistant run instead, skipping transcript
bookkeeping and stopping at the user's message or a tool call, so a
tool-using turn reports its answer rather than the narration before it.
@edwin-zvs
edwin-zvs merged commit 3d222af into main Aug 1, 2026
1 check passed
@edwin-zvs
edwin-zvs deleted the fix-service-reply-streaming branch August 1, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant