Skip to content

openai: handle newer Responses stream event shapes#2976

Merged
dgageot merged 1 commit into
docker:mainfrom
rumpl:fix/openai-responses-stream-events
Jun 3, 2026
Merged

openai: handle newer Responses stream event shapes#2976
dgageot merged 1 commit into
docker:mainfrom
rumpl:fix/openai-responses-stream-events

Conversation

@rumpl
Copy link
Copy Markdown
Member

@rumpl rumpl commented Jun 2, 2026

Newer Responses API models can announce assistant text with response.content_part.added using part.type=output_text. The adapter only recognized text, so those msg_* items fell into a misleading tool-call warning path even though no function call was involved.

Keep response.content_part.added structural. Text is streamed by response.output_text.delta, and output_item.done carries a final snapshot. Emitting both snapshots and deltas duplicates assistant text.

Use output_item.done only as a fallback for missed text. Some streams omit the top-level item_id there while still setting item.id, so resolve that ID before checking whether content was already streamed.

Also tolerate function_call_arguments.delta arriving before the matching output_item.added event by buffering the argument bytes until the tool item is registered. This avoids dropping tool arguments when stream events are delivered in that order.

Add coverage for output_text content parts, output_item.done fallback and early function-call argument deltas.

@rumpl rumpl requested a review from a team as a code owner June 2, 2026 23:46
Copy link
Copy Markdown

@docker-agent docker-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

This PR correctly handles the newer Responses API stream event shapes. The structural treatment of response.content_part.added, the itemID resolution fallback in output_item.done, and the pendingArgs buffering mechanism are all well-designed. One latent edge case was found in the argument-flush path.

// delta before the output_item.added event. Flush any such buffered
// bytes with the first named tool-call delta so the runtime can still
// reconstruct the call.
if funcName != "" {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] pendingArgs silently lost when funcName is empty in output_item.added

When a response.output_item.added event arrives for a function_call item, the buffered early-argument bytes (pendingArgs[itemID]) are only flushed inside the if funcName != "" block. If the function name is absent from this event (e.g., the name arrives later via a separate event or delta), the item is registered in itemCallIDMap, but the pre-buffered argument bytes are never deleted or delivered. Any function_call_arguments.delta events that arrived before output_item.added — and were buffered correctly — are then permanently stranded in pendingArgs and silently dropped.

if funcName != "" {
    args := a.pendingArgs[itemID]
    delete(a.pendingArgs, itemID)  // ← only reached when name is known
    // ... emit toolcall
}
// No else: if funcName == "", pendingArgs[itemID] is never freed

The itemCallIDMap registration at line 108 ensures future argument deltas emit directly (correct), but the already-buffered bytes are orphaned. Consider unconditionally flushing pendingArgs[itemID] whenever the item is first registered, regardless of whether the name is available, and handle the name-absent case separately.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

function names can't be empty

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right — I was wrong about this. Looking at the test TestWSStream_BuffersArgumentsDeltaBeforeOutputItemAdded and the OpenAI Responses API contract, output_item.added events for function_call items always include the function name in item.name. The if funcName != "" check on line 112 is defensive but will always be true for well-formed function calls, so pendingArgs will always be flushed correctly when the item is announced.

I misunderstood the API semantics and flagged correct code as buggy. Thanks for the correction, and sorry for the noise!

dgageot
dgageot previously approved these changes Jun 3, 2026
Newer Responses API models can announce assistant text with
response.content_part.added using part.type=output_text. The adapter only
recognized text, so those msg_* items fell into a misleading tool-call warning
path even though no function call was involved.

Keep response.content_part.added structural. Text is streamed by
response.output_text.delta, and output_item.done carries a final snapshot.
Emitting both snapshots and deltas duplicates assistant text.

Use output_item.done only as a fallback for missed text. Some streams omit the
top-level item_id there while still setting item.id, so resolve that ID before
checking whether content was already streamed.

Also tolerate function_call_arguments.delta arriving before the matching
output_item.added event by buffering the argument bytes until the tool item is
registered. This avoids dropping tool arguments when stream events are
delivered in that order.

Add coverage for output_text content parts, output_item.done fallback and
early function-call argument deltas.

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
@rumpl rumpl force-pushed the fix/openai-responses-stream-events branch from f85a457 to 49d4e5f Compare June 3, 2026 07:56
@dgageot dgageot merged commit 6d98c17 into docker:main Jun 3, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants