openai: handle newer Responses stream event shapes#2976
Conversation
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
This PR correctly handles the newer Responses API stream event shapes. The structural treatment of response.content_part.added, the itemID resolution fallback in output_item.done, and the pendingArgs buffering mechanism are all well-designed. One latent edge case was found in the argument-flush path.
| // delta before the output_item.added event. Flush any such buffered | ||
| // bytes with the first named tool-call delta so the runtime can still | ||
| // reconstruct the call. | ||
| if funcName != "" { |
There was a problem hiding this comment.
[MEDIUM] pendingArgs silently lost when funcName is empty in output_item.added
When a response.output_item.added event arrives for a function_call item, the buffered early-argument bytes (pendingArgs[itemID]) are only flushed inside the if funcName != "" block. If the function name is absent from this event (e.g., the name arrives later via a separate event or delta), the item is registered in itemCallIDMap, but the pre-buffered argument bytes are never deleted or delivered. Any function_call_arguments.delta events that arrived before output_item.added — and were buffered correctly — are then permanently stranded in pendingArgs and silently dropped.
if funcName != "" {
args := a.pendingArgs[itemID]
delete(a.pendingArgs, itemID) // ← only reached when name is known
// ... emit toolcall
}
// No else: if funcName == "", pendingArgs[itemID] is never freedThe itemCallIDMap registration at line 108 ensures future argument deltas emit directly (correct), but the already-buffered bytes are orphaned. Consider unconditionally flushing pendingArgs[itemID] whenever the item is first registered, regardless of whether the name is available, and handle the name-absent case separately.
There was a problem hiding this comment.
function names can't be empty
There was a problem hiding this comment.
You're absolutely right — I was wrong about this. Looking at the test TestWSStream_BuffersArgumentsDeltaBeforeOutputItemAdded and the OpenAI Responses API contract, output_item.added events for function_call items always include the function name in item.name. The if funcName != "" check on line 112 is defensive but will always be true for well-formed function calls, so pendingArgs will always be flushed correctly when the item is announced.
I misunderstood the API semantics and flagged correct code as buggy. Thanks for the correction, and sorry for the noise!
Newer Responses API models can announce assistant text with response.content_part.added using part.type=output_text. The adapter only recognized text, so those msg_* items fell into a misleading tool-call warning path even though no function call was involved. Keep response.content_part.added structural. Text is streamed by response.output_text.delta, and output_item.done carries a final snapshot. Emitting both snapshots and deltas duplicates assistant text. Use output_item.done only as a fallback for missed text. Some streams omit the top-level item_id there while still setting item.id, so resolve that ID before checking whether content was already streamed. Also tolerate function_call_arguments.delta arriving before the matching output_item.added event by buffering the argument bytes until the tool item is registered. This avoids dropping tool arguments when stream events are delivered in that order. Add coverage for output_text content parts, output_item.done fallback and early function-call argument deltas. Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
f85a457 to
49d4e5f
Compare
Newer Responses API models can announce assistant text with response.content_part.added using part.type=output_text. The adapter only recognized text, so those msg_* items fell into a misleading tool-call warning path even though no function call was involved.
Keep response.content_part.added structural. Text is streamed by response.output_text.delta, and output_item.done carries a final snapshot. Emitting both snapshots and deltas duplicates assistant text.
Use output_item.done only as a fallback for missed text. Some streams omit the top-level item_id there while still setting item.id, so resolve that ID before checking whether content was already streamed.
Also tolerate function_call_arguments.delta arriving before the matching output_item.added event by buffering the argument bytes until the tool item is registered. This avoids dropping tool arguments when stream events are delivered in that order.
Add coverage for output_text content parts, output_item.done fallback and early function-call argument deltas.