Skip to content

fix: keep streamed tool calls alive when a gateway skips index 0 - #1971

Merged
Astro-Han merged 2 commits into
mainfrom
fix/openai-compatible-toolcall-index
Aug 3, 2026
Merged

fix: keep streamed tool calls alive when a gateway skips index 0#1971
Astro-Han merged 2 commits into
mainfrom
fix/openai-compatible-toolcall-index

Conversation

@Astro-Han

@Astro-Han Astro-Han commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Every tool call fails when an OpenAI-compatible gateway labels a streamed tool_calls[].index with anything other than a contiguous run from 0. The turn dies mid-stream as "未知错误 / 已保留部分输出". Reported in #1967 against an Anthropic→OpenAI translating gateway, which reuses the Anthropic content-block index, so once a text block consumes index 0, the first tool call arrives as index 1.

index identifies which tool call a delta belongs to; it is not an array position. StreamingToolCallTracker in @ai-sdk/provider-utils treats it as one. It writes to this.toolCalls[index], leaving a hole at 0, and flush() then walks the array with for...of, which (unlike forEach) does not skip holes. It dereferences undefined.hasFinished, throws a bare TypeError inside the stream's flush handler, and the runtime, with no matching error class, reports it as unknown.

Upstream is not a viable path here. @ai-sdk/provider-utils@5.0.18 still ships the identical flush(), and the neighbouring tool-call fixes sit unmerged: of 877 open PRs on vercel/ai, 630 are community-authored with a median age of 79 days, and the closest one (vercel/ai#14277) has been open 116 days. So the fix is a patch-package patch applied in postinstall. Reported upstream anyway as vercel/ai#18333.

The patch fixes the crash, not the whole defect class: identity really lives in toolCallDelta.id, so a reused index and an omitted index stay broken exactly as they are upstream. patches/README.md records both, with evidence, so the patch is not later read as covering them.

Closes #1967

Review focus

patch-package is new infrastructure, and it is the reason @ai-sdk/openai matters here: it constructs the same StreamingToolCallTracker (node_modules/@ai-sdk/openai/dist/index.js:1211), so the openai, github-copilot chat, and custom-baseURL paths carry the identical bug. An SSE-rewriting fetch wrapper at the openai-compatible branch in model-factory.ts would cover strictly less ground while running a full SSE parse and re-serialize on every chunk of every connection.

Verification

Regression test packages/runtime/src/__tests__/model-factory-tool-call-index.test.ts drives getAIModel over a fake gateway, parameterised on index 0 / 1 / 7, plus two tool calls streamed across a hole at index 0. It asserts behaviour through the real provider stack and never references the patch, so it stays meaningful if upstream fixes this differently. Red before the patch, green after:

# node_modules restored to stock
ℹ pass 1
ℹ fail 3
    TypeError: Cannot read properties of undefined (reading 'hasFinished')
        at StreamingToolCallTracker.flush (@ai-sdk/provider-utils/dist/index.js:3570)

# node scripts/apply-dependency-patches.mjs
ℹ pass 4
ℹ fail 0

The multi-call case exists because the single-call cases all pass under a fix that appends each new index instead of honouring it, which would silently reorder real multi-call turns.

Install modes, each from a clean tree:

command result
npm ci exit 0, patch applied
npm ci --omit=dev exit 0, patch skipped with an explicit message
npm ci --workspace @maka/runtime exit 0, same
corrupted patch file exit 1, install blocked

The two middle rows are why postinstall calls a script instead of patch-package directly: both skip root devDependencies while still running the root postinstall, so a bare patch-package --error-on-fail exited 127 where main exits 0. Those trees are never what ships, and an unpatched one fails the regression test.

Also run: node --test "packages/runtime/dist/__tests__/*.test.js" (2622/2635; the 4 failures are builtin-tools path containment and reproduce identically on main on this machine, a macOS /var/private/var symlink issue), npm run lint, npm run format:check, both clean.

Packaged --dir and extracted app.asar to confirm the patch reaches the shipped app:

$ grep -n "toolCall == null" -A2 app.asar-extracted/node_modules/@ai-sdk/provider-utils/dist/index.js
3570:      if (toolCall == null) {
3571-        continue;
3572-      }

Not run: full npm run test, E2E, and a signed/notarized release build.

Rollout

patches/ and the postinstall change are new infrastructure. CI and the release workflow both run a plain npm ci, so the patch applies everywhere the app is built. Contributors with a warm node_modules need one npm install to pick it up. patch-package adds 294 lines to the lockfile; it is dev-only and does not reach the packaged app.

OpenAI-compatible gateways may label streamed `tool_calls[].index` with any
stable number — the field says which tool call a delta belongs to, not where it
sits in an array. Anthropic→OpenAI translators reuse the Anthropic content-block
index, so the first tool call arrives as index 1 once a text block consumed 0.

`StreamingToolCallTracker` stores tool calls at `toolCalls[index]`, leaving a
hole at 0, and `flush()` walks the array with `for...of`, which does not skip
holes. Dereferencing `undefined.hasFinished` throws, the stream dies, and the
turn surfaces as "未知错误 / 已保留部分输出" — every tool call through such a
gateway fails.

Patch `@ai-sdk/provider-utils` to skip empty slots, applied by patch-package in
postinstall with --error-on-fail so a stale patch blocks the install instead of
silently regressing. The regression test drives the real provider stack over a
fake gateway, so it holds whether the fix comes from the patch or from upstream.

Fixes #1967
`patch-package --error-on-fail` in the root postinstall broke two install
modes that work on main: `npm ci --workspace <name>` and `npm ci --omit=dev`
skip root devDependencies but still run the root postinstall, so both exited
127 with `patch-package: command not found`.

Move the call into a script that separates a missing tool from a failing
patch. An absent patch-package is reported and skipped, since those trees are
never what ships; a patch that exists but no longer applies still fails the
install. Every release and CI lane runs a plain root `npm ci`, and an unpatched
tree turns the tool-call index regression test red.

Also cover two tool calls streamed across an index hole. The single-call cases
pass under a fix that appends each new index instead of honouring it, which
would silently reorder real multi-call turns; and record in patches/README.md
that a reused index and an omitted index stay broken upstream, so the patch is
not read as covering the whole class.
@Astro-Han
Astro-Han force-pushed the fix/openai-compatible-toolcall-index branch from b1ee8f0 to 335d81b Compare August 3, 2026 09:48
@Astro-Han
Astro-Han merged commit ea51535 into main Aug 3, 2026
10 checks passed
@Astro-Han
Astro-Han deleted the fix/openai-compatible-toolcall-index branch August 3, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Maka crashes with 'Operation failed' when OpenAI-compatible gateway streams tool_calls with non-zero index

1 participant