fix: keep streamed tool calls alive when a gateway skips index 0 - #1971
Merged
Conversation
Astro-Han
marked this pull request as ready for review
August 3, 2026 09:47
OpenAI-compatible gateways may label streamed `tool_calls[].index` with any stable number — the field says which tool call a delta belongs to, not where it sits in an array. Anthropic→OpenAI translators reuse the Anthropic content-block index, so the first tool call arrives as index 1 once a text block consumed 0. `StreamingToolCallTracker` stores tool calls at `toolCalls[index]`, leaving a hole at 0, and `flush()` walks the array with `for...of`, which does not skip holes. Dereferencing `undefined.hasFinished` throws, the stream dies, and the turn surfaces as "未知错误 / 已保留部分输出" — every tool call through such a gateway fails. Patch `@ai-sdk/provider-utils` to skip empty slots, applied by patch-package in postinstall with --error-on-fail so a stale patch blocks the install instead of silently regressing. The regression test drives the real provider stack over a fake gateway, so it holds whether the fix comes from the patch or from upstream. Fixes #1967
`patch-package --error-on-fail` in the root postinstall broke two install modes that work on main: `npm ci --workspace <name>` and `npm ci --omit=dev` skip root devDependencies but still run the root postinstall, so both exited 127 with `patch-package: command not found`. Move the call into a script that separates a missing tool from a failing patch. An absent patch-package is reported and skipped, since those trees are never what ships; a patch that exists but no longer applies still fails the install. Every release and CI lane runs a plain root `npm ci`, and an unpatched tree turns the tool-call index regression test red. Also cover two tool calls streamed across an index hole. The single-call cases pass under a fix that appends each new index instead of honouring it, which would silently reorder real multi-call turns; and record in patches/README.md that a reused index and an omitted index stay broken upstream, so the patch is not read as covering the whole class.
Astro-Han
force-pushed
the
fix/openai-compatible-toolcall-index
branch
from
August 3, 2026 09:48
b1ee8f0 to
335d81b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every tool call fails when an OpenAI-compatible gateway labels a streamed
tool_calls[].indexwith anything other than a contiguous run from 0. The turn dies mid-stream as "未知错误 / 已保留部分输出". Reported in #1967 against an Anthropic→OpenAI translating gateway, which reuses the Anthropic content-block index, so once a text block consumes index 0, the first tool call arrives as index 1.indexidentifies which tool call a delta belongs to; it is not an array position.StreamingToolCallTrackerin@ai-sdk/provider-utilstreats it as one. It writes tothis.toolCalls[index], leaving a hole at 0, andflush()then walks the array withfor...of, which (unlikeforEach) does not skip holes. It dereferencesundefined.hasFinished, throws a bareTypeErrorinside the stream's flush handler, and the runtime, with no matching error class, reports it as unknown.Upstream is not a viable path here.
@ai-sdk/provider-utils@5.0.18still ships the identicalflush(), and the neighbouring tool-call fixes sit unmerged: of 877 open PRs onvercel/ai, 630 are community-authored with a median age of 79 days, and the closest one (vercel/ai#14277) has been open 116 days. So the fix is apatch-packagepatch applied inpostinstall. Reported upstream anyway as vercel/ai#18333.The patch fixes the crash, not the whole defect class: identity really lives in
toolCallDelta.id, so a reused index and an omitted index stay broken exactly as they are upstream.patches/README.mdrecords both, with evidence, so the patch is not later read as covering them.Closes #1967
Review focus
patch-packageis new infrastructure, and it is the reason@ai-sdk/openaimatters here: it constructs the sameStreamingToolCallTracker(node_modules/@ai-sdk/openai/dist/index.js:1211), so theopenai,github-copilotchat, and custom-baseURL paths carry the identical bug. An SSE-rewriting fetch wrapper at theopenai-compatiblebranch inmodel-factory.tswould cover strictly less ground while running a full SSE parse and re-serialize on every chunk of every connection.Verification
Regression test
packages/runtime/src/__tests__/model-factory-tool-call-index.test.tsdrivesgetAIModelover a fake gateway, parameterised on index 0 / 1 / 7, plus two tool calls streamed across a hole at index 0. It asserts behaviour through the real provider stack and never references the patch, so it stays meaningful if upstream fixes this differently. Red before the patch, green after:The multi-call case exists because the single-call cases all pass under a fix that appends each new index instead of honouring it, which would silently reorder real multi-call turns.
Install modes, each from a clean tree:
npm cinpm ci --omit=devnpm ci --workspace @maka/runtimeThe two middle rows are why
postinstallcalls a script instead ofpatch-packagedirectly: both skip root devDependencies while still running the root postinstall, so a barepatch-package --error-on-failexited 127 wheremainexits 0. Those trees are never what ships, and an unpatched one fails the regression test.Also run:
node --test "packages/runtime/dist/__tests__/*.test.js"(2622/2635; the 4 failures arebuiltin-toolspath containment and reproduce identically onmainon this machine, a macOS/var→/private/varsymlink issue),npm run lint,npm run format:check, both clean.Packaged
--dirand extractedapp.asarto confirm the patch reaches the shipped app:Not run: full
npm run test, E2E, and a signed/notarized release build.Rollout
patches/and thepostinstallchange are new infrastructure. CI and the release workflow both run a plainnpm ci, so the patch applies everywhere the app is built. Contributors with a warmnode_modulesneed onenpm installto pick it up.patch-packageadds 294 lines to the lockfile; it is dev-only and does not reach the packaged app.