feat(llm): support claude-fable-5 (structured outputs, adaptive thinking, refusal fallback) - #2231
Conversation
🦋 Changeset detectedLatest commit: d32cdc1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
The provider's model-capability table now knows claude-fable-5 (and opus-4-7/4-8): structuredOutputMode "auto" resolves to the native output_format path instead of the forced json tool these models reject, sampling parameters are stripped where rejected, and max output tokens are sized correctly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
d19497b to
70aa809
Compare
…opic agents (#2232) ## What Stacked on #2231 (provider bump). Adds `lib/v3/llm/anthropicOptions.ts` as the shared capability table for Anthropic agent models, and uses it for four things: 1. **Adaptive thinking on the hybrid/DOM agent path.** Previously this path sent no Anthropic thinking config at all. Now adaptive-capable models (Claude 4.6+, Fable 5) get `thinking: {type: "adaptive"}` via typed provider options on both `streamText` call sites. Effort is configurable (`thinkingEffort` / `STAGEHAND_THINKING_EFFORT`), defaults to the API default (high), and the new `xhigh` level (added to `ThinkingEffort`) is clamped to `high` on models that reject it (e.g. sonnet-4-6). 2. **`claude-fable-5` registration** (AgentProvider, `AVAILABLE_CUA_MODELS`, public-api test) plus the API's **built-in server-side refusal fallback**: `fallbacks: [{model: "claude-opus-4-8"}]` as a typed provider option on agent and act/extract/observe calls. The provider adds the `server-side-fallback-2026-06-01` beta header automatically, handles the `fallback` content block, and reports attribution via `usage.iterations`. 3. **The agent's final `done` call no longer forces tool choice on models that reject it.** Forced `tool_choice` is incompatible with active extended thinking, and on Fable 5 thinking is always on — so the call consults `rejectsForcedToolUse()` and goes straight to `toolChoice: "auto"` there. A narrow catch remains only as a safety net for unknown models (retry once with auto; everything else rethrows). 4. **One adaptive-model list** shared between the CUA client (drops its inline copy) and the hybrid path. No fetch wrappers, no HTTP body rewriting, no reactive-by-default error sniffing — capabilities are declared once and consulted everywhere. ## Verification - typecheck + build green; 34/34 unit tests (12 new for the capability helpers). - Offline request-body capture (mock fetch, `claude-fable-5`, `generateText`): body carries `thinking: {"type":"adaptive"}`, `output_config: {"effort":"xhigh"}`, `fallbacks: [{"model":"claude-opus-4-8"}]`; headers carry `effort-2025-11-24,server-side-fallback-2026-06-01` — all emitted by the provider. ## Not in scope - CUA-mode (raw `@anthropic-ai/sdk`) fallback wiring: the pinned SDK (0.39.0) predates the `fallbacks` param; bumping it is a separate, larger change. CUA fable-5 runs work but a refusal surfaces as `stop_reason: "refusal"` without automatic retry. - Per the [refusals docs](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback), `fallback` content blocks must be preserved when echoing assistant turns — the AI SDK provider handles this; nothing here strips them. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2d7f1ec to
bfb5bfe
Compare
There was a problem hiding this comment.
1 issue found across 11 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.
Fix all with cubic | Re-trigger cubic
…s, effort clamping everywhere - stripModelProvider moves to lib/utils.ts as the canonical helper; AnthropicCUAClient and AgentProvider drop their inline copies - the forced-done try/catch net is removed: the capability table covers the only model that rejects forced tool use, so the call sites a single deterministic toolChoice - the done call now also carries Fable 5's server-side fallback options - provider-option helpers return a JSON-compatible alias (built via satisfies AnthropicProviderOptions) and aisdk.ts's ProviderOptionMap widens to JSONValue — both call-site casts deleted - CUA effort goes through resolveAdaptiveEffort (clamps xhigh on models that reject it, keeps the medium default); STAGEHAND_THINKING_EFFORT 'none' now means no thinking on the hybrid path too Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
e639289 to
12e828b
Compare
|
@miguelg719 I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 14 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Architecture diagram
sequenceDiagram
participant App as Application
participant V3A as V3AgentHandler
participant LLM as "AISdkClient / LLMClient"
participant V3H as "V3AgentHandler (execute/stream)"
participant HandleDone as handleDoneToolCall
participant CUA as AnthropicCUAClient
participant Opts as "anthropicOptions.ts"
participant SDK as "@ai-sdk/anthropic"
participant API as "Anthropic API"
Note over App,API: Claude Fable-5 Request Path — Hybrid/DOM Agent
App->>V3A: init agent with model "claude-fable-5"
V3A->>V3A: AgentProvider maps model -> "anthropic"
Note over V3A,Opts: Agent execute/stream loops (per-turn)
loop execute loop or stream loop
V3A->>Opts: buildAgentProviderOptions()
Opts->>Opts: anthropicAdaptiveThinkingOptions()
opt Fable 5 (adaptive-capable model)
Opts->>Opts: return {thinking:{type:"adaptive"}}
alt effort configured (explicit or STAGEHAND_THINKING_EFFORT)
Opts->>Opts: resolveAdaptiveEffort()
alt effort == "xhigh"
opt model rejects xhigh (sonnet-4-6)
Opts->>Opts: clamp to "high"
end
Opts-->>Opts: return "xhigh" or clamped value
end
end
end
Opts->>Opts: anthropicFallbacksOptions()
opt Fable 5 (fallback model set)
Opts->>Opts: return {fallbacks:[{model:"claude-opus-4-8"}]}
end
Opts-->>V3A: merged providerOptions
V3A->>LLM: streamText({ providerOptions, ... })
LLM->>SDK: pass anthropic provider options
SDK->>SDK: strip temperature (rejected by Fable 5)
SDK->>API: POST (thinking+effort+fallbacks+betas)
Note over SDK,API: beta headers: effort-2025-11-24, server-side-fallback-2026-06-01
API-->>SDK: response (maybe refusal -> fallback to opus-4-8)
SDK->>SDK: include usage.iterations for fallback attribution
SDK-->>LLM: result
LLM-->>V3A: response.messages
end
Note over App,HandleDone: Final "done" Tool Call (end of agent loop)
V3A->>HandleDone: handleDoneToolCall(model, messages)
HandleDone->>Opts: rejectsForcedToolUse(modelId)
alt Fable 5 (always-on thinking)
Opts-->>HandleDone: true
HandleDone->>HandleDone: set toolChoice: "auto" (instead of forced "done" tool)
else Other models
Opts-->>HandleDone: false
HandleDone->>HandleDone: set toolChoice: {type:"tool", toolName:"done"}
end
HandleDone->>Opts: anthropicFallbacksOptions(modelId)
opt Fable 5
Opts-->>HandleDone: fallbacks config
HandleDone->>HandleDone: merge into providerOptions.anthropic
end
HandleDone->>SDK: generateText({ toolChoice, providerOptions, ... })
SDK->>API: POST (tools+toolChoice or auto+fallbacks)
alt No tool call (plain text answer)
API-->>SDK: response without tool use
SDK-->>HandleDone: result
HandleDone-->>V3A: return plain text answer
else Tool call with "done"
API-->>SDK: response with done tool
SDK-->>HandleDone: result
HandleDone-->>V3A: return done result
end
Note over App,API: Structured Output Path (act/extract/observe)
App->>LLM: act/extract/observe call
LLM->>LLM: inferProviderName("claude-fable-5") -> "anthropic"
LLM->>LLM: set structuredOutputMode: "auto"
LLM->>Opts: anthropicFallbacksOptions(modelId)
opt Fable 5
Opts-->>LLM: fallbacks config
LLM->>LLM: merge into providerOptions.anthropic
end
LLM->>SDK: generateObject({ providerOptions, ... })
SDK->>SDK: structuredOutputMode: "auto" => output_config.format (native JSON schema)
Note over SDK: no tools/tool_choice sent — native structured outputs
SDK->>API: POST (output_config.format + fallbacks)
API-->>SDK: JSON response
SDK-->>LLM: result
LLM-->>App: structured data
Note over App,CUA: CUA (Computer Use Agent) Path
App->>CUA: init agent with model "claude-fable-5"
CUA->>CUA: stripModelProvider() -> "claude-fable-5"
CUA->>Opts: isAdaptiveThinkingAnthropicModel("claude-fable-5")
Opts-->>CUA: true
CUA->>CUA: set thinking = {type: "adaptive"}
CUA->>Opts: resolveAdaptiveEffort("claude-fable-5", thinkingEffort)
Opts-->>CUA: effort value (maybe xhigh or clamped)
CUA->>CUA: set outputConfig = {effort: ...}
loop CUA tool use cycle
CUA->>API: POST (thinking adaptive + output_config effort + computer_20251124 tools)
API-->>CUA: response (tool_use items)
CUA->>CUA: execute tool, capture screenshots
end
Note over App,API: Refusal Handling (any Fable 5 call)
opt Refusal (stop_reason: refusal)
API->>SDK: response with fallback attribution in usage.iterations
Note over SDK,API: API retried on claude-opus-4-8 automatically
SDK-->>LLM: result (with usage.iterations > 1)
end
…on both paths
DEFAULT_ANTHROPIC_ADAPTIVE_EFFORT ("medium") now backs both the CUA and
hybrid/DOM agent paths. The agent model config's thinkingEffort is
plumbed through V3AgentHandler so the typed client option works on the
hybrid path too, and the STAGEHAND_THINKING_EFFORT env knob is removed
from core (eval-side tooling can reintroduce it where it belongs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shrey150
left a comment
There was a problem hiding this comment.
Pre-approving, left some comments if you choose to address - non-blocking
Other models are referenced by their literal ids throughout the codebase; the named constant added indirection without value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @browserbasehq/stagehand@3.6.0 ### Minor Changes - [#2178](#2178) [`c49a3fc`](c49a3fc) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - add support for WebMCP ### Patch Changes - [#2217](#2217) [`147e310`](147e310) Thanks [@monadoid](https://github.com/monadoid)! - Add Azure OpenAI Microsoft Entra ID model auth support. - [#2231](#2231) [`cf3603d`](cf3603d) Thanks [@miguelg719](https://github.com/miguelg719)! - Add claude-fable-5 support: native structured outputs via the @ai-sdk/anthropic bump, adaptive thinking (including the new "xhigh" effort) on the agent path, the API's built-in server-side refusal fallback to claude-opus-4-8, and auto tool choice for the final done call on models that reject forced tool use. - [#2233](#2233) [`8d7d414`](8d7d414) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - Normalize URLs in `ActCache` key derivation by sorting query parameters before hashing. Semantically equivalent URLs that differ only in parameter order (e.g. `?utm_source=email&id=42` vs `?id=42&utm_source=email`) now hit the cache instead of silently missing. Fragments and duplicate keys are preserved. - [#2229](#2229) [`fd42e65`](fd42e65) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - launch local browser with --enable-features=WebMCPTesting,DevToolsWebMCPSupport by default - [#2220](#2220) [`a64c6b7`](a64c6b7) Thanks [@monadoid](https://github.com/monadoid)! - Fix Stagehand-generated shadow-root XPath resolution so deterministic actions can target elements inside web components. - [#2132](#2132) [`ed3e566`](ed3e566) Thanks [@miguelg719](https://github.com/miguelg719)! - Add canonical verifier evidence normalization for screenshots and text signals without requiring image dependencies in core installs. - [#2133](#2133) [`840aac8`](840aac8) Thanks [@miguelg719](https://github.com/miguelg719)! - Add the rubric-based verifier engine with normalized public rubric output and bounded failure-step parsing. ## @browserbasehq/stagehand-evals@2.0.3 ### Patch Changes - Updated dependencies \[[`147e310`](147e310), [`cf3603d`](cf3603d), [`8d7d414`](8d7d414), [`fd42e65`](fd42e65), [`a64c6b7`](a64c6b7), [`c49a3fc`](c49a3fc), [`ed3e566`](ed3e566), [`840aac8`](840aac8)]: - @browserbasehq/stagehand@3.6.0 ## @browserbasehq/stagehand-server-v3@3.7.1 ### Patch Changes - [#2217](#2217) [`147e310`](147e310) Thanks [@monadoid](https://github.com/monadoid)! - Add Azure OpenAI Microsoft Entra ID model auth support. - Updated dependencies \[[`147e310`](147e310), [`cf3603d`](cf3603d), [`8d7d414`](8d7d414), [`fd42e65`](fd42e65), [`a64c6b7`](a64c6b7), [`c49a3fc`](c49a3fc), [`ed3e566`](ed3e566), [`840aac8`](840aac8)]: - @browserbasehq/stagehand@3.6.0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…ing, refusal fallback) (browserbase#2231) ## Why Claude Fable 5 breaks Stagehand today in three independent ways: it rejects forced tool use (`400 tool_choice forces tool use is not compatible with this model`), which kills both `act`/`extract`/`observe` (forced `json` tool) and the agent's final `done` call; it rejects sampling parameters like `temperature`; and its safety classifiers can decline a turn (`stop_reason: "refusal"`) with no retry story. Separately, the hybrid/DOM agent path has never sent any Anthropic thinking config, so adaptive-thinking models ran with thinking off. ## What changed - **`@ai-sdk/anthropic` 2.0.57 → 2.0.81** (v5-compatible line, within `^2.0.34`; floor raised). The provider's capability table now knows fable-5/opus-4-7/4-8: the existing `structuredOutputMode: "auto"` resolves to native `output_config.format` instead of the forced json tool, rejected sampling params are stripped, and `max_tokens` is sized from the real 128k limit — zero code changes for the primitives. - **New `lib/v3/llm/anthropicOptions.ts`** — one capability table for Anthropic agent models (adaptive-thinking set, xhigh-capable set, fable-5 constants, `rejectsForcedToolUse`). - **Adaptive thinking on the hybrid/DOM agent path** via typed provider options on both `streamText` call sites. Effort defaults to a shared **`medium`** on both agent paths and is overridable per client via `thinkingEffort` in the agent model config (plumbed through `V3AgentHandler` so the typed option now works on the hybrid path too); `ThinkingEffort` gains `xhigh` (clamped to `high` on models that reject it), and `"none"` opts out of thinking on both paths. The CUA client drops its inline model list and effort default for the shared ones. - **Server-side refusal fallback**: fable-5 requests carry `fallbacks: [{model: "claude-opus-4-8"}]` as a typed provider option on agent and act/extract/observe calls; the provider adds the `server-side-fallback-2026-06-01` beta header automatically and reports attribution via `usage.iterations`. - **Final `done` call**: consults `rejectsForcedToolUse()` and goes straight to `toolChoice: "auto"` on fable-5 (forced tool choice is incompatible with always-on thinking); a narrow `/tool_choice/i` catch remains as a safety net for unknown models. - **Registration**: `claude-fable-5` in `AgentProvider`, `AVAILABLE_CUA_MODELS`, and the public-api test. Not in scope: CUA-mode (raw `@anthropic-ai/sdk` 0.39.0) fallback wiring — the pinned SDK predates the `fallbacks` param; a refusal there surfaces as `stop_reason: "refusal"` without retry. ## Tests - `pnpm typecheck` and `build:esm` green; 34/34 unit tests (12 new covering the capability helpers: adaptive set, xhigh clamping, effort precedence, fallbacks gating, forced-tool gate). - Offline request-body capture (mock fetch, 2.0.81): `generateObject` on fable-5 sends `output_config.format = {type: "json_schema", …, additionalProperties: false}` + `structured-outputs-2025-11-13` beta with **no** `tools`/`tool_choice`; `generateText` with the agent options sends `thinking: {"type":"adaptive"}`, `output_config: {"effort":"xhigh"}`, `fallbacks: [{"model":"claude-opus-4-8"}]` and the `effort-2025-11-24,server-side-fallback-2026-06-01` betas. - Not yet run against the live API from this machine — worth one live pass (extract + DOM/hybrid agent on fable-5) before un-drafting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) ---------
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @browserbasehq/stagehand@3.6.0 ### Minor Changes - [browserbase#2178](browserbase#2178) [`c49a3fc`](browserbase@c49a3fc) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - add support for WebMCP ### Patch Changes - [browserbase#2217](browserbase#2217) [`147e310`](browserbase@147e310) Thanks [@monadoid](https://github.com/monadoid)! - Add Azure OpenAI Microsoft Entra ID model auth support. - [browserbase#2231](browserbase#2231) [`cf3603d`](browserbase@cf3603d) Thanks [@miguelg719](https://github.com/miguelg719)! - Add claude-fable-5 support: native structured outputs via the @ai-sdk/anthropic bump, adaptive thinking (including the new "xhigh" effort) on the agent path, the API's built-in server-side refusal fallback to claude-opus-4-8, and auto tool choice for the final done call on models that reject forced tool use. - [browserbase#2233](browserbase#2233) [`8d7d414`](browserbase@8d7d414) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - Normalize URLs in `ActCache` key derivation by sorting query parameters before hashing. Semantically equivalent URLs that differ only in parameter order (e.g. `?utm_source=email&id=42` vs `?id=42&utm_source=email`) now hit the cache instead of silently missing. Fragments and duplicate keys are preserved. - [browserbase#2229](browserbase#2229) [`fd42e65`](browserbase@fd42e65) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - launch local browser with --enable-features=WebMCPTesting,DevToolsWebMCPSupport by default - [browserbase#2220](browserbase#2220) [`a64c6b7`](browserbase@a64c6b7) Thanks [@monadoid](https://github.com/monadoid)! - Fix Stagehand-generated shadow-root XPath resolution so deterministic actions can target elements inside web components. - [browserbase#2132](browserbase#2132) [`ed3e566`](browserbase@ed3e566) Thanks [@miguelg719](https://github.com/miguelg719)! - Add canonical verifier evidence normalization for screenshots and text signals without requiring image dependencies in core installs. - [browserbase#2133](browserbase#2133) [`840aac8`](browserbase@840aac8) Thanks [@miguelg719](https://github.com/miguelg719)! - Add the rubric-based verifier engine with normalized public rubric output and bounded failure-step parsing. ## @browserbasehq/stagehand-evals@2.0.3 ### Patch Changes - Updated dependencies \[[`147e310`](browserbase@147e310), [`cf3603d`](browserbase@cf3603d), [`8d7d414`](browserbase@8d7d414), [`fd42e65`](browserbase@fd42e65), [`a64c6b7`](browserbase@a64c6b7), [`c49a3fc`](browserbase@c49a3fc), [`ed3e566`](browserbase@ed3e566), [`840aac8`](browserbase@840aac8)]: - @browserbasehq/stagehand@3.6.0 ## @browserbasehq/stagehand-server-v3@3.7.1 ### Patch Changes - [browserbase#2217](browserbase#2217) [`147e310`](browserbase@147e310) Thanks [@monadoid](https://github.com/monadoid)! - Add Azure OpenAI Microsoft Entra ID model auth support. - Updated dependencies \[[`147e310`](browserbase@147e310), [`cf3603d`](browserbase@cf3603d), [`8d7d414`](browserbase@8d7d414), [`fd42e65`](browserbase@fd42e65), [`a64c6b7`](browserbase@a64c6b7), [`c49a3fc`](browserbase@c49a3fc), [`ed3e566`](browserbase@ed3e566), [`840aac8`](browserbase@840aac8)]: - @browserbasehq/stagehand@3.6.0
Why
Claude Fable 5 breaks Stagehand today in three independent ways: it rejects forced tool use (
400 tool_choice forces tool use is not compatible with this model), which kills bothact/extract/observe(forcedjsontool) and the agent's finaldonecall; it rejects sampling parameters liketemperature; and its safety classifiers can decline a turn (stop_reason: "refusal") with no retry story. Separately, the hybrid/DOM agent path has never sent any Anthropic thinking config, so adaptive-thinking models ran with thinking off.What changed
@ai-sdk/anthropic2.0.57 → 2.0.81 (v5-compatible line, within^2.0.34; floor raised). The provider's capability table now knows fable-5/opus-4-7/4-8: the existingstructuredOutputMode: "auto"resolves to nativeoutput_config.formatinstead of the forced json tool, rejected sampling params are stripped, andmax_tokensis sized from the real 128k limit — zero code changes for the primitives.lib/v3/llm/anthropicOptions.ts— one capability table for Anthropic agent models (adaptive-thinking set, xhigh-capable set, fable-5 constants,rejectsForcedToolUse).streamTextcall sites. Effort defaults to a sharedmediumon both agent paths and is overridable per client viathinkingEffortin the agent model config (plumbed throughV3AgentHandlerso the typed option now works on the hybrid path too);ThinkingEffortgainsxhigh(clamped tohighon models that reject it), and"none"opts out of thinking on both paths. The CUA client drops its inline model list and effort default for the shared ones.fallbacks: [{model: "claude-opus-4-8"}]as a typed provider option on agent and act/extract/observe calls; the provider adds theserver-side-fallback-2026-06-01beta header automatically and reports attribution viausage.iterations.donecall: consultsrejectsForcedToolUse()and goes straight totoolChoice: "auto"on fable-5 (forced tool choice is incompatible with always-on thinking); a narrow/tool_choice/icatch remains as a safety net for unknown models.claude-fable-5inAgentProvider,AVAILABLE_CUA_MODELS, and the public-api test.Not in scope: CUA-mode (raw
@anthropic-ai/sdk0.39.0) fallback wiring — the pinned SDK predates thefallbacksparam; a refusal there surfaces asstop_reason: "refusal"without retry.Tests
pnpm typecheckandbuild:esmgreen; 34/34 unit tests (12 new covering the capability helpers: adaptive set, xhigh clamping, effort precedence, fallbacks gating, forced-tool gate).generateObjecton fable-5 sendsoutput_config.format = {type: "json_schema", …, additionalProperties: false}+structured-outputs-2025-11-13beta with notools/tool_choice;generateTextwith the agent options sendsthinking: {"type":"adaptive"},output_config: {"effort":"xhigh"},fallbacks: [{"model":"claude-opus-4-8"}]and theeffort-2025-11-24,server-side-fallback-2026-06-01betas.🤖 Generated with Claude Code