Skip to content

refactor(runtime): make a single provider step the ModelAdapter primitive #1381

Description

@Astro-Han

Summary

Make each ModelAdapter stream call represent exactly one provider step. Maka, not the AI SDK, should own the agent loop, tool execution, durable settlement, steering, compaction, and continuation.

This is a boundary correction, not a provider rewrite. Maka should continue using ai and @ai-sdk/* for request lowering, HTTP transport, SSE parsing, and provider-specific response normalization.

Current ownership

ModelAdapter.startStream() currently calls AI SDK streamText() with:

  • executable tools;
  • stopWhen for a bounded or unbounded multi-step loop;
  • prepareStep for steering, compaction, active-tool changes, and message shaping.

The SDK therefore owns the loop between provider requests and invokes Maka's wrapped tool executors from inside that loop. Maka has had to adapt its durable runtime semantics around SDK-local step and attempt state. Examples include:

  • attemptStepBase rebasing after a retry;
  • remaining-step-budget forwarding across startStream() retries;
  • steering injection through prepareStep;
  • guarding against transport retry after a tool may have changed external state;
  • accepting multiple SDK chunk names across migration windows.

The tool implementation itself already belongs to ToolRuntime. The current extra ownership in streamText() does not provide a capability that Maka lacks. It hides provider-step boundaries that Maka needs for persistence and recovery.

Required boundary

ModelAdapter should expose one Maka-owned operation with this meaning:

one request projection in
one provider step stream out
no local tool execution
no automatic continuation

An illustrative shape:

interface ModelStepRequest {
  model: unknown;
  system?: string;
  messages: ModelMessage[];
  tools: ModelToolDefinition[];
  activeTools: string[];
  abortSignal: AbortSignal;
}

interface ModelStepResult {
  stream: AsyncIterable<ModelStepEvent>;
  completion: Promise<ModelStepCompletion>;
}

The exact exported names can follow the existing code, but the ownership must be visible in the type. Step budgets, prepareStep, and stopAfterStep do not belong on the adapter call.

AI SDK types must stay behind ModelAdapter. Runtime code consumes Maka-owned messages, stream events, usage, finish reasons, request metadata, and typed failures.

Execution flow

Runtime projects durable history and current input
        |
        v
apply steering / context shaping / active-tool policy
        |
        v
ModelAdapter sends one provider request
        |
        v
Runtime persists streamed assistant facts
        |
        v
Runtime settles returned tool calls through ToolRuntime
        |
        v
reload durable projected history
        |
        +--> continue with another ModelAdapter call when required
        |
        +--> stop when the model produced a terminal assistant response

Tools passed to the SDK are schemas only. They do not carry SDK-owned execute callbacks. Every returned tool call crosses the existing ToolRuntime permission, admission, journal, artifact, and output boundaries before continuation.

The context-shaping code currently driven by prepareStep becomes ordinary runtime logic between adapter calls. This change should preserve its existing behavior before any policy is redesigned.

Retry and interruption semantics

  • The adapter must not hide a multi-step retry loop.
  • A provider retry is a new explicit adapter call owned by the Runtime. Main-agent adapter calls set maxRetries: 0 so the AI SDK never hides an additional attempt.
  • No retry may repeat a provider step after a tool call could have produced an external side effect.
  • Abort stops the current provider request. It does not discard already durable assistant or tool facts.
  • A provider response that contains tool calls ends the adapter call. Tool settlement and the decision to continue happen outside the adapter.
  • A durably recorded tool call without a matching tool result is indeterminate after interruption. Recovery blocks and never automatically re-executes it.
  • Auxiliary calls such as compaction may continue using their own narrow one-request methods. They do not share agent-loop state.

ModelAdapter does not distinguish a hidden “pre-output” retry: once dispatch is attempted, retry ownership stays with the Runtime because absence of visible output does not prove that the provider neither accepted nor billed the request.

Migration plan

  1. Define the one-step ModelAdapter contract and characterize current streaming output with tests.
  2. Run streamText() in one-step, zero-retry mode and remove SDK-owned tool execution.
  3. Move the loop into the existing runtime/backend path without introducing a second loop implementation.
  4. Move each prepareStep responsibility to an explicit between-step stage.
  5. Preserve provider request capture, signed thinking, usage normalization, tool-call repair, abort behavior, and context-overflow handling.
  6. Remove the old loop controls and the compatibility code that no longer has an owner.

Each slice must leave one production path. Do not keep the SDK-owned loop as a fallback.

Delivery slices

Land the change as three sequential flat PRs. Each PR starts from the latest main after the previous PR merges; do not stack them.

1. Contain the AI SDK protocol in ModelAdapter

Scope

  • Define Maka-owned model messages, tool definitions, stream events, completion metadata, usage, finish reasons, request metadata, and typed failures.
  • Keep raw AI SDK chunks, provider metadata, retry wrappers, and provider-specific normalization inside ModelAdapter.
  • Move runtime, history projection, and compaction consumers away from direct ai type imports.
  • Characterize text, signed thinking, tool calls, usage, finish reasons, errors, request capture, and provider-matrix behavior before changing loop ownership.

Exit condition

  • No AI SDK type crosses the adapter boundary.
  • The existing SDK-owned multi-step loop and executable-tool behavior remain unchanged for this slice, with exactly one production path.
  • This slice does not move tool execution or continuation ownership.

2. Expose direct ToolRuntime settlement

Scope

  • Replace the SDK-shaped wrapToolExecute() entry with one direct ToolRuntime operation that accepts a resolved Maka tool call and performs admission, permission, durable preparation, execution, outcome commit, artifact recording, telemetry, and model-output materialization.
  • Make the current AI SDK execute callback a thin caller of that operation; it must not own policy or state.
  • Preserve synchronous per-step admission, current multi-tool concurrency, permission waits, durable identities, recovery modes, plan handoff behavior, and tool-result error semantics.
  • Move ToolRuntime tests to the direct entry and retain a contract test that prevents the backend from reimplementing settlement.

Exit condition

  • Tool execution and settlement have one implementation owner in ToolRuntime.
  • The SDK still triggers the call in this slice, but the SDK-shaped wrapper no longer owns or duplicates behavior.
  • Agent-loop ownership remains unchanged.

3. Move the agent loop above ModelAdapter

Scope

  • Switch ModelAdapter to one provider step per call, pass schema-only tools, and set maxRetries: 0 for main-agent calls.
  • Implement the only agent loop in the existing AiSdkBackend.send() path: load the durable projection, project the request, call the adapter, persist assistant and tool-call facts, settle all returned tool calls through ToolRuntime, reload durable history, and decide whether to continue.
  • Preserve current multi-tool concurrency and wait for every tool outcome to become durable before the next provider request.
  • Give both parent and child runs a durable current-run reader. Represent mid-turn compaction eligibility as a separate capability instead of inferring it from reader availability.
  • Move steering, active-tool selection, pruning, compaction, final capacity checks, provider request capture, step limits, explicit network retry, overflow recovery, abort, and after-step stop to explicit Runtime stages.
  • Remove prepareStep, executable SDK tools, SDK loop stopWhen, attemptStepBase, remaining-budget forwarding, stopAfterStep, and compatibility code that no longer has an owner.
  • Validate the two replay-equivalence fixtures plus focused permission, steering, compaction, retry, abort, and interrupted-execution tests.

Exit condition

  • All acceptance criteria below pass on the single Runtime-owned production path.
  • Reverting this PR restores the pre-switch loop while leaving the independently useful adapter and ToolRuntime seams intact.

Acceptance criteria

  1. One ModelAdapter call completes after one provider step, including a response that requests tools.
  2. The SDK never invokes a Maka tool implementation.
  3. Tool calls execute once through ToolRuntime, retain their durable identities, and settle before continuation.
  4. Projected history is reloaded before every continuation request.
  5. Steering, deferred-tool activation, active pruning, compaction, permission, abort, context overflow, and transport failure retain their current externally visible behavior.
  6. Provider request capture and normalized usage remain complete for each step.
  7. No AI SDK type escapes the adapter boundary.
  8. Tests cover text-only completion, one and multiple tool calls, tool failure, permission wait, steering, compaction, retry, abort, and interrupted execution.
  9. A dangling durable tool call remains indeterminate on recovery and is never automatically replayed.

Explicitly not included

  • replacing AI SDK provider integrations;
  • implementing Anthropic or OpenAI wire protocols;
  • provider-native Tool Search;
  • Programmatic Tool Calling or a local code mode;
  • redesigning compaction, permissions, steering, or retry policy;
  • changes to Runtime Host ownership tracked by [Tracking] Runtime Host M3 domain ownership migration #1167.

Follow-up questions

These do not block #1381 and are not part of delivery slice 1. Audit them one at a time after the main-agent model protocol is contained; do not turn the audit into a blanket removal of the ai dependency.

  1. Should the history-compaction summarizer accept a Maka-owned request shape while keeping its narrow generateText() integration local?
  2. Should session-title generation expose a Maka-owned model-call contract instead of an AI SDK LanguageModel type?
  3. Should the auto-approval reviewer isolate its narrow model call behind an explicit adapter-local contract?
  4. Are jsonSchema / zodSchema imports in builtin and MCP tool construction only local schema implementation details, or does any AI SDK type cross their ownership boundary?

Only open follow-up work where a concrete ownership or type leak exists. Schema helpers that remain local implementation details need no refactor.

Resolved decisions

  1. All main-agent provider-call retries are explicit Runtime attempts. The adapter sets maxRetries: 0; auxiliary calls keep narrow independent retry policies.
  2. Steering, active-tool selection, pruning, and pre-request compaction run during explicit request projection. Decisions that depend on tool outcomes run only after durable tool settlement. The final capacity verdict and provider request capture sit immediately around the adapter call.
  3. Replay equivalence uses two minimal fixtures: a text-only terminal step with signed thinking and usage; and a multi-tool step covering success and failure, durable settlement, history reload, and terminal continuation. Permission wait, steering, compaction, retry, abort, and interruption remain focused tests rather than one monolithic fixture.
  4. Dangling tool calls block recovery as indeterminate and are never automatically replayed.
中文版本

概述

ModelAdapter 的基本单位收窄成一次 provider step。Agent loop、工具执行、持久化结算、steering、compaction 和 continuation 都由 Maka 自己负责,不再交给 AI SDK 的 streamText() 内部循环。

这不是重写 provider。ai@ai-sdk/* 继续负责请求转换、HTTP、SSE 解析和各家响应格式适配。

现有职责为什么不对

当前 ModelAdapter.startStream() 会把下面这些一起交给 streamText()

  • execute 的工具;
  • 控制多 step 的 stopWhen
  • 承担 steering、compaction、工具集变化和消息改写的 prepareStep

因此,两次 provider 请求之间何时执行工具、何时继续,实际由 SDK 控制。Maka 为了把 durable runtime 套在外面,已经出现不少补丁式状态:

  • retry 后重算 attemptStepBase
  • retry 时手动传递剩余 step budget;
  • 通过 prepareStep 注入 steering;
  • 工具可能已经产生副作用后,阻止 transport retry;
  • 为 SDK 迁移期的不同 chunk 名保留兼容分支。

工具执行本身早已归 ToolRuntime。SDK 内部 loop 没有补上 Maka 缺失的能力,反而把 Maka 需要持久化和恢复的 provider-step 边界藏起来了。

目标边界

ModelAdapter 只保留一个清楚的含义:

输入一次请求投影
输出一次 provider step 的事件流
不执行本地工具
不自动继续下一步

示意接口:

interface ModelStepRequest {
  model: unknown;
  system?: string;
  messages: ModelMessage[];
  tools: ModelToolDefinition[];
  activeTools: string[];
  abortSignal: AbortSignal;
}

interface ModelStepResult {
  stream: AsyncIterable<ModelStepEvent>;
  completion: Promise<ModelStepCompletion>;
}

最终命名可以就近改现有接口,但 ownership 必须体现在类型里。step budget、prepareStepstopAfterStep 不再属于 adapter 参数。

AI SDK 类型不能穿过 ModelAdapter。Runtime 只看到 Maka 自己定义的 message、stream event、usage、finish reason、request metadata 和 typed failure。

执行流程

Runtime 从 durable history 和当前输入生成请求
        |
        v
应用 steering / context shaping / active-tool policy
        |
        v
ModelAdapter 发送一次 provider 请求
        |
        v
Runtime 持久化 assistant 事实
        |
        v
Runtime 通过 ToolRuntime 结算 tool calls
        |
        v
重新读取 durable projected history
        |
        +--> 需要继续时再调用一次 ModelAdapter
        |
        +--> 模型给出终态回复时结束

传给 SDK 的工具只有 schema,不再带由 SDK 调用的 execute。模型返回的每个 tool call 必须经过现有 ToolRuntime 的 permission、admission、journal、artifact 和 output 边界,结算完成后才能继续。

今天挂在 prepareStep 上的 context shaping 会变成两次 adapter 调用之间的普通 runtime 逻辑。第一阶段只搬 ownership,不顺手重写策略。

Retry 与中断

  • adapter 不能藏一段多 step retry loop;
  • provider retry 是 Runtime 发起的一次新 adapter 调用。主 agent 的 adapter 调用固定 maxRetries: 0,AI SDK 不得隐藏额外 attempt;
  • 工具可能已产生外部副作用后,禁止重放同一个 provider step;
  • abort 只停止当前 provider 请求,不能抹掉已经 durable 的 assistant 或 tool 事实;
  • provider 返回 tool calls 后,本次 adapter 调用立即结束;工具结算和是否继续由外层决定;
  • 已持久化但没有匹配 tool result 的 tool call 在中断后属于不确定状态;恢复必须阻塞,且不得自动重新执行;
  • compaction 等辅助模型调用继续使用自己的单请求方法,不共享 agent-loop 状态。

ModelAdapter 不区分隐藏的“尚无可见输出”重试:一旦尝试 dispatch,retry ownership 就留在 Runtime,因为没有可见输出不能证明 provider 没有接收请求或计费。

迁移步骤

  1. 定义单-step ModelAdapter 契约,用测试锁住当前 streaming 输出。
  2. streamText() 只执行一步、关闭内部重试,并移除 SDK-owned tool execution。
  3. 在现有 runtime/backend 路径内接管 loop,不建立第二套并行 loop。
  4. prepareStep 的每项职责逐个移到显式的 between-step 阶段。
  5. 保持 provider request capture、signed thinking、usage normalization、tool-call repair、abort 和 context-overflow 行为。
  6. 删除失去 owner 的旧 loop 控制与兼容代码。

每个 slice 都必须只留下一个生产路径,不保留 SDK loop fallback。

交付切片

按三个依次合入的 flat PR 交付。前一个 PR 合并后,下一个 PR 从最新 main 开始,不做 stacked PR。

1. 把 AI SDK 协议收进 ModelAdapter

范围

  • 定义 Maka-owned model message、tool definition、stream event、completion metadata、usage、finish reason、request metadata 和 typed failure。
  • AI SDK raw chunk、provider metadata、retry wrapper 与 provider-specific normalization 全部留在 ModelAdapter 内。
  • runtime、history projection 和 compaction 不再直接导入 ai 类型。
  • 在改变 loop ownership 前,用 characterization tests 锁住 text、signed thinking、tool call、usage、finish reason、error、request capture 和 provider matrix 行为。

完成出口

  • AI SDK 类型不越过 adapter 边界。
  • 本切片保持现有 SDK-owned multi-step loop 与 executable-tool 行为不变,生产路径仍然只有一条。
  • 本切片不移动工具执行或 continuation ownership。

2. 暴露直接的 ToolRuntime settlement

范围

  • 用一个直接接收已解析 Maka tool call 的 ToolRuntime 操作替代 SDK-shaped wrapToolExecute();admission、permission、durable preparation、execution、outcome commit、artifact、telemetry 和 model-output materialization 都由它完成。
  • 当前 AI SDK execute callback 只薄调用这个操作,不拥有 policy 或 state。
  • 保持同步的 per-step admission、现有多工具并发、permission wait、durable identity、recovery mode、plan handoff 和 tool-result error 语义。
  • ToolRuntime 测试迁到直接入口,并保留 contract test,防止 backend 重新实现 settlement。

完成出口

  • 工具执行与结算只有 ToolRuntime 一个实现 owner。
  • 本切片中仍由 SDK 触发调用,但 SDK-shaped wrapper 不再拥有或复制行为。
  • Agent-loop ownership 保持不变。

3. 把 agent loop 移到 ModelAdapter 之上

范围

  • ModelAdapter 每次调用只完成一个 provider step,工具只传 schema,主 agent 调用固定 maxRetries: 0
  • 在现有 AiSdkBackend.send() 路径实现唯一 agent loop:读取 durable projection、投影请求、调用 adapter、持久化 assistant 与 tool-call facts、通过 ToolRuntime 结算全部工具、重新读取 durable history,再决定 continuation。
  • 保持现有多工具并发,并在下一次 provider request 前等待所有 tool outcome durable。
  • parent 与 child run 都获得 durable current-run reader;mid-turn compaction eligibility 改为独立 capability,不再由 reader 是否存在来隐式判断。
  • steering、active-tool selection、pruning、compaction、final capacity check、provider request capture、step limit、显式 network retry、overflow recovery、abort 和 after-step stop 都迁到显式 Runtime stage。
  • 删除 prepareStep、executable SDK tools、SDK loop stopWhenattemptStepBase、remaining-budget forwarding、stopAfterStep 和失去 owner 的兼容代码。
  • 用两条 replay-equivalence fixture 加 focused permission、steering、compaction、retry、abort 和 interrupted-execution tests 验证。

完成出口

  • 下方所有验收条件都在唯一的 Runtime-owned 生产路径通过。
  • 单独 revert 本 PR 会恢复切换前的 loop,同时保留已经独立成立的 adapter 与 ToolRuntime seam。

验收条件

  1. 每次 ModelAdapter 调用只完成一个 provider step;模型请求工具时,本次调用也必须结束。
  2. SDK 不再直接调用 Maka 工具实现。
  3. tool call 只通过 ToolRuntime 执行一次,保留 durable identity,并在 continuation 前完成结算。
  4. 每次 continuation 请求前重新读取 projected history。
  5. steering、deferred-tool activation、active prune、compaction、permission、abort、context overflow、transport failure 的外部行为保持不变。
  6. 每个 step 的 provider request capture 和 normalized usage 保持完整。
  7. AI SDK 类型不越过 adapter 边界。
  8. 测试覆盖纯文本、单/多工具调用、工具失败、权限等待、steering、compaction、retry、abort 和 interrupted execution。
  9. durable tool call 缺少匹配结果时,恢复保持 indeterminate,绝不自动重放。

明确不做

  • 替换 AI SDK provider integration;
  • 自己实现 Anthropic 或 OpenAI wire;
  • provider-native Tool Search;
  • Programmatic Tool Calling 或本地 code mode;
  • 重写 compaction、permission、steering 或 retry policy;
  • 改动 [Tracking] Runtime Host M3 domain ownership migration #1167 已定义的 Runtime Host ownership。

Follow-up 问题

这些问题不阻塞 #1381,也不属于交付切片 1。主 agent model protocol 收口后再逐项审计,不把审计扩大成清除全部 ai 依赖。

  1. history-compaction summarizer 是否应接收 Maka-owned request shape,同时把窄 generateText() integration 保留在本地?
  2. session-title generation 是否应暴露 Maka-owned model-call contract,而不是 AI SDK LanguageModel 类型?
  3. auto-approval reviewer 是否应把它的窄 model call 隔离在明确的 adapter-local contract 后?
  4. builtin 与 MCP tool construction 中的 jsonSchema / zodSchema import 是否只是本地 schema 实现细节,还是有 AI SDK 类型越过了其 ownership boundary?

只有发现具体 ownership 或 type leak 时才开启 follow-up;保持局部实现细节的 schema helper 不需要重构。

已定决策

  1. 主 agent 的 provider-call retry 全部是 Runtime 显式 attempt。adapter 固定 maxRetries: 0;compaction 等辅助调用保留各自独立的窄 retry policy。
  2. steering、active-tool selection、pruning 和请求前 compaction 都在显式 request projection 阶段执行。依赖工具结果的决策只能发生在 durable tool settlement 之后。最终 capacity verdict 与 provider request capture 紧邻 adapter call。
  3. Replay 等价性只使用两条最小 fixture:包含 signed thinking 与 usage 的纯文本终态;以及覆盖成功和失败、durable settlement、history reload 与终态 continuation 的多工具 step。permission wait、steering、compaction、retry、abort 和 interruption 继续由 focused tests 覆盖,不合并成一个巨型 fixture。
  4. dangling tool call 在恢复时属于 indeterminate 并阻塞,绝不自动重放。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions