feat(core): add embedded v2 session runtime and tool foundation

[kitlangton]

Summary

This draft PR builds the Effect-native embedded OpenCode V2 foundation needed by local-first consumers such as OpenCord. It separates durable prompt admission from execution, routes Sessions through Location-scoped runtime graphs, refines the existing durable Session event model with replay-safe cursors and projection hardening, and adds the first Core-owned coding-agent tool catalog.

The ordinary V1 Session path remains the default. The new embedded runner and session.next.* contracts remain experimental and opt-in while the remaining prompt-loop parity slices are designed and reviewed.

What This Adds

Embedded Session Runtime

• Adds the Effect-native @opencode-ai/core/opencode facade.
• Adds idempotent Session creation from caller-supplied IDs.
• Adds idempotent durable prompt admission through a session_input inbox.
• Supports explicit steer and queue delivery modes.
• Routes execution by Session ID through SessionStore -> LocationServiceMap -> SessionRunner.
• Coalesces process-local wakes and joins concurrent same-Session resumes.
• Keeps advisory wakes inbox-driven: they run only when durable admitted input can be promoted.
• Defers automatic post-crash activity retry until durable activity identity, provider-dispatch ambiguity, and visible recovery semantics are designed together.
• Returns explicit unavailable errors for unfinished Session operations rather than silently succeeding.

Native Provider Loop

• Executes one explicit llm.stream(request) provider turn at a time.
• Records each local tool call durably before starting side effects.
• Starts local tool calls eagerly while continuing to consume the provider stream.
• Awaits every started local tool before reloading projected history and deciding whether to continue.
• Durably settles raw stream failures and unresolved hosted tools instead of leaving incomplete projections.
• Treats dismissed structured questions as interruption rather than ordinary model-facing tool errors.
• Preserves provider-native continuation metadata across durable replay, including separate call-side and result-side tool metadata and V1-style model-switch safety.
• Round-trips Anthropic, OpenAI Responses, OpenAI-compatible Chat, Bedrock Converse, and Gemini continuation metadata needed by supported protocols.

Provider timeout, retry, and watchdog policy is intentionally deferred. The runner does not impose a universal inactivity or absolute stream timeout. A later slice should add bounded pristine-attempt retries and configurable provider-specific watchdogs deliberately.

Durable Events And Replay

• Keeps the pre-existing synchronized session.next.* event family and projected Session-message model; this branch does not introduce them.
• Adds durable session_input inbox storage and delivery-aware pending indexes.
• Adds session_message.seq and covering indexes so projected history follows durable aggregate order rather than timestamps or caller IDs.
• Changes text, reasoning, and tool-input deltas from synchronized events into ephemeral live-only fragments while retaining durable full-value checkpoints.
• Adds an explicit durable-event union and an embedded replay-and-tail Session stream.
• Brands the embedded aggregate continuation position as EventV2.Cursor.
• Keeps SQLite as replay source of truth and uses coalesced process-local wake signals only as advisory edges.
• Reserves admitted prompt IDs against conflicting durable event publication before projection.
• Hardens replay-owner propagation, aggregate envelope validation, and strict transfer-owner conflicts.

Tool Foundation

Adds Location-scoped Core-owned tools:

• read
• write
• edit
• apply_patch
• glob
• grep
• bash
• question
• todowrite
• skill
• webfetch
• websearch

Important behavior:

• Mutation tools resolve canonical targets before approval, escalate external-directory approval when required, lock canonical targets, and revalidate immediately before commit.
• write preserves UTF-8 BOM semantics.
• apply_patch preserves sequential non-transactional behavior while reporting partial application honestly, using create-only add semantics, rejecting malformed grammar, and rejecting moves until their semantics are hardened.
• read pages oversized UTF-8 files with bounded line and byte limits rather than rejecting them.
• Search roots are revalidated immediately before ripgrep traversal.
• Managed oversized tool output uses Session-owned opaque tool-output:// resources with bounded paging and payload-size integrity checks.
• bash follows normal configured-rule and saved-approval permission semantics. It remains foreground-only until owner-bound observation, cancellation, durable status, and completion delivery exist.
• webfetch follows V1-like HTTP(S) hostname and redirect behavior while retaining requested-URL permission checks, timeout caps, response-size bounds, and text conversion.
• The V2 skill tool reuses the single Location-scoped SkillV2 registry, bounds selected-skill output, and hardens remote discovery against cache-path traversal and cross-origin resource fetches.

Shared Runtime Foundations

• Adds a reusable process-local Core BackgroundJob registry and keeps the legacy OpenCode adapter instance-scoped; V2 model-facing background execution remains deferred.
• Fixes immediate-settlement races by publishing job state before starting child fibers.
• Hardens PluginV2 lifecycle: plugin child scopes now close with their Location layer, and same-ID add/remove operations serialize.
• Hardens the SystemContext algebra with namespaced keys, own-property-safe checkpoints, and duplicate-key validation at the interpreter boundary.
• Adds an unused SessionSystemContext scaffold for Location environment and date facts. It is not wired into provider request assembly yet.

HTTP API And SDK

• Adds selected experimental V2 provider, model, filesystem, Session, permission, and question routes.
• Adds explicit public provider and model DTOs so HTTP and generated SDK clients do not receive internal settings, request overlays, custom secret-bearing data, or credential-bearing URL details.
• Exposes public API URLs as origin-only HTTP(S) values; malformed or non-HTTP(S) URLs are omitted.
• Preserves required V2 mutation request bodies while retaining legacy optional-body normalization.
• Regenerates OpenAPI and SDK types.

Legacy Compatibility

• Preserves V1 prompt dual-write behavior for configured references and file attachments.
• Preserves fractional V1 user-message timestamps accepted by pagination compatibility tests.
• Keeps V1 and durable V2 attachment settlement consistent when URL/file sources cannot yet be materialized.
• Fixes recorded native LLM cassette isolation so replay tests do not reach live providers.
• Fixes cross-platform mutation fixtures and Windows transient SQLite cleanup.
• Fixes the HttpApi exerciser lifecycle so cached handler scopes and isolated memo maps do not retain SQLite resources across reset.

Draft Blockers Before Broad Exposure

This remains intentionally draft. The foundation is useful for supervised embedded canaries, but it is not a drop-in replacement for the full V1 prompt loop yet.

Highest-priority follow-ups:

• Wire selected-agent request assembly: agent model/options/system prompt, provider prompt, environment/date context, AGENTS.md, nearby nested instructions, skills guidance, and plugin transforms.
• Materialize policy-filtered tool definitions per Session, agent, model, provider, and client rather than advertising the full static catalog.
• Add public Session cancel and status contracts, implement wait, and expose the remote lifecycle needed by HTTP consumers.
• Design bounded pristine-attempt provider retries and configurable provider-specific watchdogs.
• Implement compaction and context-pressure recovery.
• Implement prompt attachment preprocessing and consistent URL/file materialization.
• Project native V2 cost, token totals, and activity timestamps onto SessionInfo.
• Restore snapshots, patches/diffs, and revert semantics before presenting V2 mutations as a stable editing replacement.
• Add foreground task dispatch and child-session creation; keep background execution deferred until its durable contract exists.
• Add doom-loop protection and agent-specific step limits below a hard global ceiling.
• Finish SystemContext removal semantics and wire SessionSystemContext into provider request assembly.

The detailed ledger and deferred hardening list live in:

specs/v2/schema-changelog.md
specs/v2/todo.md

Schema Changes

• Adds durable Session inbox storage and indexes.
• Adds projected Session message ordering fields and covering indexes.
• Refines the pre-existing experimental session.next.* family rather than introducing it: streamed deltas become ephemeral and embedded replay cursors become explicit.
• Adds optional projected tool result metadata while preserving the existing call-side metadata slot.
• Adds model-facing schemas for the first Core tool catalog.
• Regenerates OpenAPI and SDK types for the current experimental V2 surface.

Pre-launch session.next.* databases remain disposable experimental state. Reset experimental V2 data when upgrading across incompatible event-schema iterations. This is not a rolling-upgrade compatibility claim for previously persisted experimental histories.

Verification

Passed locally after the latest stabilization commit:

Focused Core changed-surface regression suite:
  264 pass
  0 fail

Focused LLM protocol continuation regressions:
  104 pass
  0 fail

Legacy BackgroundJob compatibility suites:
  46 pass
  0 fail

Core / LLM / OpenCode / SDK package typechecks:
  passed

Workspace-wide pre-push Turbo typecheck:
  21 successful packages

git diff --check:
  passed

The complete Core CI suite remains independently red only in the existing macOS filesystem-watcher readiness tests:

847 pass
6 watcher readiness failures

Refreshed GitHub Actions are running after push.

[carltonawong]

One acceptance boundary I’d keep explicit for session.next.*: before a resumed/tailed Session is allowed to run the next model/tool step, expose a small resume receipt, not just a replay cursor.

For this PR’s shape that could be: Location/session id, admitted session_input id, current EventV2.Cursor, projection version or last durable event applied, whether a process-local wake was only advisory vs promoted from the inbox, pending local tool calls with approval state carried/expired, and whether provider continuation metadata was restored or intentionally reset.

That gives HTTP/SDK consumers a testable invariant: replay may rebuild history from SQLite, but execution only continues after the runtime can show which durable boundary it is trusting and what must be revalidated before side effects.

[Hona]

This migration breaks upgraded beta DBs with pre-durable session projections. On an affected DB: 2,040 session_message rows total, 1,993 have no matching event.id across 237 sessions, 47 match across 1 session, and 0 sessions mix matched/unmatched rows. These are historical projections rather than partial corruption, but 20260603040000_session_message_projection_order rejects them with: Cannot migrate session_message projections without matching durable events. Suggested fix: backfill durable rows from event.seq; for legacy-only sessions assign seq using the prior deterministic (time_created, id) order; keep rejecting mixed sessions. Desktop currently surfaces this as an indefinite loading animation because the sidecar exits before readiness.

[neriousy]

Error: Unexpected error, check log file at /home/neriousy/.local/share/opencode/log/dev.log for more details

Cannot migrate session_message projections without matching durable events
error: script "dev" exited with code 1

potential fix: #30720