feat: compare Maka and Pi agent in headless Harbor - #85

Astro-Han · 2026-06-22T02:27:27Z

Summary

add a pi-agent headless backend bridge that maps Pi JSON-mode output into Maka runtime events and token usage
allow Harbor cells to run Pi through the in-repo adapter, using a preinstalled Pi CLI selected by MAKA_PI_COMMAND and explicit MAKA_PI_PROVIDER credentials scoped to the Pi child process
keep Pi-specific provider/model strings out of the AI SDK provider parser
inline the only production Pi backend registration path inside Harbor cell wiring so backends.ts stays fake-only internal plumbing
make Harbor cell config construction an explicit backend switch with non-empty pi-agent and fake slugs
cover the Pi transport, Harbor env entrypoint, adapter wiring, and runtime backend behavior with focused tests

Supersedes closed PR #78 after main was rewritten/rebased.

Review fixes

Pi transport now emits tool calls only from toolcall_end and tool results only from message_end, so runtime traces keep final args/results instead of partial start payloads.
MAKA_BACKEND=pi-agent no longer parses MAKA_MODEL through Maka's AI SDK provider whitelist.
Harbor adapter no longer runs root-level npm install -g for Pi; benchmark images/specs must provide pi or set MAKA_PI_COMMAND.
Pi CLI prompts are passed through stdin instead of argv, avoiding long-prompt argv limits and process-list prompt exposure.
Pi transport now fails closed on unsupported/non-JSON stdout, missing agent_end, late non-zero exits, and stdin write errors before reporting completion.
Pi CLI child env is scoped to non-secret runtime basics plus the selected Pi provider credentials, instead of inheriting the full Harbor/process environment.
The default Pi CLI path now fails fast without MAKA_PI_PROVIDER; explicit test-registered Pi backends can still omit it because they own their transport wiring.
AI SDK _API_KEY_FILE support is not included in this Pi bridge PR; it should land separately if needed.

Verification

npm run -w @maka/runtime test — 590 pass, 0 fail
npm run -w @maka/headless test — 239 pass, 0 fail
git diff --check

Real Harbor smoke

Using the same deepseek-v4-pro model and DeepSeek API key for both Maka and Pi:

task	Maka	Pi	result
fix-git	$0.000272	$0.001959	both pass
prove-plus-comm	$0.000416	$0.003106	both pass
regex-log	$0.001036	$0.011134	both pass
cobol-modernization	$0.001599	$0.013240	both pass
vulnerable-secret	$0.000371	$0.003821	both pass

Across these five scored samples, both harnesses passed 5/5. Pi used about 9x the estimated cost under DeepSeek official deepseek-v4-pro pricing.

filter-js-from-html was excluded from the scored comparison because both agent cells completed but Harbor's Selenium verifier did not finish before the batch run was interrupted.

Note: these real Pi smoke runs used a container setup that made the Pi CLI available. After review hardening, the adapter verifies an existing Pi binary instead of installing it at root during adapter setup.

…) (#149) * fix(headless): make ai-sdk Harbor cell cost and prompt hash controller-consistent The prompt-optimization controller round-trips two values it reads back from the Harbor cell output, but the ai-sdk cell path broke both, so every task would fail plumbing classification once wired to the real controller (unit tests passed only because they used self-consistent fake runners): - Prompt hash: harborCellSystemPrompt() prepended a fixed preamble to config.systemPrompt before the runtime hashed it, so the emitted systemPromptHash was stableHash(preamble+candidate) while the controller expects hashSystemPrompt(candidate) -> prompt_hash_mismatch every task. An explicit prompt is now passed through byte-for-byte (the preamble remains only as the default for prompt-less ad-hoc cell runs). This also makes "the prompt being optimized" equal "the prompt that runs". - Cost: builtin pricing has no deepseek-v4-flash entry, so computeCost(null) returned 0 -> costUsd=0 with tokens>0 -> zero_cost_with_tokens plumbing failure, and the budget ceiling would never trip. The cell now honors the same MAKA_TRIAL_*_USD_PER_1M env the Python adapter (trial_pricing.py) already reads, so one pricing source feeds both runtime cell cost and Harbor trial cost. Tests cover explicit-prompt pass-through and the pricing override. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): secret-safe API key files + trial pricing into the ai-sdk cell Two more prerequisites for running the prompt-optimization loop on the real ai-sdk + DeepSeek path: - Key files: the ai-sdk cell now resolves an API key from `<NAME>_API_KEY_FILE` (e.g. DEEPSEEK_API_KEY_FILE) in addition to the raw env var. The Harbor adapter already forwards the *_API_KEY_FILE path, so the secret can stay in a mounted file and only its path travels through the Harbor CLI / job config — never on a command line. A raw key still takes precedence. (Resolves the deferred AI-SDK _API_KEY_FILE support noted in PR #85.) - Trial pricing reaches the cell: maka_agent.py now forwards MAKA_TRIAL_*_USD_PER_1M (+ MAKA_TRIAL_PRICING_SOURCE) into the in-container cell env, so the cell's pricing override actually activates and emits real costUsd for models absent from builtin pricing (deepseek-v4-flash). Without this the override was unreachable and cost stayed 0. Tests: cell key-file resolution (file + raw-precedence), adapter forwards trial pricing env into _cell_env. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): host HarborTaskRunner that drives `harbor run` for the prompt loop Implements the production HarborTaskRunner the fixed-prompt controller injects (previously only test fakes existed). Per task it generates an isolated single-task Harbor JobConfig (agent maka_agent:MakaAgent, MAKA_BACKEND=ai-sdk), runs `harbor run --config`, and parses results: - reward from the per-trial verifier/reward.txt (the per-trial result.json has no reward field), parsed to a finite number; - status/errorClass/promptHash/cost from agent/maka-cell-output.json; - runtimeEventsPath rewritten to the host trial path (agent/runtime-events.jsonl) so the controller's reward-hack scan and structural smoke can read raw events instead of the container-local path the cell records. Infra vs benchmark split: harbor non-zero exit, missing cell output, or missing/empty/non-numeric reward throw HarborInfraError (controller -> infra_failed, excluded from scoring); a present-but-failed cell (e.g. runtime_error) is returned so the controller marks it unscored+eligible. The system prompt is passed verbatim (trailing newline preserved) for the hash round-trip, the API key travels only as a mounted *_API_KEY_FILE path, and each task uses its own run/round/task job dir to avoid cross-reads under the controller's concurrency. Process invocation is injectable (HarborProcessRunner); tests drive the full parse/classify/JobConfig path with a fake that writes real trial-shaped output. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat: real LLM meta-agent for prompt optimization (deepseek-v4-flash) The prompt-candidate loop only had a scripted meta-agent for tests. Add the real one: - runtime: runOneShotCompletion — a single-turn, tool-less text completion over the same getAIModel + `ai` SDK stack the agent backend uses. Keeps the AI SDK dependency in runtime. - headless: createAiSdkMetaAgent / createAiSdkMetaAgentCompletion wrap it as the loop's MetaAgentCompletion. A JSON-only system instruction plus extractJsonObject (strips ```json fences / surrounding prose) make the model output survive the strict parseMetaAgentResult. The underlying generator is injectable so the loop is testable without a network call. Default model is deepseek-v4-flash (deepseek-chat is a deprecated alias). Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat: add RSI prompt-optimization loop driver Compose the existing baseline-calibration, candidate-round, fixed-prompt controller, acceptance-policy, and structural-smoke layers into one unattended driver: calibrate noise bands, then per round ask the meta-agent for a candidate prompt, sweep held-in + held-out via Harbor, scan held-in trajectories for reward-hacking, and KEEP (advance the lineage) or DISCARD (roll the candidate commit back). Harbor and the meta-agent are injected, so the full composition is tested with fakes and no network or containers. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix: make Harbor cell timeouts configurable and drop deprecated model default Hard-coded timeouts turn slow-but-healthy tasks into infra failures, and the deprecated deepseek-chat alias is being retired. Make both the cell wall-clock budget (MAKA_CELL_TIMEOUT_SEC) and the in-container Bash per-command floor (MAKA_CELL_COMMAND_TIMEOUT_MS) operator-configurable with safe defaults, and default the model to deepseek-v4-flash in both the adapter and the cell. The adapter forwards the command-timeout floor to the cell; malformed or non-positive timeout values fall back to the default. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat: add prompt-optimization run setup wiring Compose a real RSI run: deterministic id-sorted held-in/held-out partition of cached Terminal-Bench tasks, DeepSeek v4-flash/v4-pro pricing constants, canary-GUID reward-hack verifier patterns (the canary lives only in a task's test files, so seeing it in model output is the reward-hack signal), cached-task discovery, and runPromptOptimizationRun gluing the real Harbor runner + DeepSeek meta-agent + CLI git before the loop. Expensive components stay injectable so the wiring is tested without Docker or the network. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix: harden RSI loop against codex-found integration holes Adversarial review surfaced several real defects before the first run: - Native DeepSeek would 400: the runner sent a provider-qualified model id (deepseek/deepseek-v4-flash) while also setting MAKA_PROVIDER, and the cell preserves the slash when a provider is set. Strip the provider- matching prefix for both MAKA_MODEL and model_name; a gateway model whose prefix differs (anthropic/... via openai-compatible) keeps its slash. - Secret exfiltration: the cell passed the full env (incl. *_API_KEY / *_API_KEY_FILE) to every task tool subprocess, so a candidate prompt could cat the key. Scrub provider secrets from the tool child env; the LLM backend already captured the key. - Held-out leak: the driver relied on the caller to hide the held-out TSV. Always pass it as a controller-only artifact so the candidate round rejects if it is visible to the meta-agent. - Duplicate cached task ids would silently collide in the controller's event keying; discovery now fails loud. - Unattended hang: harbor run had no wall-clock ceiling. Add a configurable timeout (default 45m) so a stuck Docker/Harbor becomes an infra failure. - Crash-safety: revert a discarded candidate before persisting the discard decision, so the WAL and git HEAD cannot disagree. - Budget: check the cost/infra stop guard before each round so an over-budget baseline cannot start a new candidate. Documents that the canary reward-hack scan is a best-effort backstop. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat: add runnable RSI prompt-optimization run script A thin Harbor entry that resolves config, builds the seed prompt repo (program.md + system_prompt.md), discovers and partitions cached Terminal-Bench tasks, derives canary verifier patterns, and calls runPromptOptimizationRun, then persists the result and structural smoke report. The DeepSeek key travels as a file path only. Defaults to a 60/20 partition over 10 rounds; cheap-smoke overrides documented in the header. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix: put the harbor adapter dir on PYTHONPATH so harbor imports maka_agent The first real run failed immediately: harbor (a uv-installed tool) could not resolve the bare `maka_agent:MakaAgent` import path because the adapter directory was not on its PYTHONPATH, and the runner invoked harbor with an unmodified env. Prepend <repo>/packages/headless/harbor to PYTHONPATH for the harbor process (preserving any inherited value). Verified: harbor's python imports maka_agent (name = maka) only with this set. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix(runtime): pause stream watchdog during all tool executions The stream idle watchdog (120s default) was paused only for subagent tools. A regular tool that ran longer than the idle timeout — an apt-get install, a build, an ML training step — left the watchdog armed, so at 120s it fired and aborted the whole invocation with "Model stream idle timeout" (redacted to "Request timed out"), even though the model stream is legitimately silent between steps while any tool runs. In the ai-sdk step loop a tool always executes between model requests, so provider silence during a tool is expected for every tool, not just subagents. Pause the watchdog for all tool executions; the tool carries its own timeout (Bash timeout_ms, the cell command timeout) and the trial/run layer is the outer backstop. Surfaced by the Harbor RSI smoke: a held-in trial whose first command was a ~120s `apt-get install r-base` was killed at exactly +120000ms with 0 tokens and no prompt hash, while a sibling trial that never ran a single >120s command completed normally. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): allow explicit task-id selection for prompt-opt runs MAKA_PROMPT_HELD_IN_IDS / MAKA_PROMPT_HELD_OUT_IDS (comma-separated) override the count-based id-sorted partition so a controlled smoke can target known tasks instead of whatever sorts first. Both must be set together, must be disjoint, and every id must exist or the run aborts before spending Docker time. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): drop baseline-unstable tasks instead of aborting the run The driver calibrated directly on every configured task, so a single baseline trial that did not complete (scored + eligible) across all sweeps tripped the strict completeness check and aborted the whole run — fatal for an unattended 60/20 run over hard Terminal-Bench tasks where occasional non-completion is expected. Wire the existing selectStablePromptTasks into the baseline step: drop tasks that fail to complete across every baseline sweep (completion-only filter — a flaky-pass task's variance is honest noise the band already absorbs), then calibrate and run all candidate rounds on the stable subset. Dropped tasks are excluded from candidate sweeps too, so they neither cost more nor skew a decision. The run aborts only when a whole partition has no stable task left. Candidate-round non-completion was already handled gracefully (lower coverage -> coverage_regressed -> discard); only baseline calibration was fragile. Result gains droppedHeldInTaskIds / droppedHeldOutTaskIds for transparency; the runner logs them. Structural smoke is unaffected: an unscored task_completed still counts as completed and trips no failure (only plumbing failures do). Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix(runtime): give the Grep tool a self-timeout and honour turn abort Now that the stream idle watchdog is paused during all tool executions, it no longer incidentally caps a tool's wall-clock time, so every process-spawning tool must carry its own bound. Bash already does (spawn timeout + abortSignal); Grep ran ripgrep via execAsync with neither, so a pathological tree (network mount, /proc, a FIFO) could hang it and — with the watchdog paused — hang the whole turn. Add a generous self-timeout and forward the turn's abort signal so the rg process is killed, not leaked. Found by codex challenge review of the watchdog change. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): add a minimum-stable-task floor and reject duplicate task ids Two robustness gaps from the codex challenge review of the baseline stability filter: - The driver only aborted when a partition had zero stable tasks, so a harness/cache regression that silently dropped most tasks would still calibrate and decide on a tiny, unrepresentative subset and could promote an overfit prompt. Add minStableHeldInTasks / minStableHeldOutTasks (default 1, threaded through runPromptOptimizationRun and exposed as MAKA_PROMPT_MIN_STABLE_HELD_IN / _HELD_OUT) so a run can fail loud when too few tasks survive baseline. - Explicit MAKA_PROMPT_HELD_IN_IDS / _HELD_OUT_IDS accepted duplicates, which would double-weight a task in the pass-rate summary. Reject duplicates up front with a clear error. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): expose MAKA_PROMPT_MAX_INFRA_FAILURE_RATE in the run script The loop already supports an infra-failure-rate stop guard; surface it as an env knob so a long unattended run aborts early if Docker/Harbor breaks mid-run (e.g., the VM dies) instead of burning hours emitting infra_failed events. Left unset by default. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * feat(headless): optional duration cap to drop pathologically slow tasks Some Terminal-Bench tasks are very slow for a small model (many pdflatex / compile / training iterations). With Docker capped at ~8GB the run must stay at low concurrency, so a handful of multi-minute tasks dominate wall-clock and can push a full 10-round run past 12 hours. Add maxStableTaskDurationMs (threaded through runPromptOptimizationRun, env MAKA_PROMPT_MAX_STABLE_TASK_MS): a task whose baseline trial exceeded the cap in any sweep is dropped from the calibrated set and every candidate round via the existing selectStablePromptTasks 'too_slow' path. Unset = no cap (prior behaviour). Calibration stays honest — the cap only trims which tasks the loop iterates on. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix(headless): drop held-in tasks with no canary verifier pattern A held-in task whose tests carry no `terminal-bench-canary GUID` string cannot be scanned for reward-hacking, so the controller quarantines every round it completes in — which fails the structural smoke. headless-terminal is such a task and sorts into the default 60-task held-in partition. Drop no-pattern tasks from held-in (an unverifiable task does not belong in the set the meta-agent optimizes); held-out keeps them since it is not reward-hack-scanned. When held-in ids were pinned explicitly, fail loud instead of silently changing the caller's set. Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * chore: ignore python __pycache__ artifacts Claude-Session: https://claude.ai/code/session_015yqs8KqJo3DB5DtkGeiNTA * fix(headless): harden RSI run guards against silent disabling Addresses review findings on the prompt-optimization runner: - P1: default the cost ceiling to $30 and validate every guard knob via fail-loud parsers (envFinitePositiveNumber/envRatio/envNonNegativeInt), so a malformed env value throws instead of yielding NaN that passes a `!== undefined` check and silently disables the guard. - P1: exit non-zero when the structural smoke does not pass, so CI and shell callers don't treat a structurally-broken run as success. - P2: scale the min-stable floors with the actual post-drop partition sizes (ceil(size * ratio), at least 1) instead of a flat default of 1; an explicit MAKA_PROMPT_MIN_STABLE_* still wins for cheap smokes. - P2: scan a task's tests/ tree recursively for canary GUIDs so verifier material nested in fixtures (tests/data/…) is found, not silently treated as "no pattern" and dropped from held-in. Extracts the parsers into a pure, unit-tested prompt-optimization-env module and adds behavior tests for the validators and the recursive canary scan. Full headless + runtime suites green. * fix(headless): commit missing prompt-optimization-env source + harden min-stable P1 (blocking): the prompt-optimization-env.ts source was left untracked in the prior commit, so index.ts/runner/tests imported a module absent from the tree — a clean checkout's build would fail. Commit the source. P2: resolveMinStable now rejects an explicit floor of 0. The loop's guard is `selectedTaskIds.length < floor`, so a 0 floor can never trip and would silently disable the stable-sample protection; an explicit value must be a positive integer (cheap smokes already pass 1). The loop-level "0 stable tasks aborts calibration" behavior is already covered in prompt-optimization-loop.test.ts. * refactor(headless): keep vendor pricing out of the public API (P3) DeepSeek pricing was exported from @maka/headless's public index, leaking vendor-specific run config into the package's generic surface. - Delete DEEPSEEK_V4_PRO_PRICING outright — it was defined, exported and tested but never consumed (dead code). - Move DEEPSEEK_V4_FLASH_PRICING into the runner (run-prompt-optimization.mjs), its only consumer; drop the definition from prompt-optimization-run.ts and the export from index.ts. The harbor-adapter wiring test still asserts the constant is present in the runner source. @maka/headless's public API now exposes only the generic loop / Harbor runner / discovery helpers; the genuinely generic, injectable runPromptOptimizationRun and its task-discovery helpers stay (they carry no vendor specifics). The pricing-value unit test is dropped: the figures are operational config, not correctness logic, and the wiring test still pins the constant name. * fix(headless): fail loud on a malformed runner pricing object The DeepSeek pricing object now lives in the plain-JS runner, so it lost both the HarborTaskPricing type-check and the value unit test. A mistyped rate field would leave it undefined and the runner would silently emit wrong/zero costUsd. Assert each canonical rate field is a finite, non-negative number at startup — before any Docker time. The field-name -> MAKA_TRIAL_* forwarding contract remains covered in harbor-task-runner.test.ts. * fix(headless): require a positive round count for a full run (P2) MAKA_PROMPT_ROUNDS=0 produced a baseline-only run that trivially passed the structural smoke (minimumRounds 0), contradicting the unattended >=1-round validation the runner exists to perform. Add a fail-loud envPositiveInt parser (rejects 0/negative/non-integer) and use it for rounds. Unit-tested. * fix(headless): retry a thrown Harbor infra error once before failing the task #64 LOOP step 6: a thrown Harbor/Docker error is an infra failure and is often transient. Retry the harborRunner once; only record task_infra_failed when the retry also throws. Keeps a single Docker hiccup from poisoning a candidate's pass-eligible rate. * feat(headless): gate the held-out sweep behind the held-in gate and check budget at sweep boundaries #64 LOOP steps 7-10: split acceptanceReason into a held-in gate and a held-out gate sharing one decision source. The loop now runs the held-in sweep, evaluates heldInGateReason, and only spends the held-out sweep when the candidate clears held-in (delta beyond the noise band). A candidate that regresses or sits within noise is discarded without paying for held-out. Also enforce the cost ceiling at sweep boundaries (step 14): guard before each baseline sweep and before the held-out sweep, reverting the candidate prompt and breaking out cleanly when the budget is hit, so the loop never overshoots into an extra sweep. Concurrency stays at 4 — the ceiling is handled at the loop's stopGuard, never passed into the controller. * refactor(headless): move runner-only env helpers off the public package entry The env-parsing helpers (envFinitePositiveNumber, envNonNegativeInt, envPositiveInt, envRatio, resolveMinStable, smokeExitCode) are consumed only by the prompt-optimization runner, not by package API consumers. Expose them through a dedicated "@maka/headless/prompt-optimization-env" subpath export instead of the main entry, and repoint the Harbor runner's import. Keeps the package's public surface to genuine contracts; tests still import the helpers via their relative module path. * fix(headless): harden prompt-optimization runner config against silent misconfig Three runner-config fixes from review: - Pin the model to deepseek/deepseek-v4-flash instead of reading MAKA_PROMPT_MODEL. DEEPSEEK_V4_FLASH_PRICING is tied to that model, so an override would let cost and smoke accounting silently use the wrong rates. The loop is contractually a deepseek-v4-flash run; provider/baseUrl stay overridable since they don't change the pricing. - Parse MAKA_PROMPT_MAX_CONCURRENCY with the positive-int parser instead of the finite-positive one, so a fractional value (e.g. 1.5) fails loud at startup rather than being silently floored later in the controller. Widen envPositiveInt to accept an optional fallback (mirroring envFinitePositiveNumber) so the undefined-default knob stays type-honest. - Clarify the cost-ceiling comment: it is a round/sweep-boundary ceiling (checked before each baseline sweep and the held-out sweep), not a hard mid-task cap, so a single in-flight sweep can complete past it. * fix(headless): reject rounds < 1 in the loop API so baseline-only runs cannot pass the smoke The runner already rejects MAKA_PROMPT_ROUNDS=0 via envPositiveInt, but runPromptOptimizationLoop is callable directly and only rejected negative rounds. A 0-round run is baseline-only and would trivially pass the structural smoke (minimumRounds 0), so it must not share the normal structural-pass semantics. Enforce rounds >= 1 at the public API too. * fix(headless): enforce numeric input invariants in the core loop and controller The CLI env parsers validated env strings, but runPromptOptimizationLoop and runFixedPromptController are public and callable directly, where a NaN, fraction, or 0 would slip past a `value < 1` / `cost >= ceiling` comparison and silently disable a guard or change semantics (rounds 1.5 runs two rounds; a NaN ceiling never trips; minStable 0 disables the stable-task floor). Add a single source of truth for the invariants — numeric-guards (assertPositiveInt / assertNonNegativeInt / assertFinitePositive / assertRatio) — and enforce it at both public APIs: - loop: validate rounds, baselineRuns, zScore, minStableHeldIn/Out, maxStableTaskDurationMs, costCeilingUsd, maxInfraFailureRate, maxConcurrency. - controller: validate costCeilingUsd and maxInfraFailureRate up front; normalizeMaxConcurrency now rejects a fractional value instead of flooring it, and always runs (even when a stop guard forces concurrency to 1) so the value is never left unchecked. - env parsers become string->number then delegate to the same guards, so the CLI and the core enforce one identical contract. * refactor(headless): make the env helpers a package-private import, not a public subpath The runner-only env helpers were reachable as the public subpath export "@maka/headless/prompt-optimization-env", widening the package's contract for internals that only the Harbor runner uses. Move them behind a Node package "imports" entry (#prompt-optimization-env) so the runner can still import them but external consumers cannot — the public surface is just the main entry. * fix(headless): address prompt runner review findings * fix(headless): fail closed on in-container ai-sdk secrets * fix(headless): reject pi-agent in Harbor adapter * fix(headless): narrow prompt runner public exports * fix(headless): guard prompt task partitions

Astro-Han added 10 commits June 22, 2026 14:31

feat: support pi token usage frames

d107496

feat: add pi cli json transport

a07fcc3

feat: run pi agent through harbor cells

142cd61

feat: wire pi harbor adapter

4d63974

fix: handle pi stdin write failures

9b8c447

fix: catch late pi stdin failures

9715172

fix: stabilize pi usage frame typing

43e0969

fix: harden pi cli transport completion

7aa7d11

refactor: simplify pi transport helpers

8bb51a8

fix: require pi provider for default cli transport

fdb5862

Astro-Han force-pushed the codex/pi-agent-headless-smoke branch from 44a0e99 to fdb5862 Compare June 22, 2026 06:35

Astro-Han merged commit 115bc9d into main Jun 22, 2026

Astro-Han deleted the codex/pi-agent-headless-smoke branch June 22, 2026 06:36

Navigation Menu

Uh oh!

feat: compare Maka and Pi agent in headless Harbor - #85