Skip to content

feat(runtime): add PTY and stdin control to background Bash - #778

Merged
Astro-Han merged 12 commits into
apache:mainfrom
M4n5ter:feat/interactive-pty-stdin
Jul 13, 2026
Merged

feat(runtime): add PTY and stdin control to background Bash#778
Astro-Han merged 12 commits into
apache:mainfrom
M4n5ter:feat/interactive-pty-stdin

Conversation

@M4n5ter

@M4n5ter M4n5ter commented Jul 12, 2026

Copy link
Copy Markdown
Member
English

Closes #775.

Summary

  • Add opt-in PTY execution to explicitly backgrounded Bash while preserving the existing foreground and background pipe paths.
  • Add permission-gated WriteStdin for exact terminal input and resize, returning the persisted terminal state at the control's parser cut.
  • Represent PTY output as a bounded, redacted terminal-screen snapshot that remains observable through Read(ref) and stoppable through the existing generic StopBackgroundTask({ ref }).
  • Wire the same ShellRun lifecycle into the main Desktop and TUI agents, including durable revision updates, truthful parent-Bash projection, and host shutdown cleanup.

Why

The current runtime-owned background Bash lifecycle handles servers, watchers, and other pipe-oriented commands, but it cannot operate programs that require a real TTY or input after startup. REPLs, debuggers, interactive setup commands, confirmation prompts, and full-screen terminal applications otherwise fall back to shell-level workarounds that bypass Maka's lifecycle, ordering, redaction, recovery, and UI contracts.

Key decisions and rationale

Decision Rationale
Extend the existing ShellRun lifecycle instead of introducing a terminal-session model. Keeps admission, persistence, termination, recovery, and UI projection under one owner.
Allow PTY only for explicitly backgrounded Bash. Preserves foreground Bash timeout and return semantics and avoids implicit execution-mode changes.
Reuse Read(ref) and generic StopBackgroundTask({ ref }); add only PTY-specific WriteStdin. Keeps runtime-resource observation and background-task termination generic, leaving the same extension path for future task types.
Publish a bounded terminal-screen snapshot rather than a stdout/stderr tail. Represents clear/redraw, alternate screen, cursor, and cell width correctly without exposing an unbounded transcript or model context.
Return the terminal state at the control's immediate parser cut; use Read(ref) for later output. Keeps the control schema and timing deterministic and avoids implying that snapshot output was caused by this input.
Keep model-generated input exact in canonical audit/replay; project it only at permission and human-presentation boundaries. Permission governs execution rather than whether a generated call enters history, replay stays semantically valid, and human surfaces receive a bounded redacted projection instead of the canonical payload. This is ordinary audited input, not a secure secret channel.
Scope turn-level permission memory to the complete ref + input + size side effect without truncation. A resize or benign input approval cannot authorize unrelated input on the same PTY, while an exact repeated control can still reuse approval.
Execute exact input unchanged; render only a bounded, redacted, control-escaped human preview. Preserves terminal semantics while making the side effect understandable; byte count is only truncation/fallback metadata, and secure or masked password entry remains out of scope.
Treat a resize to the current dimensions as a successful no-op and report applied separately from changed. Avoids redundant native resize side effects and prevents the UI from claiming a size changed when it did not.
Expose PTY and WriteStdin only in the main Desktop and TUI hosts. Those hosts currently own the live ShellRun lifecycle, update channel, and shutdown cleanup; Headless and child-agent capability should not precede that ownership.
Recover running records as orphaned instead of reattaching native PIDs. Makes restart behavior honest and avoids claiming control without a live driver and terminal state.

Design

Layer Responsibility in this change
core Defines structured pipe/PTY output, strict compact-versus-snapshot ShellRun results, monotonic revisions, operation metadata, safe activity projection, and shared terminal views.
storage Persists complete session-scoped ShellRun snapshots with serialized writes; operation metadata never enters durable records. Running records without a live handle recover conservatively as orphaned.
runtime Keeps one ShellRun manager for foreground pipes, background pipes, and PTYs; owns admission, native drivers, xterm parsing, per-ref control ordering, persistence, process-tree termination, and unique finalization.
Desktop / TUI hosts Expose PTY and WriteStdin only where a host owns the lifecycle, publish durable ShellRun revisions, and terminate live runs during host shutdown.
UI / transcript Reconcile updates by revision into the parent Bash activity; show a bounded semantic input preview and truthful resize outcome rather than a second terminal surface or opaque byte count.

Tool contract

  • Bash({ run_in_background: true, pty: true }) is the only PTY start path. PTY is rejected for foreground Bash, and omitting it preserves current pipe behavior.
  • PTYs start at 80x24. WriteStdin accepts exact Unicode input and/or a complete { cols, rows } resize; when both are present, resize commits before input. No newline is added implicitly.
  • For example, if gdb -tui is cramped at 80x24, the model can resize the running PTY to 120x40 before continuing. The debugger receives the terminal-size change and redraws whichever source, assembly, register, or command windows are active, while the parser uses the same geometry for the returned screen state.
  • Resizing to the current dimensions is an idempotent success: the operation reports applied: true, changed: false, and the UI does not repeat a misleading “resized” message beside an input action.
  • WriteStdin persists and returns the terminal state at the parser cut immediately after its control commit. It does not wait for later output or claim causal attribution; subsequent output is observed through Read(ref).
  • Canonical tool-call history retains exact WriteStdin args, including denied calls. Permission and presentation use a bounded redacted projection, and turn-level permission memory distinguishes the complete ref + input + size side effect without truncation.
  • Read uses strict mutually exclusive file and runtime-resource branches: { path, offset?, limit? } | { ref }. Runtime resources are whole snapshots and do not accept file pagination.
  • StopBackgroundTask({ ref }) keeps its generic schema. WriteStdin rejects pipe and unauthorized/unknown refs; a valid PTY that is already terminal or no longer live returns its snapshot with an unapplied control operation instead of being mutated.

Terminal and lifecycle guarantees

  • PTY bytes are parsed through a headless terminal. The model sees screen/scrollback, cursor, dimensions, status, and revision rather than raw ANSI or synthetic stdout/stderr.
  • Clear/redraw, soft wrapping, Unicode cell widths, and alternate-screen transitions update terminal state instead of appending stale frames.
  • Output is redacted before durable or model-visible publication. Terminal escape sequences cannot trigger host clipboard, notification, title, link, window, or bell side effects.
  • Same-ref controls commit in FIFO order. Control and Read(ref) cuts reserve durable writes in the same order, so revisions cannot publish an older screen after a newer one. Stop closes later control admission and waits for termination, final parser drain, and durable persistence.
  • Timeout, user cancellation, session close, host shutdown, and integrity failure converge on the same process-tree termination and finalization lifecycle.
  • Maka does not reattach native PIDs after restart. Native handles and terminal buffers are explicitly bounded.

Scope

Included:

  • Main Desktop and TUI agent hosts
  • Pipe and PTY execution under one ShellRun manager
  • PTY input, resize, screen projection, durable updates, and process-tree cleanup
  • Compact parent-Bash projection in Desktop and TUI
  • Real native PTY and terminal-parser coverage

Not included:

  • Headless, child-agent, automation, or remote PTY control
  • Pipe-mode stdin
  • Human direct attach or an embedded terminal pane
  • Secure or masked password entry
  • A new task list, task center, or shell-specific status/wait/stop tool family
  • Reattaching the process after host restart or retaining an unlimited terminal transcript

Impact

Agent-facing changes for the main Desktop and TUI hosts:

  • Bash gains optional pty, valid only with explicit background execution.
  • WriteStdin is added as an always-available PTY capability on those hosts.
  • Read's existing runtime-resource form becomes a strict { ref } branch alongside its file branch.
  • Pipe Bash, generic background stop, Headless Bash, and child-agent tools keep their existing capability boundaries.

Current writes and business logic use only the new ShellRun shape. At durable read/restore ingress, the exact immediately preceding ToolResult and shell-run.json shapes are normalized into the current form; unknown, mixed, or malformed shapes still fail closed. This is a narrow review-requested boundary, not a long-lived dual schema or migration mode.

New runtime dependencies are node-pty@^1.2.0-beta.14, @xterm/headless@^6.0.0, and @xterm/addon-unicode11@^0.9.0. The PTY stack is loaded lazily and node-pty is explicitly allowlisted for its install script.

Verification

  • The feature implementation passed root npm run typecheck, npm test, and npm run build.
  • Runtime coverage uses real child processes, node-pty, the durable ShellRun store, and xterm parsing. It covers exact Unicode/control input, resize ordering, clear/redraw, alternate screen, Unicode width, parser backpressure, protocol replies, redaction boundaries, persistence failure, concurrent control/Stop races, timeout, shutdown, and detached-descendant cleanup.
  • Desktop hands-on verification covered a real TTY, exact Unicode input, 100x30 resize, clear/redraw, Ctrl-C, terminal status/revision updates, Read(ref), and bounded human-readable WriteStdin summaries.
  • TUI hands-on verification covered PTY startup, resize, Enter/Ctrl-C, Read(ref), StopBackgroundTask, compact/expanded terminal projection, and parent-Bash reconciliation.

Windows/ConPTY behavior is implemented and typechecked but was not exercised on a Windows host in this change. No formal packaged-Electron validation is claimed; the repository does not currently expose that packaging boundary as a verification target.

Reviewer notes

The highest-value review boundaries are:

  1. shell-run-manager.ts, completion-latch.ts, and process-tree-terminator.ts: admission, termination ownership, process exit, and unique finalization.
  2. pty-screen-collector.ts and pty-process-driver.ts: parser backpressure, protocol replies, screen epochs, side-effect isolation, and native driver behavior.
  3. shell-run-contract.ts, shell-run-tool-result.ts, and storage: compact handoff versus complete snapshots, revision/persistence ordering, and operation-record separation.
  4. shell-tools.ts, builtin-tools.ts, and permission projection: public schema strictness, exact input semantics, bounded human previews, and host capability boundaries.
  5. Desktop/TUI/UI projection: monotonic revision reconciliation and keeping one truthful parent Bash terminal surface.

Review status

This PR is ready for review. The implementation and verification are complete, while #775 remains open for design discussion; the tool and output contracts are not being presented as irreversible. I will adapt the implementation if review identifies a simpler or more Maka-aligned boundary.

简体中文

本 PR 合并后关闭 #775

摘要

  • 为显式后台 Bash 增加可选 PTY,同时保留现有前台 pipe 与后台 pipe 路径。
  • 增加需要权限的 WriteStdin,支持精确终端输入与 resize,并返回该控制对应 parser cut 上已持久化的终端状态。
  • 把 PTY 输出表示为有界、已脱敏的终端画面 snapshot,继续通过 Read(ref) 观察,并通过现有通用 StopBackgroundTask({ ref }) 停止。
  • 在 main Desktop 与 TUI agent 中接入同一 ShellRun 生命周期,包括 durable revision 更新、真实的父 Bash 投影和 host shutdown 清理。

原因

当前由 runtime 持有的后台 Bash 生命周期已经适合 server、watcher 等 pipe 型命令,但仍无法操作需要真实 TTY 或启动后输入的程序。REPL、调试器、交互式初始化命令、确认提示和全屏终端应用只能退回 shell 技巧,而这些技巧绕开了 Maka 的生命周期、顺序、脱敏、恢复与 UI 契约。

关键决策与理由

决策 理由
扩展现有 ShellRun 生命周期,而不引入 terminal-session 模型。 让 admission、持久化、终止、恢复与 UI 投影继续由同一所有者负责。
仅为显式后台 Bash 允许 PTY。 保持前台 Bash 的 timeout 与返回语义,避免隐式切换执行模式。
复用 Read(ref) 与通用 StopBackgroundTask({ ref }),只新增 PTY 专属的 WriteStdin 保持 runtime-resource 观察和后台任务终止协议通用,也为未来其他任务类型保留同一接入路径。
发布有界的 terminal-screen snapshot,而不是 stdout/stderr tail。 正确表达清屏/重绘、alternate screen、光标与字符单元宽度,同时避免暴露无限 transcript 或占用无限模型上下文。
返回 control 后即时 parser cut 的终端状态,后续输出统一通过 Read(ref) 获取。 让控制 schema 与时序保持确定,也避免暗示 snapshot 中的输出由本次输入导致。
Canonical 审计与 replay 保留模型生成的精确输入,只在权限与人类展示边界做投影。 权限决定是否执行,而不是生成的调用是否进入历史;replay 保持语义有效,人类界面接收有界脱敏投影而不是 canonical payload。这是普通审计输入,不是安全 secret channel。
回合级权限记忆按完整且不截断的 ref + input + size 副作用区分。 一次 resize 或无害输入授权不能放行同一 PTY 上无关的后续输入,而完全相同的重复控制仍可复用授权。
原样执行精确输入;人类界面只展示有界、脱敏且已转义控制字符的预览。 保持终端语义,同时让副作用便于理解;bytes 只作为截断/回退元数据,安全或遮罩密码输入仍不在本次范围。
把调整到当前尺寸视为成功 no-op,并分别记录 appliedchanged 避免重复 native resize 的副作用,也避免 UI 声称尺寸发生了实际未发生的变化。
只在 main Desktop 与 TUI host 暴露 PTY 和 WriteStdin 目前只有这些 host 完整持有 live ShellRun 生命周期、更新通道和 shutdown 清理;Headless 与 child-agent 不应先于该所有权暴露能力。
将重启时仍为 running 的记录恢复为 orphaned,不重新 attach native PID。 如实表达重启后的控制能力,避免在缺少 live driver 与终端状态时声称仍可控制。

设计

本次改动中的职责
core 定义结构化 pipe/PTY output、严格区分 compact 与完整 snapshot 的 ShellRun result、单调 revision、operation metadata、安全 activity 投影和共享终端 view。
storage 以串行写入持久化完整的 session-scoped ShellRun snapshot;operation metadata 永不进入 durable record。缺少 live handle 的 running record 保守恢复为 orphaned
runtime 用一个 ShellRun manager 统一前台 pipe、后台 pipe 与 PTY;持有 admission、native driver、xterm parsing、per-ref 控制顺序、持久化、进程树终止和唯一 finalization。
Desktop / TUI host 只在真正持有生命周期的 host 暴露 PTY 与 WriteStdin,发布 durable ShellRun revision,并在 host shutdown 时终止 live run。
UI / transcript 按 revision 把更新合并回父 Bash activity;展示有界的语义输入预览和真实 resize 结果,而不是第二个终端面板或 opaque byte count。

工具契约

  • Bash({ run_in_background: true, pty: true }) 是唯一 PTY 启动路径。前台 Bash 拒绝 PTY;省略 PTY 时保留当前 pipe 行为。
  • PTY 初始尺寸为 80x24。WriteStdin 接受精确 Unicode input 和/或完整 { cols, rows } resize;同时提供时先提交 resize,再提交 input。不隐式追加换行。
  • 例如,若 gdb -tui 在 80x24 下过于拥挤,模型可以先把仍在运行的 PTY 调整到 120x40 再继续操作。调试器会收到终端尺寸变化,并重绘当前启用的源码、汇编、寄存器或命令窗口;parser 则以相同几何尺寸生成返回画面。
  • 调整到当前尺寸是幂等成功:operation 返回 applied: true, changed: false;与 input 同时出现时,UI 不再重复具有误导性的“已调整”文案。
  • WriteStdin 在 control commit 后立即持久化并返回该 parser cut 的终端状态;它不等待后续输出,也不声称 snapshot 中的输出由本次输入导致。后续输出通过 Read(ref) 观察。
  • Canonical tool-call 历史保留精确的 WriteStdin 参数,包括被拒绝的调用。权限与展示使用有界脱敏投影;回合级权限记忆按完整且不截断的 ref + input + size 副作用区分。
  • Read 使用严格互斥的文件与 runtime-resource 分支:{ path, offset?, limit? } | { ref }。runtime resource 是完整 snapshot,不接受文件分页参数。
  • StopBackgroundTask({ ref }) 保持通用 schema。WriteStdin 会拒绝 pipe ref 和未授权/未知 ref;合法但已经终态或不再 live 的 PTY 不会被修改,而是返回 snapshot,并标记 control operation 未应用。

终端与生命周期保证

  • PTY bytes 通过 headless terminal 解析。模型看到的是 screen/scrollback、cursor、尺寸、状态和 revision,而不是 raw ANSI 或人为构造的 stdout/stderr。
  • 清屏/重绘、soft wrap、Unicode cell width 与 alternate-screen 切换更新终端状态,不追加陈旧画面。
  • 所有 durable 或模型可见输出在发布前完成脱敏。终端转义序列不能触发宿主剪贴板、通知、标题、链接、窗口或响铃 side effect。
  • 同 ref 控制按 FIFO 提交。control 与 Read(ref) 的 cut 以相同顺序预留 durable 写入,避免较新 revision 发布较旧画面;Stop 关闭后续控制入口,并等待终止、final parser drain 与 durable persist。
  • timeout、用户取消、session close、host shutdown 与 integrity failure 收束到同一进程树终止和 finalization 生命周期。
  • Maka 重启后不重新 attach native PID。native handle 与终端缓冲都有明确上限。

范围

包含:

  • Main Desktop 与 TUI agent host
  • 同一 ShellRun manager 下的 pipe 与 PTY 执行
  • PTY input、resize、screen projection、durable update 和进程树清理
  • Desktop 与 TUI 中克制的父 Bash 投影
  • 真实 native PTY 与 terminal-parser 覆盖

不包含:

  • Headless、child-agent、automation 或 remote PTY control
  • Pipe-mode stdin
  • 人工直接 attach 或嵌入式终端面板
  • 安全或遮罩密码输入
  • 新任务列表、task center,或 shell 专属 status/wait/stop 工具族
  • host 重启后重新 attach 进程,或保留无限 terminal transcript

影响

Main Desktop 与 TUI host 的 agent-facing 变化:

  • Bash 增加可选 pty,且只允许与显式后台执行配合。
  • 这些 host 始终提供 PTY 能力专属的 WriteStdin
  • Read 现有的 runtime-resource 形式收敛为与文件分支并列的严格 { ref } 分支。
  • Pipe Bash、通用后台停止、Headless Bash 与 child-agent 工具保持现有能力边界。

当前写路径与业务逻辑只使用新的 ShellRun shape。在 durable read/restore ingress,紧邻本 feature 前的精确 ToolResult 与 shell-run.json shape 会被规范化为当前形式;未知、混合或损坏 shape 仍然 fail closed。这是 review 明确要求的窄边界,不是长期双 schema 或 migration mode。

新增 runtime 依赖为 node-pty@^1.2.0-beta.14@xterm/headless@^6.0.0@xterm/addon-unicode11@^0.9.0。PTY stack 采用 lazy load,并为 node-pty 的安装脚本增加显式 allowlist。

验证

  • Feature 实现已通过根级 npm run typechecknpm testnpm run build
  • Runtime 覆盖使用真实 child process、node-pty、durable ShellRun store 与 xterm parser;覆盖精确 Unicode/control input、resize 顺序、清屏/重绘、alternate screen、Unicode width、parser backpressure、protocol reply、跨边界脱敏、持久化失败、并发 control/Stop 竞态、timeout、shutdown 和脱组后代清理。
  • Desktop 人工验证覆盖真实 TTY、精确 Unicode input、100x30 resize、清屏/重绘、Ctrl-C、终端 status/revision 更新、Read(ref) 和有界、可读的 WriteStdin 摘要。
  • TUI 人工验证覆盖 PTY 启动、resize、Enter/Ctrl-C、Read(ref)StopBackgroundTask、终端折叠/展开投影和父 Bash reconciliation。

Windows/ConPTY 路径已经实现并通过 typecheck,但本次没有在 Windows host 上实际运行。不声称完成正式 packaged Electron 验证;当前仓库也没有把该打包边界暴露为验证目标。

审查重点

最值得审查的边界是:

  1. shell-run-manager.tscompletion-latch.tsprocess-tree-terminator.ts:admission、termination ownership、process exit 与唯一 finalization。
  2. pty-screen-collector.tspty-process-driver.ts:parser backpressure、protocol reply、screen epoch、side-effect 隔离和 native driver 行为。
  3. shell-run-contract.tsshell-run-tool-result.ts 与 storage:compact handoff/完整 snapshot、revision/persist 顺序和 operation/record 分离。
  4. shell-tools.tsbuiltin-tools.ts 与权限投影:公开 schema 严格性、精确 input 语义、有界人类预览和 host capability 边界。
  5. Desktop/TUI/UI 投影:单调 revision reconciliation,以及维持唯一、真实的父 Bash 终端面板。

审查状态

本 PR 已可进入正式审查。实现与验证已经完成,同时 #775 仍用于设计讨论;工具与输出契约并不被视为不可调整。若审查找到更简单或更符合 Maka 的边界,我会相应调整实现。

@Astro-Han

Copy link
Copy Markdown
Contributor

Substantial PR, and the lifecycle ownership, fail-closed paths, and real node-pty/xterm test coverage are strong. I verified the items below against the diff; severity is P0-P3.

P1

  1. WriteStdin input reaches storage before the permission decision. The persisted ToolCallMessage (tool-runtime.ts callMsg ~line 261) carries raw args; only the tool_start event args (line 276) go through projectToolActivityArgs. So WriteStdin({ input: "password=..." }) writes the raw input to session JSONL at appendMessage(callMsg) (line 266), before the permission decisionMsg (line 419); rejecting the prompt still leaves the secret on disk, and the dialog only shows the redacted preview. The "must not contain secrets" hint isn't enforced. Closing it: persist the projectToolActivityArgs projection everywhere durable, and reject redactSecrets(input) !== input before execution.

  2. WriteStdin scope key widens one approval into arbitrary input. permission.ts ~line 561 scopes WriteStdin as shell_unsafe:WriteStdin:<ref> with only the ref (no input/size), while Bash scopes by normalized command. WriteStdin can execute arbitrary shell input, so a resize-only approval with "remember for this turn" auto-approves a later rm -rf /\r on the same ref (test "scope memory follows the PTY ref rather than control contents" pins this). Folding normalized input + size into the scope key, or disallowing rememberForTurn for WriteStdin, would close the gap.

  3. TUI crashes restoring sessions with old Bash results. pi-transcript-format.ts terminal case now calls formatShellOutput(content.output) with no guard (the shell_run case has content.output ?); an older terminal result has no output, so undefined.mode throws on restore. Desktop has a malformed-terminal fallback, TUI doesn't. Guarding the terminal case like shell_run, or migrating old records on read, would fix it.

  4. Old ShellRun records become unreadable after upgrade. shell-run-store.ts hasOnlyKeys rejects old fields (stdoutTail/stderrTail/latestOutputStream) and now requires revision + output; listSessionShellRuns silently skips unparseable dirs. Existing background-task refs, orphan recovery, context summary, and Desktop hydration lose those records. If "clear old pre-release state" is the intent, detecting the old shape and telling the user to clear it (or a small migration in normalizeShellRunRecord) is friendlier than a silent drop.

P2

  1. observe_for_ms breaks concurrent attribution. The observe window runs outside the mutateAtCut queue, so a later WriteStdin can commit while an earlier one is still observing, and the first call's returned snapshot can include the second call's output while its operation still describes the first input (test "keeps PTY control FIFO while an earlier WriteStdin observes output" pins this). Read(ref) already gives a full, permission-free snapshot at a clean cut. I'd lean toward dropping observe_for_ms so WriteStdin returns at the current cut and observation always goes through Read; if the one-step convenience is worth keeping, it needs end-to-end serialization, which adds latency and complexity.

P3 (non-blocking)

  • process-tree-terminator.ts uses execFile('/bin/ps', ...). Minimal/distroless images may only have /usr/bin/ps or none; the fallback process.kill(-pid) then misses descendants that escaped the group, and the detach-cleanup test wouldn't hold there. ps via PATH, or a /bin/ps then /usr/bin/ps fallback, would be safer.
  • buildPtyShellSpawnPlan hardcodes /bin/sh -c on POSIX, ignoring the rest of ShellPlan, while the pipe path uses shell: true. Behavior is similar today; a comment that PTY mode is fixed to /bin/sh (or unifying the plans) would keep them from drifting.
  • sanitizeBuffer redacts a whole logical line to [redacted] on any secret, and the whole screen if the joined scrollback+screen trips redactSecrets after the per-line pass. Conservative, but one secret can blank a lot of normal output. If intentional, a one-line note on the tradeoff would help; if not, a targeted redact of just the secret span is possible.
  • runTermination SIGKILLs twice then marks integrityFailure; a process in D-state (NFS/IO) survives SIGKILL and stays alive after the manager drops it. OS limit, not a bug. Worth a line in the acceptance criteria that SIGKILL-unreachable processes are out of scope.
  • scheduleAutomaticFlush for PTY uses Date.now() / lastSnapshotWallTime while the rest uses the injectable this.input.now(), so fake-clock tests can't control PTY flush throttling. Using this.input.now() for lastSnapshotWallTime would keep the test seam consistent.
  • Design note (doc, not a code defect): the RFC/PR doesn't say why the in-process parser route was chosen over the tmux capture-pane route OpenHands uses. The strongest reason looks like cross-platform (Desktop/TUI include native Windows, where tmux isn't a dependable runtime dependency), not resize (tmux panes resize too). A short "why not tmux" note would turn a silent choice into an argued one.

The lifecycle and test coverage are in good shape. The two security and two compatibility items in P1 are what I'd want resolved before merge; the rest can be tracked.

简体中文

这是个很扎实的 PR,生命周期归属、fail-closed 路径,以及用真 node-pty/xterm 的测试覆盖都做得不错。下面这些我都对着 diff 核过,按 P0-P3 分级。

P1

  1. WriteStdin 的 input 在权限决策之前就落盘。持久化的 ToolCallMessagetool-runtime.ts callMsg 约 261 行)带的是原始 args,只有 tool_start 事件的 args(276 行)过了 projectToolActivityArgs。所以 WriteStdin({ input: "password=..." }) 会在 appendMessage(callMsg)(266 行)把原始 input 写进 session JSONL,早于权限 decisionMsg(419 行);即便用户拒绝,密码已经落盘,而权限框只展示脱敏后的预览。"must not contain secrets" 只是提示,没有强制。建议:所有持久化路径(ToolCallMessage、trace、artifact、invocation summary)都改用 projectToolActivityArgs 投影,并在执行前拒绝 redactSecrets(input) !== input 的输入。

  2. WriteStdin 的 scope key 把一次批准放大成任意输入。permission.ts 约 561 行把 WriteStdin 的 scope 定成 shell_unsafe:WriteStdin:<ref>,只有 ref、没有 input/size,而 Bash 是按规范化 command 记。WriteStdin 同样能执行任意 shell 输入,所以一次 resize-only 批准勾"本回合记住"后,对同一个 ref 发 rm -rf /\r 也会自动放行(测试 "scope memory follows the PTY ref rather than control contents" 固化了这个行为)。把规范化 input + size 纳入 scope key,或禁止 WriteStdin 用 rememberForTurn,就能收窄。

  3. TUI 恢复含旧 Bash 结果的会话会崩。pi-transcript-format.tsterminal case 现在直接 formatShellOutput(content.output),没 guard(shell_run case 有 content.output ? 判断);旧 terminal 结果没有 output,读 undefined.mode 就抛异常。Desktop 有 malformed-terminal 兜底,TUI 没有。像 shell_run 那样给 terminal case 加 guard,或读时迁移旧记录,就能修。

  4. 旧 ShellRun 记录升级后读不出来。shell-run-store.tshasOnlyKeys 拒绝旧字段(stdoutTail/stderrTail/latestOutputStream),并强制要求 revision + outputlistSessionShellRuns 静默跳过解析失败的目录。已有的 background-task ref、orphan 恢复、上下文摘要、Desktop hydration 都看不到这些记录。如果"清掉 pre-release 旧 state"是有意为之,识别旧 shape 并提示用户清除(或在 normalizeShellRunRecord 里做小迁移),比静默丢弃更友好。

P2

  1. observe_for_ms 破坏并发归因。observe 窗口跑在 mutateAtCut 队列外,所以后一个 WriteStdin 能在前一个还在 observe 时就提交,第一个调用返回的 snapshot 可能掺进第二个的输出,而它的 operation 仍描述的是第一次输入(测试 "keeps PTY control FIFO while an earlier WriteStdin observes output" 固化了这个行为)。Read(ref) 已经能在干净的 parser cut 给出完整、无权限的 snapshot。我倾向砍掉 observe_for_ms,让 WriteStdin 在当前 cut 返回、观察统一走 Read;如果省一步的便利确实值得留,那它需要端到端串行化,但那会加延迟和复杂度。

P3(非阻断)

  • process-tree-terminator.tsexecFile('/bin/ps', ...)。最小/distroless 镜像可能只有 /usr/bin/ps 或根本没有;fallback 的 process.kill(-pid) 会漏掉逃出进程组的后代,detach-cleanup 测试在那种环境也不成立。用 PATH 里的 ps,或 /bin/ps/usr/bin/ps 兜底,更稳。
  • buildPtyShellSpawnPlan 在 POSIX 写死 /bin/sh -c,忽略 ShellPlan 的其余部分,而 pipe 路径用 shell: true。目前行为差不多;加一句注释说明 PTY 模式固定 /bin/sh(或统一两个 plan),免得以后跑偏。
  • sanitizeBuffer 命中 secret 就把整 logical line 替换成 [redacted],逐行处理后再对拼接的 scrollback+screen 过一次 redactSecrets,命中则整屏替换。保守,但一个 secret 能抹掉大量正常输出。如果是有意的,在 PR 里补一句取舍说明就行;如果不是,可以只脱敏 secret 那一段。
  • runTermination 连发两次 SIGKILL 后标 integrityFailure;卡在 D-state(NFS/IO)的进程扛得住 SIGKILL,manager 丢掉它之后它还活着。这是 OS 限制,不是 bug。建议在验收标准里加一句:SIGKILL 杀不掉的进程不属 runtime 清理范围。
  • scheduleAutomaticFlush 在 PTY 上用 Date.now() / lastSnapshotWallTime,而 manager 其余地方用可注入的 this.input.now(),所以 fake-clock 测试控不了 PTY flush 节流。lastSnapshotWallTime 改用 this.input.now() 就能让测试 seam 一致。
  • 设计层面(文档,非代码缺陷):RFC/PR 没说为什么选进程内 parser 路线、而不是 OpenHands 用的 tmux capture-pane 路线。最强的理由看起来是跨平台(Desktop/TUI 含原生 Windows,tmux 在那里不是可靠的 runtime 依赖),不是 resize(tmux pane 也能 resize)。补一句"为什么不用 tmux",就能把一个沉默的选择变成有论证的选择。

生命周期和测试覆盖整体不错。合并前我希望先解决 P1 里的两个安全项和两个兼容项;其余可以记下来慢慢弄。

@Astro-Han

Astro-Han commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Small process note, not a blocker: at +7376/-1572 across 85 files this is a lot for one review pass; the quality and test coverage are strong, but I leaned on an external review to catch what I missed. For future "contract reshape + new capability" changes, worth considering two PRs, contract first then the new capability on top, so each fits in one pass. No rush here.

简体中文

一个小提醒,不阻断:+7376/-1572、85 个文件,一次 review 真看不全,质量和测试覆盖都很强,我是靠一次外部 review 才补上漏的。以后遇到“契约重塑 + 新能力”这种,可以考虑拆两个 PR,先契约再叠新能力,每个都能一次看全。这次不急。

@M4n5ter

M4n5ter commented Jul 12, 2026

Copy link
Copy Markdown
Member Author
English

@Astro-Han Thank you for the detailed review. The examples are concrete and helped separate several intentional tradeoffs from two boundaries that should be tightened. I went back through the permission, persistence, replay, and TUI paths. My current reading and proposed direction are below.

1. Raw WriteStdin input and the audit boundary

You are right about the current ordering: the raw ToolCallMessage.args is persisted before permission is decided, and denying the request does not remove it.

This was intentional in #775: permission controls whether the side effect executes, while the model-generated tool call remains part of the canonical audit/model history. The same ordering currently applies to Bash commands and Write/Edit contents. Projecting ToolCallMessage.args with projectToolActivityArgs would also make canonical history inaccurate: { inputPreview: ... } is a human-facing projection, not a valid WriteStdin invocation, yet RuntimeEvent replay treats stored function-call args as semantic input.

I am also hesitant to enforce redactSecrets(input) !== input as a security boundary. It would reject harmless text such as echo 'token=placeholder'\r, while a bare password such as hunter2\r could still pass. That would create both false positives and false confidence rather than a secure input channel.

There is, however, a related inconsistency that should be fixed: session JSONL currently retains raw args, while the live tool_start projection can enter the canonical RuntimeEvent ledger as inputPreview. Different recovery paths can therefore replay different calls. I propose keeping canonical tool-call args exact and applying projection only at permission and human-presentation boundaries, including the TUI. I will also replace “must not contain secrets” with clearer wording that this is ordinary audited input, not a secure secret channel.

If Maka wants the stronger invariant that denied tool calls never persist raw arguments, I think that should be designed once at the ToolRuntime semantic-ledger boundary for all tools rather than introduced as a WriteStdin-only exception. I would be glad to discuss that broader contract separately.

2. Permission scope

I agree with this finding. The TUI makes the problem stronger because y currently sends rememberForTurn: true automatically. A resize-only approval can therefore authorize unrelated input on the same ref without an explicit capability-level decision.

I propose including the exact side effect in the scope: ref + input + size, while excluding observe_for_ms. For example, approving { ref: r, size: { cols: 100, rows: 30 } } would not authorize { ref: r, input: "rm -rf /\r" }. Repeating the same Enter/control action can still reuse its scope, but different terminal input requires a new decision.

3. Compatibility

The PR originally followed the repository’s pre-release assumption and explicitly replaced the old schema without migration. Given your review, I agree that a narrow compatibility concession is worthwhile.

I propose normalizing only the exact known legacy shapes at read/restore boundaries:

  • legacy terminal results with top-level stdout/stderr become output: { mode: "pipes", ... };
  • legacy ShellRun tails become pipe output and receive an initial revision;
  • current runtime/UI logic continues to operate on one canonical shape.

This avoids scattered TUI guards and permanent dual-schema branches. Truly malformed data should remain isolated or diagnostic rather than being accepted as legacy data.

4. observe_for_ms

The concurrent behavior is real, although the current contract intended a slightly different interpretation: operation describes what that call committed, while output is the complete terminal state at the later parser cut, not output exclusively caused by that input.

For example, if call A writes one and observes for one second, and call B writes two during that window, A may observe a screen containing both. Serializing the whole observation window would instead make B wait up to 30 seconds, which is why only the control commit is in the FIFO sequencer.

That said, the ambiguity itself is a fair cost. Since Read(ref) already provides a clean observation path, I am open to removing observe_for_ms rather than adding end-to-end serialization. Would the preferred contract be: WriteStdin commits the control and returns at the resulting parser cut, while later output is always obtained through Read(ref)? If so, I will simplify the implementation in that direction.

5. P3 items

I plan to add a fixed /bin/ps/usr/bin/ps fallback rather than relying on mutable PATH, and to document the tmux decision: the in-process path keeps lifecycle/parser ownership inside Maka and supports native Windows/ConPTY without an external session server.

The POSIX PTY and pipe paths both currently resolve to /bin/sh; I can make that intentional equivalence explicit. The coarse terminal redaction is deliberate because secrets may cross wrapped rows or hard boundaries; I will document that conservative loss-of-context tradeoff. SIGKILL-unreachable processes will also be stated as an OS-level limit.

I would not directly replace the scheduling Date.now() with input.now(): the former measures elapsed flush time, while the latter is the injectable durable timestamp source and is not necessarily an elapsed-time clock. If deterministic scheduling tests become necessary, a separate monotonic clock/timer seam would preserve that distinction.

If this direction matches your intent, I will implement the agreed changes, rerun the full validation, and report the exact resulting contract.

简体中文

@Astro-Han 感谢这次细致的审查。你给出的例子很具体,帮助我把有意的设计取舍与确实需要收紧的边界区分开来。我重新核对了权限、持久化、replay 和 TUI 链路,目前的判断与建议如下。

1. WriteStdin 原始输入与审计边界

你对当前顺序的判断是准确的:原始 ToolCallMessage.args 会在权限决定前持久化,拒绝请求也不会删除它。

这是 #775 中有意采用的边界:权限控制副作用是否执行,而模型已经生成的工具调用仍属于 canonical 审计和模型历史。当前 Bash command、Write/Edit 内容也使用相同顺序。用 projectToolActivityArgs 替换 ToolCallMessage.args 还会让 canonical 历史失真:{ inputPreview: ... } 是人类展示投影,不是合法的 WriteStdin 调用,但 RuntimeEvent replay 会把持久化的 function-call args 当作语义输入。

我也不倾向把 redactSecrets(input) !== input 当作安全边界。例如它会拒绝无害的 echo 'token=placeholder'\r,却可能放过裸密码 hunter2\r。这会同时产生误报和虚假的安全感,而不能形成安全输入通道。

不过这里确实存在一个应当修复的相关不一致:session JSONL 保留原始 args,而 live tool_start 的投影可能以 inputPreview 进入 canonical RuntimeEvent ledger,导致不同恢复路径 replay 出不同的调用。我建议 canonical tool-call args 保持精确,只在权限和人类展示边界投影,包括 TUI。同时把“must not contain secrets”改成更准确的说明:这是普通、会被审计的输入,不是安全 secret channel。

如果 Maka 希望建立“被拒绝的工具调用绝不持久化原始参数”这一更强不变量,我认为应当在 ToolRuntime 的 semantic-ledger 边界为所有工具统一设计,而不是加入 WriteStdin 特例。我愿意另行讨论这个更广的契约。

2. 权限 scope

我认同这个 finding。TUI 使问题更明显,因为目前按 y 会自动发送 rememberForTurn: true。因此一次纯 resize 授权可能在用户没有明确批准终端控制能力的情况下,放行同 ref 上完全不同的输入。

我建议把精确副作用纳入 scope:ref + input + size,但排除只影响观察的 observe_for_ms。例如,批准 { ref: r, size: { cols: 100, rows: 30 } } 不会授权 { ref: r, input: "rm -rf /\r" }。重复完全相同的 Enter/control 操作仍可复用 scope,不同输入则重新请求权限。

3. 兼容性

PR 原先遵循仓库尚未发布的假设,并明确直接替换旧 schema、不做迁移。考虑到你的反馈,我认同为兼容做一次范围严格的让步是值得的。

我建议只在 read/restore 边界正规化已知的精确旧 shape:

  • 顶层含 stdout/stderr 的旧 terminal result 转为 output: { mode: "pipes", ... }
  • 旧 ShellRun tail 转为 pipe output,并赋予初始 revision;
  • runtime 和 UI 内部仍只处理一个 canonical shape。

这样可以避免零散的 TUI guard 和长期双 schema 分支。真正损坏的数据仍应被隔离或记录诊断,而不是被当成 legacy 数据宽松接受。

4. observe_for_ms

并发现象确实存在,不过当前契约原本采用的是另一种解释:operation 描述该调用提交了什么,output 则表示稍后的 parser cut 上完整的终端当前状态,并不承诺只包含该输入导致的输出。

例如调用 A 写入 one 并观察一秒,调用 B 在窗口内写入 two,A 可能看到同时含有两者的终端画面。若把整个观察窗口串行化,B 最长会被阻塞 30 秒,因此当前只有 control commit 进入 FIFO sequencer。

不过这种歧义本身确实有成本。既然 Read(ref) 已提供干净的观察路径,我愿意删除 observe_for_ms,而不是增加端到端串行化。想确认你偏好的契约是否是:WriteStdin 提交控制并在相应 parser cut 返回,之后的新输出统一通过 Read(ref) 获取?如果是,我会按这个方向简化实现。

5. P3 项目

我计划增加固定的 /bin/ps/usr/bin/ps fallback,而不是依赖可变的 PATH;同时补充不用 tmux 的原因:进程内实现让生命周期和 parser 继续由 Maka 持有,并能支持原生 Windows/ConPTY,而不引入外部 session server。

POSIX 下 PTY 与 pipe 当前都会落到 /bin/sh,我可以明确记录这是有意保持一致。较粗粒度的终端脱敏也是有意的,因为 secret 可能跨越 wrapped row 或硬边界;我会补充这种保守丢失上下文的取舍。SIGKILL 仍无法终止的进程也会明确为 OS 级限制。

我不建议直接把调度使用的 Date.now() 换成 input.now():前者测量刷新间隔,后者是可注入的 durable 时间戳来源,不一定代表 elapsed time。若未来需要确定性调度测试,单独注入 monotonic clock/timer seam 会更准确。

如果这个方向符合你的意图,我会实现达成一致的修改,重新运行完整验证,并报告最终契约。

@M4n5ter

M4n5ter commented Jul 12, 2026

Copy link
Copy Markdown
Member Author

Small process note, not a blocker: at +7376/-1572 across 85 files this is a lot for one review pass; the quality and test coverage are strong, but I leaned on an external review to catch what I missed. For future "contract reshape + new capability" changes, worth considering two PRs, contract first then the new capability on top, so each fits in one pass. No rush here.

简体中文
一个小提醒,不阻断:+7376/-1572、85 个文件,一次 review 真看不全,质量和测试覆盖都很强,我是靠一次外部 review 才补上漏的。以后遇到“契约重塑 + 新能力”这种,可以考虑拆两个 PR,先契约再叠新能力,每个都能一次看全。这次不急。

English

@Astro-Han You're right; I did not give reviewability enough weight here.

The reason this became one large PR is fairly simple: I had already spent some time privately designing and experimenting with this feature. Once the implementation became coherent, I treated “one complete feature” as “one PR” and submitted it after整理, without stepping back to separate the contract reshaping from the capability delivery from a reviewer's perspective. That concentrated too much review work into a single pass, and I am a little embarrassed that I missed such a basic consideration.

For future changes of this shape, I will split them along independently reviewable boundaries: establish the contract and underlying structure first, then add the capability on top. Even if the work is developed together on a private branch, that does not mean it should arrive upstream as a single PR.

简体中文

@Astro-Han 你说得对,这次我没有充分考虑 PR 的可审查性。

这个 PR 之所以会这么大,原因其实很简单:我前一段时间已经在私下设计和实验这个 feature。等实现整理成一套完整内容后,我下意识地把“一个完整 feature”等同成了“一个 PR”,没有再站在 reviewer 的角度,把契约重塑和新能力交付拆开。这让一次审查承担了过多内容,漏掉这么基本的考虑,我确实有点不好意思。

以后遇到类似改动,我会按能够独立审查的边界拆分:先建立契约和底层结构,再在其上提交新能力。即使这些工作是在同一条私有分支上连续完成,也不意味着它们应该作为一个上游 PR 一次性提交。

@Astro-Han

Copy link
Copy Markdown
Contributor

@M4n5ter Thanks, the code analysis tracks. I'm good with the directions on scope, legacy normalization, and removing observe_for_ms. A few specifics to pin down, and one threat-model call I'd like to confirm.

On P1-1, I accept the rebuttals: projecting ToolCallMessage.args would break RuntimeEvent replay, and redactSecrets isn't a security boundary. I'm also fine with the broader invariant that permission governs execution while model-generated tool calls (including denied ones) are canonical audit history. That matches how Bash/Write/Edit already behave, and a WriteStdin-only exception would be the wrong layer. So I'd resolve P1-1 as an accepted threat-model decision, on top of the ledger/session dual-track unification you proposed: keep canonical args exact in both the live RuntimeEvent and session-backfill paths, apply projection only at permission and presentation boundaries, and rewrite "must not contain secrets" to say this is ordinary audited input, not a secure secret channel. Worth recording that tradeoff in the RFC or PR so it's an argued decision, not a silent one.

A few specifics on the other fixes:

  • Scope: ref + input + size is right. One risk. The existing scope helpers truncate text to 512 and JSON to 1024 (permission.ts ~line 567), so two inputs that differ only past the truncation point would collide. Use the full normalized value or an untruncated hash, and cover that case in the test.
  • Legacy normalization: please cover both durable sources, session JSONL tool_result.content and the RuntimeEvent ledger function_response.result, not just the TUI path, with a regression test on real main-generated fixtures.
  • observe_for_ms removal: if WriteStdin still returns a snapshot, returning only operation/state/revision (no output attribution) would be cleanest; otherwise make sure the contract doesn't imply the returned output came from this call.

Looking forward to the updates.

简体中文

@M4n5ter 谢谢,代码分析对得上。scope、legacy 归一化、砍 observe_for_ms 这几个方向我都没问题。有几个具体点想定一下,外加一个 threat-model 想跟你确认。

P1-1 我接受反驳:投影 ToolCallMessage.args 确实会破坏 RuntimeEvent 回放,redactSecrets 也当不了安全边界。你提的那个更广 invariant(permission 管执行,model 生成的 tool call 含 denied 都算 canonical 审计历史)我也同意,跟 Bash/Write/Edit 现在的行为一致,单独给 WriteStdin 开例外是错的层。所以 P1-1 我定为已接受的 threat-model 决策,叠在你提的 ledger/session 双轨统一之上:live RuntimeEvent 和 session-backfill 两条都保持 canonical 原文,projection 只用在 permission 和展示边界,"must not contain secrets" 改成"这是普通审计输入、不是安全密钥通道"。这个取舍值得在 RFC 或 PR 里记一笔,让它是个有论证的决策。

其他几个具体点:

  • scope:ref + input + size 对的。有个风险,现有 scope helpers 文本截 512、JSON 截 1024(permission.ts 约 567 行),两个只在截断点之后不同的 input 会撞,用完整规范化值或无截断 hash,测试覆盖这个 case。
  • legacy 归一化:两个持久化来源都要覆盖,session JSONL 的 tool_result.content 和 RuntimeEvent ledger 的 function_response.result,不只是 TUI 路径,用真 main 生成的 fixture 做回归测试。
  • 砍 observe_for_ms 后:如果 WriteStdin 还返回 snapshot,只返回 operation/state/revision、不带 output 归因最干净;否则确保契约不暗示返回的 output 来自这次调用。

等你更新。

@M4n5ter

M4n5ter commented Jul 12, 2026

Copy link
Copy Markdown
Member Author
English

@Astro-Han I have pushed the agreed follow-up. The resulting contract is:

  • Canonical versus presentation args: ToolCallMessage, live RuntimeEvent, session backfill, and model replay all retain the exact model-generated WriteStdin args. Permission prompts, Desktop/TUI activity, and summaries use the bounded redacted projection. Denied calls therefore remain canonical audit history, as discussed; the tool description now says explicitly that input is ordinary audited data, not a secure secret channel.
  • Permission scope: turn memory is keyed by the complete normalized ref + input + size side effect, without the generic 512/1024 truncation. For example, two inputs with the same long prefix but different suffixes no longer share approval. Observation fields are gone, so they cannot affect scope.
  • Control/observation split: observe_for_ms and the internal observation wait have been removed. WriteStdin commits the control, reserves persistence in parser-cut order, and returns the terminal state at that immediate cut. The snapshot is current terminal state rather than output attributed to that input; later output is observed through Read(ref).
  • Narrow legacy ingress: only the exact immediately preceding terminal/ShellRun ToolResult and shell-run.json shapes are normalized, at session JSONL, RuntimeEvent read-model/model-replay, and ShellRun-store ingress. The regressions exercise actual FileSessionStore/FileShellRunStore JSON and RuntimeEvent function_response paths. Current business logic still sees one shape; mixed, contradictory, or malformed shell data fails closed with diagnostics instead of falling through as a generic result.
  • Process/documentation follow-up: POSIX process snapshots now try fixed /bin/ps and then /usr/bin/ps. The integration notes record the in-process parser/tmux decision, /bin/sh equivalence, conservative terminal redaction, and the SIGKILL/D-state boundary.

While exercising the final Desktop path, I also tightened the human projection without changing the runtime protocol: WriteStdin retains a bounded, redacted input audit row, and the parent PTY Bash surface updates in place from durable revisions so it reads like one terminal rather than a byte counter or a second terminal.

The PR description has also been revised to match these final contracts.

简体中文

@Astro-Han 已按我们确认的方向推送后续修改,最终契约如下:

  • Canonical 参数与展示参数分离: ToolCallMessage、live RuntimeEvent、session backfill 和模型 replay 都保留模型生成的精确 WriteStdin 参数;权限框、Desktop/TUI activity 与摘要使用有界、脱敏后的投影。因此,正如前面讨论的,即使调用被拒绝,它仍属于 canonical 审计历史;工具说明也已明确写成普通、会被审计的数据,而不是安全 secret channel。
  • 权限 scope: 回合内授权记忆按完整规范化的 ref + input + size 副作用区分,不再经过通用的 512/1024 截断。例如,两段长输入即使前缀完全相同,只要后缀不同,也不会复用授权。观察字段已经删除,因此不会参与 scope。
  • 控制与观察分离: 已删除 observe_for_ms 及内部观察等待。WriteStdin 提交控制,按 parser cut 顺序预留持久化,并返回该即时 cut 上的终端状态。这个 snapshot 表示终端当前状态,而不是归因于本次输入的输出;之后到达的内容通过 Read(ref) 观察。
  • 严格、有限的旧数据入口: 只在 session JSONL、RuntimeEvent read-model/model-replay 和 ShellRun store 入口,规范化紧邻本 feature 前的精确 terminal/ShellRun ToolResult 与 shell-run.json shape。回归覆盖实际 FileSessionStore/FileShellRunStore JSON 与 RuntimeEvent function_response 路径。当前业务逻辑仍只处理一种 shape;混合、矛盾或损坏的 shell 数据会 fail closed 并留下诊断,不会退回通用 result 路径放行。
  • 进程与文档补充: POSIX 进程快照现在固定先尝试 /bin/ps,再尝试 /usr/bin/ps。集成文档记录了进程内 parser 与 tmux 的取舍、/bin/sh 等价性、保守终端脱敏,以及 SIGKILL/D-state 边界。

在最终 Desktop 实测中,我也收紧了人类可见投影,但没有改变 runtime 协议:WriteStdin 继续保留有界、脱敏的输入审计行;父 PTY Bash surface 则按 durable revision 原位更新,使整个交互呈现为一个终端,而不是字节计数器或第二个终端面板。

PR 正文也已修订为上述最终契约。

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@M4n5ter Approving. The follow-up fixes are solid — the agreed contract holds (canonical args, full scope key, legacy ingress on both durable paths, observe_for_ms gone), and the test coverage on legacy fixtures, scope collision, and concurrent cut ordering is good.

A few things I'd treat as fast follow-ups, none of them blocking this merge:

  1. Permission prompt completeness. The permission request sent to the UI carries the projected args (permission-engine.ts:217), so the "view full parameters" panel and TUI summary only show the 160-char inputPreview, not the full input. For a tool that can write 64 KiB, the user can approve a long harmless prefix hiding a dangerous suffix. Worth giving the permission dialog a separate full-input inspector (escaped, with ref + resize shown), distinct from the activity-card projection which should stay bounded.

  2. WriteStdin turn memory. The TUI forces rememberForTurn: true on every allow (pi-tui-runner.ts:273-285), and the scope is ref+input+size with no terminal-version binding. Approving y\r once auto-approves the same input in a later, different prompt. Simplest first step: don't remember WriteStdin within a turn; later bind an expectedRevision checked in the same cut.

  3. applied semantics. input.applied is set right after pty.write() returns void (shell-run-manager.ts:294), but node-pty may still drop bytes silently on a non-EAGAIN error. Renaming to queued or delivery_unknown would stop the model acting on a false "delivered".

  4. ref consistency (small one): the WriteStdin schema declares ref as a bare z.string(), and the projection copies it verbatim while bounding/redacting input. Validating the maka://... format + a length cap, and treating ref like input in the projection, closes that gap.

The rest (persist-failure retry idempotency, process-tree graceful-signal gaps, TUI exit timer vs worst-case shutdown, native node-pty as optional adapter, full-snapshot IPC, monotonic-clock flush) I'd file as separate issues rather than growing this PR further. It's already large enough that full surface coverage in one pass is hard.

Thanks for the iteration. The fixes landed cleanly.

简体中文

@M4n5ter 批准合入。后续修复很扎实,商定的契约都落实了(canonical 原文 args、完整 scope key、两条持久化路径的 legacy ingress、observe_for_ms 已删),legacy fixture、scope 碰撞、并发 cut 排序的测试覆盖也到位。

有几个点我建议作为快速跟进,都不阻塞这次合并:

  1. 权限弹窗完整性。 发给 UI 的权限请求带的是投影后 args(permission-engine.ts:217),所以"查看完整参数"面板和 TUI 摘要只显示 160 字 inputPreview,不是完整 input。一个能写 64 KiB 的工具,用户可能批准一个长无害前缀、后缀藏着危险操作。建议给权限弹窗一个独立的完整输入检查器(转义控制字符,显示 ref + resize),跟活动卡片的投影分开,后者保持有界。

  2. WriteStdin 的 turn memory。 TUI 每次允许都强制 rememberForTurn: truepi-tui-runner.ts:273-285),scope 是 ref+input+size,没绑终端版本。批准一次 y\r,本轮稍后另一个不同的提示里同样的 y\r 会自动放行。第一步最简单:WriteStdin 不在本轮内记住;后续在同一切片内核对 expectedRevision

  3. applied 语义。 input.appliedpty.write()(返回 void)之后立即置真(shell-run-manager.ts:294),但 node-pty 遇到非 EAGAIN 错误可能静默丢字节。改成 queueddelivery_unknown,免得模型基于假的"已送达"继续操作。

  4. ref 一致性(小点):WriteStdin schema 把 ref 声明成裸 z.string(),投影里直接复制完整、不限长,而 input 是有界脱敏的。校验 maka://... 格式 + 长度上限,投影里让 refinput 一致,就补上了。

其余的(persist 失败的重试幂等、进程树的温和信号缺口、TUI 退出计时器 vs 最坏关闭路径、node-pty 作为可选 adapter、全量快照 IPC、单调时钟 flush)建议开独立 issue,别再往这个 PR 里塞。它已经大到单轮 review 很难全覆盖表面积了。

谢谢这几轮迭代,修得很干净。

@M4n5ter

M4n5ter commented Jul 13, 2026

Copy link
Copy Markdown
Member Author
English

@Astro-Han Thank you for the approval and for separating merge blockers from follow-up work. I agree with all four points.

I will keep them out of this already large PR and track them in a focused follow-up issue: #856. The issue keeps the activity projection bounded and redacted while restoring exact args at the permission boundary for a deliberate full escaped-input inspector; disables WriteStdin turn memory; renames the input result to queued so it matches the guarantee exposed by node-pty; and moves canonical ref validation to the schema and projection boundaries as well. Revision-bound control remains a separate enhancement rather than expanding this follow-up.

The permission-facing changes and the runtime contract changes can then land as independently reviewable updates.

Thanks again for the careful review and the concrete examples.

简体中文

@Astro-Han 感谢批准,也感谢你明确区分阻断合并的问题与后续工作。这四点我都认同。

我不会继续扩大这个已经很大的 PR,而会在一个聚焦的后续 issue 中跟踪它们:#856。该 issue 保持 activity 投影有界、脱敏,同时在 permission 边界恢复精确参数,为主动展开的完整转义输入检查提供依据;禁用 WriteStdin 的回合内授权记忆;把输入结果重命名为 queued,使其符合 node-pty 实际暴露的保证;并把 canonical ref 验证同步前移到 schema 与投影边界。Revision-bound control 保持为独立增强,不扩大本次 follow-up。

权限侧修改与 runtime 契约修改可以随后作为彼此独立、易于审查的更新提交。

再次感谢这次细致的审查和具体的例子。

@Astro-Han

Copy link
Copy Markdown
Contributor

LGTM, Merging!

@Astro-Han
Astro-Han merged commit 98e3846 into apache:main Jul 13, 2026
3 checks passed
@M4n5ter
M4n5ter deleted the feat/interactive-pty-stdin branch July 13, 2026 05:29
Astro-Han added a commit that referenced this pull request Aug 3, 2026
Every projection test called project() directly, so the wiring between the
projection and the renderer was untested: a pass added downstream of it — a
spread copy of each turn — restores the original defect with the whole suite
green. That is the shape of #778 breaking #472, which this work exists to
prevent.

Extends the existing render-memo-boundary seam to mount the real ChatView
and count renders per turn through a prop the test owns. The spread-copy
mutation now fails this test.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
Every projection test called project() directly, so the wiring between the
projection and the renderer was untested: a pass added downstream of it — a
spread copy of each turn — restores the original defect with the whole suite
green. That is the shape of #778 breaking #472, which this work exists to
prevent.

Extends the existing render-memo-boundary seam to mount the real ChatView
and count renders per turn through a prop the test owns. The spread-copy
mutation now fails this test.
Astro-Han added a commit that referenced this pull request Aug 3, 2026
…ing it per token (#2034)

* refactor(ui): make a turn's timeline its only tool authority

turn.tools and turn.timeline were two structures carrying the same tools,
each rewritten separately by projectTurnTools. Derive tools by flattening
the timeline instead, in materializeTurns, overlayLiveTurn and
projectTurnTools alike, so a turn has one tool map and the rewrite pass is
idempotent — projecting an unchanged tool set now returns the same turn
object rather than an equal one.

Also splits the shell-run overlay into its folded-updates and per-tool
halves so a caller can apply one update to one tool without re-folding the
whole list.

* feat(ui): project the transcript incrementally and report affected turns

The transcript simulated incremental updates by re-deriving the whole view
model per token and trusting memo's reference comparison to work out what
had not changed. Two links in that chain were not idempotent, so the guess
was wrong: overlayShellRunUpdates re-folded from its original input every
delta and never wrote its result back, and a background command's durable
revision permanently leads the tool_result snapshot persisted in messages,
so the owning turn was rebuilt on every token; and every message refresh
replaced every turn wholesale.

createTranscriptProjection owns that derived state instead. It remembers
the settled turns and hands the previous object back for any turn a refresh
did not change, remembers the result of applying each shell-run update to
each tool, and reports affectedTurnIds — derived from its own output, since
the turn an event names is not the set of turns it affects (a ShellRun
result folds into the Bash that owns its ref, which can live in an earlier
turn).

Coverage asserts identity, not just rendered output, and covers the
invalidation boundaries in both directions: session switch, deletion,
edit-and-resend, turns missing from the durable snapshot, live-to-settled
handoff, lineage, ownership flips at a fixed revision, and live-only tools.

* perf(ui): keep the prompt rail out of the streaming path

The rail's IntersectionObserver effect depended on the whole turns array,
so every streamed token tore it down and rebuilt it — one querySelector and
one observe per turn across the entire transcript. Key the effect on which
turns exist, which is all the observer set depends on, memoize the rail, and
hand it back the previous entries array when no prompt or answer text moved.

* fix(desktop): intern the per-turn props a memoized TurnView reads

A stable turn only skips a memoized TurnView if every sibling prop is
stable too. deriveAppShellTurnViewModel rebuilt turnFooterActionsByTurn and
turnLineageBadgesByTurn from scratch on every refreshMessages, which fires
at each step and tool boundary, so the memo failed on the footer array and
the whole transcript re-rendered regardless of how stable its turns were —
the transcript projection's refresh half bought nothing at the renderer.

Intern both by value against the previous derivation, the same rule the
projection uses for turns. Value equality is the only option available:
messages arrive freshly deserialized over IPC, so no input carries identity
across a refresh and identity has to be re-established from the output.

* test(desktop): pin the transcript render boundary on the real ChatView

Every projection test called project() directly, so the wiring between the
projection and the renderer was untested: a pass added downstream of it — a
spread copy of each turn — restores the original defect with the whole suite
green. That is the shape of #778 breaking #472, which this work exists to
prevent.

Extends the existing render-memo-boundary seam to mount the real ChatView
and count renders per turn through a prop the test owns. The spread-copy
mutation now fails this test.

* refactor(ui): scope the shell-run overlay and drop its unused entry point

Three follow-ups from review:

- The overlay rebuilt every turn's timeline to discover that only the turns
  holding a background command had moved. Scope the rebuild to those turns.
- affectedTurnIds built a Map and a Set over the whole transcript on every
  streamed token for an answer no render path reads. Compute it on read.
- overlayShellRunUpdates had no caller but its own test, so nothing would
  have gone red when it and the projection diverged. Remove it and drive its
  ShellRun behaviours through the projection that actually ships.

Also stores its own copy of the update list, so the element-wise comparison
is against a snapshot rather than against the caller's array aliased to
itself, and covers two gaps the existing tests left: an ownership flip at an
unchanged revision (the previous fixture's leading revision masked the
ownership comparison entirely) and a lock that a text delta never re-reads
the message log.

* refactor(ui): derive per-turn presentation from the projected turns

The shell derived footer actions, failed-turn labels and lineage badges by
materializing the transcript a second time from the raw message log, so the
turns it keyed them by were not the turns the renderer drew. Those props then
had to be interned by value to line up with a memoized TurnView again.

ChatView now hands its projected turns back to the shell through a
`deriveTurnPresentation` callback, and the shell keys its cache on those turn
objects: a turn the projection did not move costs one WeakMap hit and hands the
same props back. The second authority and the interning both go away.

Also drops `affectedTurnIds` (test-only, and repeating an input reported the
previous step's set), the shell-run per-tool memo (its stated benefit was
already provided by reconcileTurnIdentities, and it could serve a result
derived from a stale tool), and makes `valuesEqual` fail closed outside plain
objects and arrays.

Refs #2030

* test(desktop): pin per-turn presentation reuse on a growing transcript

* fix(ui): fold shell-run children independently of tool order

A turn's tools are a flattening of its timeline, and a live overlay moves
that turn's tools to the end of it. That can order a background command's
child tool ahead of the Bash that owns the run, and the fold scanned only
the tools it had already folded — so it stopped folding there, leaving an
orphan tool row and a parent that never took the child's revision.

Look the parent up by ref over the whole list instead, and merge the
children once their parent's position is known. The invariant test now
carries two tools per turn on both the settled and the live path, so a
reversal between `tools` and `timeline` is observable at all.

* refactor(desktop): repair what the presentation move left behind

Moving the per-turn presentation onto the projected turns superseded the
interning it replaced, but left three traces of it.

A literal NUL byte in the cache key's `join` made git treat the file as
binary, so the diff that carries this PR's core was unreviewable. Two
docblocks still pointed at `deriveAppShellTurnViewModel`, which no longer
exists, and `valuesEqual` still justified its export by an interning rule
this change removed — it now documents the coupling that is real: the
shell keys a WeakMap on the turn objects this comparison decides.

`useAppShellTurnPresentation` claimed its own identity had to stay
constant. Nothing memoizes on it, so the `useCallback` bought nothing
while the render-phase ref write it required would have silently frozen
pending state the day ChatView is memoized. Dropping both leaves the
cache where it belongs, in the ref that holds the derivation.

* refactor(ui): drop the prompt rail's redundant second stabilization

The rail derived a NUL-joined id key and mirrored its turns into a ref so
its observer effect would not re-run per streamed token. ChatView already
hands the rail the same array while no rail-visible field moves, so both
layers guarded the same thing — and each covered for the other well
enough that removing either one on its own kept every test green.

Keying the effect on the turns array leaves one guard instead of two, and
turns the caller's reuse into something the observer count can see.

* test(desktop): cover the per-turn fields the presentation feeds TurnView

The stand-in this suite drives ChatView with returns empty maps for
everything but the footer actions, so failure copy, lineage badges, and
the resume pairing were derived in one test and rendered in another,
never in the same frame. Cutting any of those five wires kept the suite
green. A failed, regenerated transcript now runs the real derivation
through the real ChatView and asserts each value reaches the DOM — with
two failed turns, so offering the resume on every one of them is visible
rather than hidden behind the single turn that could render it.

The observer guard was also counting ResizeObserver and
IntersectionObserver into one tally, so `observe > 0` passed even when
the rail observed nothing. Each kind now counts separately and the guard
pins the exact lifecycle.

* docs(ui): state that the presentation deriver must outlive one render

The prop asked only for purity and idempotence, and a deriver rebuilt in
the render body satisfies both while discarding the cache that makes the
projection worth doing — with nothing turning red. Say so on the prop,
and on the one-shot helper that is exactly the shape someone would copy
out of a story.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(runtime): RFC - interactive PTY and stdin for background Bash

2 participants