Read Playbook run progress from the session that runs it - #1147
Merged
Conversation
Both Playbook e2e regressions that keep `main` red assert that a block should be shimmering and find that it is not. One cause: a run adopts session state transitions that have nothing to do with it. A PTY harness announces `Running` when it spawns and idles again at its first prompt. A Run is armed before its prompt goes out (#1122), so when that boot pair is still in flight the run reads it as its own turn starting and ending, and settles its shimmer milliseconds after arming it. Daemon traces from the failing test show exactly that: ARM at +0.0ms, `Running` at +0.9ms (latches `seen_running`), `AwaitingInput` at +2.0ms (clears the run), and the annotation edit 28ms later finds no run to declare against. A fork Run was worse. The run is keyed by the Playbook's session but its turn happens in the fork, so the *owner* — a bystander doing no work for this run — settled the fork's blocks just by finishing something of its own, or by finishing booting. A run now carries the session its turn actually happens in and the instant its prompt reached that session, and only reads lifecycle signals from that session after that instant. Since the daemon's view of a session's state is advanced by the same in-order event drain that reports these transitions, an idle observed at dispatch also means every event emitted before it has been consumed — so a `Running` after it can only be a new turn. When the session was not idle at dispatch, the turn already in flight is not this run's: its turn starts at the next `Running` after an intervening idle, which also stops a Run queued behind a live turn from settling when *that* turn ends. The idle-without-a-turn backstop now runs from delivery rather than from arming, which matters for a fork whose prompt waits on harness readiness (spec 0149). Rule recorded in specs/0176. Local flake rate under CPU contention, before → after: fork Run 19/20 → 0/25 failures, shimmer 5/20 → 0/25.
Contributor
Author
|
Full Loop detail behind the table in the description — each iteration is a fresh
The fork test failing 10/10 locally with no load is the same race as CI's ~50%, just with the ordering pinned the other way: on this machine the owner's boot |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The flake
Two Playbook e2e tests fail intermittently and take
mainred with them (6 of the last 12mainruns):playbook_view.rs::playbook_selection_fork_run_delivers_and_submits_prompt—assert!(annotated_block.shimmer)playbook_shimmer.rs::playbook_stops_shimmering_when_the_owning_session_dies— the precondition poll, "Run never lit the executed region"Both assert a block should be shimmering and find it is not, and both have the same cause.
Root cause
A run adopts session state transitions that have nothing to do with it.
A PTY harness announces
Runningwhen it spawns and idles again at its first prompt. A Run is armed before its prompt goes out (#1122), so when that boot pair is still in flight the run reads it as its own turn starting and ending — and settles its own shimmer milliseconds after arming it.Traced against a real daemon (the failing e2e fixture, instrumented):
The fork case is worse than a timing window. A run is keyed by the Playbook's session, but a fork Run's turn happens in the fork — so the owner, a bystander doing no work for this run, settled the fork's blocks just by finishing something of its own. That is a live bug, not only a test artifact: Run a selection into a fork, then keep working in the Playbook session, and the fork's shimmer disappears while the fork is still going.
The shimmer test failed the same way; it passed locally only because the client's first
playbook.getbeat the daemon's processing of that pair by ~1ms. On a loaded runner it loses.The fix
A run now carries the session its turn actually happens in and the instant its prompt reached that session, and reads lifecycle signals only from that session, only after that instant.
The daemon's view of a session's state is advanced by the same in-order event drain that reports these transitions — so "the session was idle when we dispatched" is a statement about the event stream: everything emitted before that idle has already been consumed, and a
Runningafter it can only be a new turn. When the session was not idle at dispatch, the turn already in flight isn't ours; the run's turn starts at the nextRunningafter an intervening idle. That also fixes a Run queued behind a live turn settling when that turn ends.The idle-without-a-turn backstop now runs from delivery rather than from arming, which matters for a fork whose prompt waits on harness readiness (spec 0149).
Rule recorded in
specs/0176-playbook-run-tracks-its-execution-session.md.Flake rates (this machine, 12 spinner processes of CPU contention)
origin/main)Without load, the fork test failed 10/10 on
origin/mainand 0/25 after.Tests
cargo test -p construct-daemon --lib— 536 passed (3 new: pre-dispatch transitions ignored, fork run reads the fork's lifecycle, dead Playbook session still clears its run). Two existing tests were updated to encode the new rule rather than the old one.cargo test -p construct-cli— 1321 passedcargo test -p construct-protocol— passedcargo test --workspace— see comment belowNo recording: this is daemon lifecycle logic with no new user-visible surface (the visible effect is a shimmer that stops vanishing).
Binaries
The PR touches
crates/daemon,crates/protocol, and a CLI test fixture; the daemon is what runs this code, and it lives in the singleconstructbinary:/Users/moon/agentd/.claude/worktrees/fix-playbook-run-owner-idle-race/target/debug/construct