fix(core): recover a conversation whose engine session was pruned upstream - #114
Merged
Merged
Conversation
Claude Code prunes its transcript files under ~/.claude/projects/<slug>/ after a retention window. The core stores engine_session_id forever under the default idle_policy 'infinite', so a conversation left idle past that window resumes a uuid the engine no longer holds. On this install 3 of 9 sessions carried a pruned id; the oldest surviving transcript is dated 2026-07-20 & the three dead sessions were last active 2026-07-02, 07-14, & 07-15. isMissingSessionError() matches the phrasing each engine uses for that failure. claude returns "No conversation found with session ID: <uuid>" & codex returns "thread not found"; gemini mints its own resume id, so it has no dead-resume failure mode. The regex deliberately doesn't match a spend-limit result, a 529, a "spawn node EACCES", or a 413, all of which this box has logged. Retrying those as a fresh session would throw away a live conversation's history for a fault that has nothing to do with resume. sessions.clearEngineId() nulls engine_session_id plus the last_stable_hash & last_stable_engine markers, keeping working_dir, idle policy, outbound route, & turn count. It isn't remove(): those four columns describe the conversation, not the engine session, & the conversation is still alive. Dropping the stable marker matters on codex, where inject-once would otherwise skip re-sending a stable block the fresh session never received. 8 tests cover the matcher & the store change, using an ASMLTR_CORE_DB temp file so they never touch the live sessions table.
Engine session ids are only persisted on success, so a pruned id was never cleared. The SDK threw "No conversation found with session ID: <uuid>", the error propagated to /v2/handle as a 500 & to /v2/stream as an error frame, & the row kept the dead id. Every later message on that conversation failed identically. One idle month bricked a channel for good, with no way back short of hand-editing the sessions table. Three of this box's Discord conversations were sitting in that state, including the owner's DM. Both runTurn call sites (the inbound pipeline & the operator inject/steer) now catch that one error, clear the dead id, record a `session-expired` control event carrying the dead uuid & the engine's reason, & rerun the turn once with resume:null. The inbound pipeline reruns with the FULL system prompt, because a fresh session has never been sent the stable block regardless of what inject-once decided for the resumed one. A turn whose AbortController fired (Stop button, or a steer with interrupt) is excluded, as is any turn that carried no resume id. Recovery is one attempt, not a loop: if the fresh session also fails, that error propagates as before. Verified against the running install. A session row rewritten to a bogus uuid returned HTTP 500 twice in a row before the change. After it, the same request returned HTTP 200 with the reply, persisted a new id, & the follow-up turn resumed that id & recalled the prior turn's answer, so the recovered session is a real continuing session rather than a respawn per turn. 85 of 85 tests pass.
This was referenced Aug 18, 2026
jarethmt
pushed a commit
that referenced
this pull request
Aug 24, 2026
session-resume-recovery.test.js opened the sessions DB on import and never closed it. On Node 24 in CI, better-sqlite3's native Statement finalizers ran during the environment teardown and tripped `Assertion failed: (env) != nullptr`, crashing the test process on exit — every assertion passed, only the exit crashed, which is what turned `test` red on main since PR #114. Closing the handle in an after() hook finalizes the statements while the env is still alive.
jarethmt
pushed a commit
that referenced
this pull request
Aug 24, 2026
Every CI run since #114 merged has failed, always the same way: node::RemoveEnvironmentCleanupHook(v8::Isolate*, CleanupHook, void*) at ../src/api/hooks.cc:142 Assertion failed: (env) != nullptr Statement::~Statement() [core/node_modules/better-sqlite3/build/Release/better_sqlite3.node] Nothing asserts. Every test in the file passes and then the process aborts, which marks the file failed and takes the run with it. #114 is not at fault beyond being the first test to load the session store, and neither is the test: the abort is raised by the library's own destructor. Node 24.19.0, released 2026-08-03, added cleanup hooks to node::ObjectWrap (nodejs/node#63642). NAN-style native addons register through that path, so their destructors now call RemoveEnvironmentCleanupHook after the environment is gone, and that is an assert rather than a throw. better-sqlite3 11.10.0 is NAN-based. 13.0.0 is the first release built on N-API, which never touches node::ObjectWrap, so the assertion cannot fire. Measured, one variable at a time, each with a fresh npm ci: better-sqlite3 11.10.0 + node 24.19.0 abort, 5 runs out of 5 better-sqlite3 11.10.0 + node 24.18.0 clean better-sqlite3 13.0.3 + node 24.19.0 88 pass, 0 fail better-sqlite3 13.0.3 + node 24.18.0 88 pass, 0 fail Pinning the workflow to 24.18.x would also have turned CI green and was the wrong fix. The core loads better-sqlite3 at import and runs under pm2 on the host, so the exposure is not limited to CI: whenever the host moves to 24.19.0, the core starts aborting at teardown. A pin would have hidden that behind a green badge. Upgrade path checked rather than assumed. v12.0.0 dropped EOL node 18 and old Electron, and this repo already requires node >=24. v13.0.0's breaking change is packaging only: prebuild-install is gone and prebuilt binaries ship with the package. Neither changes the API, and the repo's whole usage is prepare, exec, close, pragma and transaction. Regenerating the lock hoists better-sqlite3 to one root entry where there were three per-workspace entries. Verified that core, connectors and insights/collector all still resolve it, on both node lines, from the committed package.json and lock via npm ci. Worth knowing when reproducing: on node 24.18.0 there is nothing to see. A prebuilt binary compiled against 24.18 headers also stays clean when run under 24.19.0, so a partial upgrade hides it too. The abort needs a fresh build against 24.19.0 headers.
gtwy
pushed a commit
to gtwy/asmltr
that referenced
this pull request
Sep 1, 2026
…ssion fix(core): recover a conversation whose engine session was pruned upstream
gtwy
pushed a commit
to gtwy/asmltr
that referenced
this pull request
Sep 1, 2026
session-resume-recovery.test.js opened the sessions DB on import and never closed it. On Node 24 in CI, better-sqlite3's native Statement finalizers ran during the environment teardown and tripped `Assertion failed: (env) != nullptr`, crashing the test process on exit — every assertion passed, only the exit crashed, which is what turned `test` red on main since PR jarethmt#114. Closing the handle in an after() hook finalizes the statements while the env is still alive.
gtwy
pushed a commit
to gtwy/asmltr
that referenced
this pull request
Sep 1, 2026
Every CI run since jarethmt#114 merged has failed, always the same way: node::RemoveEnvironmentCleanupHook(v8::Isolate*, CleanupHook, void*) at ../src/api/hooks.cc:142 Assertion failed: (env) != nullptr Statement::~Statement() [core/node_modules/better-sqlite3/build/Release/better_sqlite3.node] Nothing asserts. Every test in the file passes and then the process aborts, which marks the file failed and takes the run with it. jarethmt#114 is not at fault beyond being the first test to load the session store, and neither is the test: the abort is raised by the library's own destructor. Node 24.19.0, released 2026-08-03, added cleanup hooks to node::ObjectWrap (nodejs/node#63642). NAN-style native addons register through that path, so their destructors now call RemoveEnvironmentCleanupHook after the environment is gone, and that is an assert rather than a throw. better-sqlite3 11.10.0 is NAN-based. 13.0.0 is the first release built on N-API, which never touches node::ObjectWrap, so the assertion cannot fire. Measured, one variable at a time, each with a fresh npm ci: better-sqlite3 11.10.0 + node 24.19.0 abort, 5 runs out of 5 better-sqlite3 11.10.0 + node 24.18.0 clean better-sqlite3 13.0.3 + node 24.19.0 88 pass, 0 fail better-sqlite3 13.0.3 + node 24.18.0 88 pass, 0 fail Pinning the workflow to 24.18.x would also have turned CI green and was the wrong fix. The core loads better-sqlite3 at import and runs under pm2 on the host, so the exposure is not limited to CI: whenever the host moves to 24.19.0, the core starts aborting at teardown. A pin would have hidden that behind a green badge. Upgrade path checked rather than assumed. v12.0.0 dropped EOL node 18 and old Electron, and this repo already requires node >=24. v13.0.0's breaking change is packaging only: prebuild-install is gone and prebuilt binaries ship with the package. Neither changes the API, and the repo's whole usage is prepare, exec, close, pragma and transaction. Regenerating the lock hoists better-sqlite3 to one root entry where there were three per-workspace entries. Verified that core, connectors and insights/collector all still resolve it, on both node lines, from the committed package.json and lock via npm ci. Worth knowing when reproducing: on node 24.18.0 there is nothing to see. A prebuilt binary compiled against 24.18 headers also stays clean when run under 24.19.0, so a partial upgrade hides it too. The abort needs a fresh build against 24.19.0 headers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A conversation whose engine session gets pruned upstream is wedged permanently. Every message to it fails, and nothing self-heals.
Claude Code deletes its transcript files under
~/.claude/projects/<slug>/<uuid>.jsonlafter a retention window. The core storesengine_session_idforever under the defaultidle_policy = 'infinite', so a conversation idle past that window passes a uuid the engine no longer holds tooptions.resume. The SDK fails the turn:Because
sessions.recordEngineId()only runs on success, the dead id stays in the row. The next message resumes the same dead id and fails the same way./v2/handlereturns 500,/v2/streamemits an error frame, and the user sees their connector's generic error text after a clean identity allow.On the install where this was found, 3 of 9 sessions were in that state, including the owner's Discord DM. The oldest surviving transcript was dated 2026-07-20; the three dead sessions were last active 2026-07-02, 07-14, and 07-15. One idle month is enough to brick a channel, and the only way out was editing the
sessionstable by hand.The fix
engines.isMissingSessionError(err)recognizes that one failure. claude words it "No conversation found with session ID: ", codex words it "thread not found". gemini mints its own resume id, so it has no dead-resume failure mode.sessions.clearEngineId(key)nullsengine_session_id,last_stable_hash, andlast_stable_engine, keepingworking_dir, idle policy, outbound route, and turn count. It isn'tremove(): those columns describe the conversation, not the engine session. The stable marker has to go too, or inject-once would skip re-sending a stable block that the fresh session never received.Both
runTurncall sites (the inbound pipeline indispatch, and the operator inject/steer) catch that error, clear the dead id, record asession-expiredcontrol event with the dead uuid and the engine's reason, then rerun the turn once withresume: null. The inbound pipeline reruns with the full system prompt, not the volatile tail. Recovery is one attempt, not a loop.The engine-side history is already gone when this fires, so restarting the session loses nothing that failing would have preserved.
What is deliberately not retried
Retrying an unrelated failure as a fresh session would discard a live conversation's history for a fault that has nothing to do with resume. The matcher rejects every one of these, all taken from the install's own error log:
You've hit your monthly spend limitYou've hit your session limit · resets 5:50pmAPI Error: 529 OverloadedFailed to spawn Claude Code process: spawn node EACCESrequest entity too largeAn aborted turn (Stop button, or a steer with
interrupt:true) is excluded as well, as is any turn that carried no resume id.Verification
8 new tests in
test/session-resume-recovery.test.jscover the matcher and the store change, against anASMLTR_CORE_DBtemp file so they never touch a live sessions table. Full suite: 85 of 85 pass.End to end on the running install, with a session row rewritten to a bogus uuid:
No conversation found with session IDThe follow-up turn is the part that matters: the recovered session is a real continuing session, not a respawn on every message. The
session-expiredevent lands in the collector, so a silent restart shows up in the dashboard rather than passing unnoticed.