Skip to content

fix(v8-bridge): yield a macrotask between bridge socket reads (undici keep-alive leak)#122

Merged
NathanFlurry merged 1 commit into
mainfrom
fix/v8-bridge-socket-read-macrotask-yield
Jun 25, 2026
Merged

fix(v8-bridge): yield a macrotask between bridge socket reads (undici keep-alive leak)#122
NathanFlurry merged 1 commit into
mainfrom
fix/v8-bridge-socket-read-macrotask-yield

Conversation

@NathanFlurry

@NathanFlurry NathanFlurry commented Jun 24, 2026

Copy link
Copy Markdown
Member

Problem

The in-VM bridge socket read loop (_pumpBridgeReads in crates/execution/assets/v8-bridge.source.js) is async but contained no await. Because _netSocketReadRaw is synchronous, the loop drained every available byte and emitted readable/data for an entire HTTP response in one synchronous burst.

That collapses the event-loop turn boundaries undici's keep-alive socket recycling depends on:

  1. undici's H1 parser reaches onMessageComplete in the same synchronous stack.
  2. It resolves the caller's fetch synchronously, and only defers socket reuse with setImmediate(client[kResume]).
  3. The caller's await fetch() continuation (a microtask) dispatches the next request before that setImmediate macrotask runs.
  4. Every pooled Client is still kNeedDrain, so Pool.kGetDispatcher constructs a new Client + socket per request.

Each new Client registers ~4 EventEmitter listeners, so the count climbs until it trips MaxListenersExceededWarning and the in-VM Node process exhausts memory and is OOM-killed (the V8 isolate's 128 MiB heap cap fires near_heap_limit_callbackterminate_execution). It only reproduces in the VM because real Node delivers response bytes across multiple I/O turns, so setImmediate(kResume) reliably fires before the next request.

Fix

await a setImmediate-backed macrotask yield before delivering each payload, so socket bytes surface across distinct event-loop turns — exactly as on real Node where each readable arrives in its own I/O callback. This lets setImmediate(kResume) clear kNeedDrain before the next dispatch, so the pool reuses one keep-alive socket.

Empirically (from the original investigation): 40 sequential requests went from 40 sockets / 41 Clients to 1 socket / 2 Clients.

The single v8-bridge.source.js feeds both the execution and v8-runtime crates, so both pick up the fix from one source.

Test

Adds a deterministic regression test to the javascript_v8 suite that scripts two socket chunks over the mocked net bridge and schedules a setImmediate (standing in for setImmediate(kResume)) the instant the first chunk lands. It asserts the macrotask interleaves before the second chunk:

  • Synchronous-burst bug → trace data:chunk-1,data:chunk-2,immediate → guest throws → test fails.
  • Fixed → trace data:chunk-1,immediate,data:chunk-2 → test passes.

Verified both directions locally (reverting the source edit reproduces the failure).

@railway-app railway-app Bot temporarily deployed to secure-exec / secure-exec-pr-122 June 24, 2026 16:07 Destroyed
@railway-app railway-app Bot temporarily deployed to rivet-frontend / secure-exec-pr-122 June 24, 2026 16:07 Destroyed
@railway-app

railway-app Bot commented Jun 24, 2026

Copy link
Copy Markdown

🚅 Deployed to the secure-exec-pr-122 environment in rivet-frontend

Service Status Web Updated (UTC)
secure-exec 😴 Sleeping (View Logs) Jun 24, 2026 at 4:15 pm

🚅 Deployed to the secure-exec-pr-122 environment in secure-exec

Service Status Web Updated (UTC)
secure-exec 😴 Sleeping (View Logs) Web Jun 24, 2026 at 4:14 pm

_pumpBridgeReads drained the synchronous _netSocketReadRaw in a tight loop
and emitted readable/data for an entire HTTP response in one synchronous
burst. That collapses the event-loop turn boundaries undici's keep-alive
socket recycling depends on: onMessageComplete resolves the caller's fetch
synchronously and only defers reuse via setImmediate(client[kResume]), so
the caller's microtask dispatches the next request before kResume runs.
Every Client stays kNeedDrain, the pool allocates a fresh Client+socket per
request, and each registers EventEmitter listeners until the in-VM Node
process dies (MaxListenersExceededWarning + unbounded memory / OOM).

Await a setImmediate-backed macrotask yield before delivering each payload
so socket bytes surface across distinct event-loop turns, exactly as they
do on real Node where each readable arrives in its own I/O callback. This
lets setImmediate(kResume) clear kNeedDrain before the next dispatch, so the
pool reuses one keep-alive socket. Empirically: 40 sequential requests went
from 40 sockets / 41 Clients to 1 socket / 2 Clients.

Adds a deterministic regression test (javascript_v8 suite) that scripts two
socket chunks over the mocked net bridge and asserts a setImmediate scheduled
when the first chunk lands interleaves before the second chunk. It fails on
the synchronous-burst trace (data:chunk-1,data:chunk-2,immediate) and passes
on the fixed trace (data:chunk-1,immediate,data:chunk-2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@NathanFlurry NathanFlurry force-pushed the fix/v8-bridge-socket-read-macrotask-yield branch from a27574f to 4f6f4d0 Compare June 25, 2026 08:52
@railway-app railway-app Bot temporarily deployed to rivet-frontend / secure-exec-pr-122 June 25, 2026 08:52 Destroyed
@railway-app railway-app Bot temporarily deployed to secure-exec / secure-exec-pr-122 June 25, 2026 08:52 Destroyed
@NathanFlurry NathanFlurry merged commit 9d2572a into main Jun 25, 2026
0 of 3 checks passed
NathanFlurry added a commit that referenced this pull request Jun 25, 2026
…idge leak

Three native-sidecar fixes behind long-lived VM latency/stall.

1) Read-side shadow-tree re-walk
Read-side guest fs ops (Exists/Stat/Lstat/ReadFile/Pread) reconcile the host
shadow tree into the kernel VFS, re-reading+re-writing EVERY file on EVERY op
(O(whole tree), super-linear). Add an rsync-style (size,mode,mtime) lstat skip for
unchanged files. Test read_side_ops_skip_unchanged_shadow_files: warm read over an
unchanged 800-file tree >4x cheaper than cold (cold=22.7s, warm=28ms debug).

2) EventEmitter shim: spurious MaxListenersExceededWarning
ensureEventEmitterInitialized() defaulted _maxListenersWarned but not
_maxListeners, so emitters that acquire _events outside our ctor (undici's
Client/Pool/Agent) had _maxListeners=undefined and 'total <= undefined' warned on
the FIRST listener. Default _maxListeners so the threshold is meaningful.

3) undici client-per-request leak (the real net-bridge leak)
The bridge's UndiciAgent was created with an UNBOUNDED per-origin pool. Requests
that overlap while clients are still connecting each find every client kNeedDrain
and spawn a fresh Client+socket -- and for HTTPS the LLM path is HTTP/2 (ALPN), so
each spawn is a whole new h2 session. The synchronous bridge reads widen that
connect window (the #122 per-payload macrotask yield only helps h1 same-socket
reuse, nothing for the h2 connect-window herd). Over a long many-call turn the
abandoned clients accumulate connect/close/drain/error/finish/readable/end/
terminated listeners without bound -> http2 degradation -> 'Request was aborted.'
Bound connections (6, browser-like; h2 multiplexes within each) so excess requests
queue on existing clients instead of spawning new ones.

Test keepalive_no_listener_leak now drives CONCURRENT requests (the trigger) and
asserts the host-side connection count stays bounded: 160 requests over 6
connections with the cap vs 16 (2x concurrency) without it. A process.on('warning')
handler surfaces any MaxListenersExceededWarning to stderr (the warning alone is
insufficient -- leaked listeners spread across per-client emitters).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NathanFlurry added a commit that referenced this pull request Jun 25, 2026
…idge leak

Three native-sidecar fixes behind long-lived VM latency/stall.

1) Read-side shadow-tree re-walk
Read-side guest fs ops (Exists/Stat/Lstat/ReadFile/Pread) reconcile the host
shadow tree into the kernel VFS, re-reading+re-writing EVERY file on EVERY op
(O(whole tree), super-linear). Add an rsync-style (size,mode,mtime) lstat skip for
unchanged files. Test read_side_ops_skip_unchanged_shadow_files: warm read over an
unchanged 800-file tree >4x cheaper than cold (cold=22.7s, warm=28ms debug).

2) EventEmitter shim: spurious MaxListenersExceededWarning
ensureEventEmitterInitialized() defaulted _maxListenersWarned but not
_maxListeners, so emitters that acquire _events outside our ctor (undici's
Client/Pool/Agent) had _maxListeners=undefined and 'total <= undefined' warned on
the FIRST listener. Default _maxListeners so the threshold is meaningful.

3) undici client-per-request leak (the real net-bridge leak)
The bridge's UndiciAgent was created with an UNBOUNDED per-origin pool. Requests
that overlap while clients are still connecting each find every client kNeedDrain
and spawn a fresh Client+socket -- and for HTTPS the LLM path is HTTP/2 (ALPN), so
each spawn is a whole new h2 session. The synchronous bridge reads widen that
connect window (the #122 per-payload macrotask yield only helps h1 same-socket
reuse, nothing for the h2 connect-window herd). Over a long many-call turn the
abandoned clients accumulate connect/close/drain/error/finish/readable/end/
terminated listeners without bound -> http2 degradation -> 'Request was aborted.'
Bound connections (6, browser-like; h2 multiplexes within each) so excess requests
queue on existing clients instead of spawning new ones.

Test keepalive_no_listener_leak now drives CONCURRENT requests (the trigger) and
asserts the host-side connection count stays bounded: 160 requests over 6
connections with the cap vs 16 (2x concurrency) without it. A process.on('warning')
handler surfaces any MaxListenersExceededWarning to stderr (the warning alone is
insufficient -- leaked listeners spread across per-client emitters).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NathanFlurry added a commit that referenced this pull request Jun 25, 2026
…idge leak (#128)

Three native-sidecar fixes behind long-lived VM latency/stall.

1) Read-side shadow-tree re-walk
Read-side guest fs ops (Exists/Stat/Lstat/ReadFile/Pread) reconcile the host
shadow tree into the kernel VFS, re-reading+re-writing EVERY file on EVERY op
(O(whole tree), super-linear). Add an rsync-style (size,mode,mtime) lstat skip for
unchanged files. Test read_side_ops_skip_unchanged_shadow_files: warm read over an
unchanged 800-file tree >4x cheaper than cold (cold=22.7s, warm=28ms debug).

2) EventEmitter shim: spurious MaxListenersExceededWarning
ensureEventEmitterInitialized() defaulted _maxListenersWarned but not
_maxListeners, so emitters that acquire _events outside our ctor (undici's
Client/Pool/Agent) had _maxListeners=undefined and 'total <= undefined' warned on
the FIRST listener. Default _maxListeners so the threshold is meaningful.

3) undici client-per-request leak (the real net-bridge leak)
The bridge's UndiciAgent was created with an UNBOUNDED per-origin pool. Requests
that overlap while clients are still connecting each find every client kNeedDrain
and spawn a fresh Client+socket -- and for HTTPS the LLM path is HTTP/2 (ALPN), so
each spawn is a whole new h2 session. The synchronous bridge reads widen that
connect window (the #122 per-payload macrotask yield only helps h1 same-socket
reuse, nothing for the h2 connect-window herd). Over a long many-call turn the
abandoned clients accumulate connect/close/drain/error/finish/readable/end/
terminated listeners without bound -> http2 degradation -> 'Request was aborted.'
Bound connections (6, browser-like; h2 multiplexes within each) so excess requests
queue on existing clients instead of spawning new ones.

Test keepalive_no_listener_leak now drives CONCURRENT requests (the trigger) and
asserts the host-side connection count stays bounded: 160 requests over 6
connections with the cap vs 16 (2x concurrency) without it. A process.on('warning')
handler surfaces any MaxListenersExceededWarning to stderr (the warning alone is
insufficient -- leaked listeners spread across per-client emitters).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant