Repository navigation
http2: release queued data memory on stream cancellation - #67113
Conversation
|
Review requested:
|
Release the memory charged for queued outbound data when a stream is canceled, so subsequent requests can reuse the session's memory budget. Refs: nodejs#66739 Assisted-by: Codex Signed-off-by: inoway46 <inoueyuya416@gmail.com>
f8ee532 to
15bcc85
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #67113 +/- ##
========================================
Coverage 90.65% 90.66%
========================================
Files 797 797
Lines 278268 278793 +525
Branches 53517 53617 +100
========================================
+ Hits 252261 252763 +502
+ Misses 16386 16381 -5
- Partials 9621 9649 +28
🚀 New features to boost your workflow:
|
|
Landed in 80a74e1 |
Allow clients to receive responses when queued request data exceeds maxSessionMemory, while preserving limits on received headers. Fixes: nodejs#66739 Refs: nodejs#67113 Assisted-by: Codex Signed-off-by: inoway46 <inoueyuya416@gmail.com>
Signed-off-by: Fatih Güzel <fatihguzeldev@gmail.com>
|
Could this get Impact. undici ≥ 8 multiplexes up to 100 streams on one long-lived Measured on v24.22.2 (undici 8.12.3, With the default 10 MB budget a single aborted 16 MiB upload loses 2–5 MB of it for the session's lifetime (checked by bisecting Minimal repro (no undici, plain const http2 = require('node:http2')
const session = http2.connect('https://<any-h2-origin-accepting-uploads>', { maxSessionMemory: 1 })
function post (body, ms) {
return new Promise((resolve) => {
const req = session.request({ ':method': 'POST', ':path': '/upload' })
req.on('response', (h) => resolve({ status: h[':status'] }))
req.on('close', () => resolve({ rstCode: req.rstCode }))
req.end(body)
if (ms) setTimeout(() => req.destroy(), ms) // abort while body is still queued
})
}
;(async () => {
console.log(await post(Buffer.alloc(16 << 20), 300)) // { rstCode: 8 } CANCEL, expected
console.log(await post(Buffer.alloc(4096))) // { rstCode: 11 } ENHANCE_YOUR_CALM — leaked budget
console.log(await post(Buffer.alloc(4096))) // { rstCode: 11 } forever
})()Context: e2b-dev/e2b SDK-425 — concurrent |
|
Thanks for the detailed report! Given the production impact, I agree this is worth backporting to the supported LTS lines. |
…TP/2 dispatchers (#2086) ## Summary Fixes [SDK-425](https://linear.app/e2b/issue/SDK-425): concurrent `sandbox.files.write()` calls fail with `TypeError: fetch failed` → `NGHTTP2_ENHANCE_YOUR_CALM` since `e2b` ≥ 2.35 (undici 8). Simpler alternative to #2083 (byte budget + one-off sessions), same measurements. **Cause.** The RST is sent by Node's own http2 client, not by E2B infra. Node queues each buffered request body in full against the session's 10 MB `maxSessionMemory`; if a response HEADERS frame arrives on *any* stream while that budget is exhausted, Node resets that stream with `ENHANCE_YOUR_CALM`. undici 7 kept one request per session (so our 10-connection pool = 10 budgets); undici 8 multiplexes, so every envd REST request in the process shares one session and one budget — parallel or large uploads reset each other and the reads next to them. Streams destroyed before their body is flushed (timeouts, aborts, killed sandboxes) additionally leak their queued bytes on Node ≤ 24 (fixed on `main` by nodejs/node#67113), so a session degrades until every small write on it fails — the customer's "climbing since Monday". **Fix (JS SDK only — Python uses pyqwest/hyper, not Node http2):** 1. `Filesystem` POSTs `/files` (octet-stream, streamed, gzip and multipart uploads) through `createEnvdUploadFetch` — an HTTP/1.1 dispatcher pool of its own (`E2B_ENVD_UPLOAD_CONNECTIONS`, default 50, like `E2B_ENVD_RPC_CONNECTIONS`); everything else (`read`, `makeDir`, RPC, health…) stays on HTTP/2, and `httpVersion` / `E2B_HTTP_VERSION` keep governing those. The `ConnectionOpts.httpVersion` JSDoc documents the exception. ```ts // sandbox/filesystem/index.ts const uploadFetch = createEnvdUploadFetch({ proxy: connectionConfig.proxy }) … this.envdApi.api.POST('/files', { …, fetch: this.uploadFetch }) ``` One request per connection, no shared memory budget — what undici 7 effectively gave 2.27.1 (which capped uploads at 10 per process; this pool allows 50, beyond that uploads queue in undici with their request timeout running). 2. `buildDispatchedFetch` (`src/undici.ts`): a request that fails with `ERR_HTTP2_STREAM_ERROR` + `NGHTTP2_ENHANCE_YOUR_CALM`/`NGHTTP2_INTERNAL_ERROR` (`isPoisonedHTTP2SessionError`) retires that dispatcher and builds a fresh one for subsequent requests. The failed request is **not** replayed (envd may already have applied the write). Only the dispatcher that served the failing request can replace the current one. Safety net for whatever else might fill a session. No API change. **Measured** (built package, real sandboxes, Node 24.22; ALPN confirmed: writes `http/1.1`, reads `h2`): | scenario | 2.62.1 | this PR | |---|---|---| | 10 × 2 MB parallel writes, 5 rounds | 16/50 failed | 0/50 | | 30 × 1 MB, 20 × 1 MB, 4 × 16 MB, customer pattern (makeDir + 10 parallel small writes ×200) | — | 0 failed | | 16 MB write + 10 concurrent `files.read()` | 30/30 reads failed | 0/33 | | poisoned session (`maxSessionMemory=1`, aborted big writes) | 50/50 fail, forever | 0 failed | | streamed (async iterable), gzip, multipart, string, aborted upload | — | all OK | **Upstream:** backport request on nodejs/node#67113 (nodejs/node#67113 (comment)) and undici issue nodejs/undici#6149. Link to Devin session: https://app.devin.ai/sessions/98f09d080013403b954d3f7b07c51b3b Open in Devin Desktop: https://app.devin.ai/desktop/session/98f09d080013403b954d3f7b07c51b3b?variant=devin Requested by: @mishushakov --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mish@e2b.dev <mish@e2b.dev>
Canceling a stream with queued outbound data leaves that data charged to
maxSessionMemory. Subsequent requests on the same session can then fail even though the queued data has been discarded.Release the remaining outbound memory accounting when cleaning up the stream's write queue, and add a regression test for session reuse after cancellation.
Found while investigating #66739. This does not change response-header admission or resolve the early-response issue itself.
Refs: #66739