Skip to content

child_process: spawns take ~60 ms longer when the Node process was started via a shebang (macOS, v26.13.0 regression) #67163

Open

Description

@gigamonkey

Version

v26.13.0 (regressed; v26.12.0 and v26.5.0 are fine)

Platform

Darwin bitbox.local 25.4.0 Darwin Kernel Version 25.4.0: Thu Mar 19 19:33:25 PDT 2026; root:xnu-12377.101.15~1/RELEASE_ARM64_T6041 arm64 arm Darwin

Subsystem

child_process (libuv process spawning)

What steps will reproduce the bug?

[This writeup is by Claude so I can't 100% vouch for it but it seems right to me. At the very least the reproduction script is legit: it clearly shows the difference between v26.13.0 and v26.12.0 on my machine. I am not an expert at node internals or any macOS issues that might be causing this just for me but everything in this report seems to match what I observed as Claude helped me debug this problem.]

Node binaries are the official nodejs.org arm64 tarballs (installed with
mise, but the issue is independent of the installer; the node binaries'
code signatures are identical between the fast and slow versions, see
below).

# A script whose interpreter is node, spawning a trivial child 100 times.
cat > /tmp/sb <<'EOF'
#!/usr/bin/env node
const { execFile } = require("child_process");
const N = 100; let i = 0; const t = Date.now();
(function next() {
  if (i++ === N) { console.log(((Date.now() - t) / N).toFixed(1), "ms per spawn", process.version); return; }
  execFile("git", ["--version"], next);
})();
EOF
chmod +x /tmp/sb

/tmp/sb          # kernel starts the process via the shebang -> node
node /tmp/sb     # same script, node started directly

How often does it reproduce? Is there a required condition?

Every time. Results on the same machine, same minute (#!<abs path to node>
scripts so each version is the interpreter; git --version is the child):

node binary started via shebang started as node script
v26.5.0 3.3 ms per spawn 3.2 ms
v26.12.0 3.2 ms per spawn —
v26.13.0 55.5 ms per spawn 3.3 ms

Three consecutive rounds of the v26.13.0 case: 61.8 / 61.9 / 58.4 ms via
shebang, 3.4 / 3.4 / 3.3 ms started directly. /usr/bin/env node typed at
the shell (not through the kernel's #! handling) is also fast, so the
condition is specifically that the kernel launched the process as a script
interpreter.

The required condition is only how the root of the process tree was
launched — the slowness is inherited by descendants:

  • A #!/usr/bin/env node script that spawnSyncs a second node which
    runs the benchmark: the second node is slow too (61.7 ms).
  • A #!/bin/bash script that does exec node … (so the process that is
    now node was created via a shebang with bash as the interpreter) is slow
    as well, and so are the node children it spawns. The same bash script
    run as bash script is fast.
  • Started directly, node spawning node spawning children: fast (3.5 ms).

Things that make no difference (each tested): environment variables
(replaying the full environment of the slow case into a directly-started
node is fast); PATH length (35 entries vs 2 — the failed posix_spawn PATH
probes are not the cost); which git (Apple's /usr/bin/git stub,
Homebrew's, by absolute path); nice/priority (normal in both); fsync in
the child (a child that does no I/O, git --version, shows it).

What is the expected behavior? Why is that the expected behavior?

A spawn should cost the same few milliseconds whether or not the node
process was started by the kernel as a script interpreter, as it does in
v26.12.0 and earlier.

What do you see instead?

~55–90 ms per child_process spawn, every spawn, for the life of the
process and all its node descendants.

Where the time goes: the child itself is not slow and process creation is
not slow — the parent is late noticing the exit. Instrumented in a real
workload (an Express app's git store, ~850 execFile("git", …) calls per
test run), with timestamps at execFile(), the child's spawn event, and
the completion callback, plus GIT_TRACE_PERFORMANCE=1 for git's own
clock:

directly started tree shebang-started tree
execFile → spawn event 0.6–1.0 ms 0.8–1.0 ms
git's own elapsed (GIT_TRACE_PERFORMANCE) ~2.5 ms ~2.5 ms
execFile → callback 6.8 ms p50 / 8.0 ms p90 92.7 ms p50 / 160 ms p90

So ~85 ms elapses between the child exiting and the parent's callback.
The per-command p50s are identical for every git subcommand (config,
rev-parse, hash-object, …), consistent with a fixed per-spawn delay
rather than work.

Additional information

Real-world impact. Almost everything that launches node in a dev
workflow does it through a shebang: npm itself (bin/npm →
npm-cli.js, #!/usr/bin/env node; version managers' bash wrappers are
shebang scripts too), npx, and every node_modules/.bin/* shim. So
npm test on v26.13.0 on macOS runs any suite that spawns processes with
this tax on every spawn, while node --test … typed directly does not.
Our gate's git-heavy tests went from ~300 ms to 5–9 s per test (10–50×),
and two tests with real-world deadlines (a 1 s push timeout; "a Save
returns within 10 s") failed. It took two days to find because nothing
run "by hand" reproduces it.

Not a code-signing / Gatekeeper difference. codesign -dv --verbose=2
on the two binaries: both flags=0x10000(runtime),
TeamIdentifier=HX7739G8FX, Runtime Version=15.5.0, same entitlements.
macOS's syspolicyd does log a violates validation category policy
line for the short-lived children that posix_spawn PATH probing creates,
but that happens equally in the fast and slow cases and the PATH-length
test shows it is not the cost.

Suspicion. Something in the process-spawning or child-exit-notification
path changed between v26.12.0 and v26.13.0 (libuv upgrade?) and behaves
differently when the process's original exec was the kernel's interpreter
path (imgact_shell) — e.g. a check that looks at the process's
executable path / argv[0] / P_* flags, or a kqueue/EVFILT_PROC vs
SIGCHLD difference where the shebang-launched process has a different
state. The bisect range is exactly v26.12.0 → v26.13.0; happy to test
nightlies or a patch.

Workarounds we use. Pin macOS hosts to v26.12.0; launch scripts with
an explicit interpreter (bash script, node script, node …/npm/bin/npm-cli.js run …) so no process in the tree is shebang-started.

Activity

  1. richardlau commented on Oct 31, 2026

    @richardlau
    Member

    Suspicion. Something in the process-spawning or child-exit-notification path changed between v26.12.0 and v26.13.0 (libuv upgrade?) and behaves differently when the process's original exec was the kernel's interpreter path (imgact_shell) — e.g. a check that looks at the process's executable path / argv[0] / P_* flags, or a kqueue/EVFILT_PROC vs SIGCHLD difference where the shebang-launched process has a different state. The bisect range is exactly v26.12.0 → v26.13.0; happy to test nightlies or a patch.

    It looks like libuv/libuv#5390.

  2. gigamonkey commented on Oct 31, 2026

    @gigamonkey
    Author
    1. unix,process: skip missing PATH entries in spawn libuv/libuv#5390

    Though isn't that issue about misses in the PATH? That is specifically not the problem i ran into: we tested with both my regular 30+ item path and with a stripped down 2-item path and it didn't change the time. Or maybe I'm not understanding what that issue was about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions