Skip to content

Event Loop Utilization metrics is inflated when using a profiler / when a stream is backpressured #67110

Open

Description

@Hackatosh

Version

v24.13.1

Platform

Linux hbs-worker-express-66f54f58f9-znzfw 6.8.0-1070-azure #78-Ubuntu SMP Thu Sep 24 15:55:52 UTC 2026 x86_64 Linux

Subsystem

performance (eventLoopUtilization)

What steps will reproduce the bug?

I have observed inflated reported Event Loop Utilization in the following cases :

  • When using Pyroscope which itself uses @datadog/pprof. When enabling continuous profiling on a idle NodeJS API server, Event Loop Utilization goes from 10-15% to 50%. This limitation is documented by Datadog themselves.
    It is an inflated metrics as there is almost no increase in CPU consumption.

  • When using the V8 Profiler, I observe the same kind of behavior (reported ELU goes up, CPU stays the same). A reproduction is possible with this script :

idle.mjs :

import { performance } from 'node:perf_hooks';

const start = performance.eventLoopUtilization();
setTimeout(() => {
  const u = performance.eventLoopUtilization(start).utilization;
  console.log(`ELU over 2 s idle = ${(u * 100).toFixed(1)}%`);
}, 2000);

If you run node idle.mjs : ELU is almost 0%
If you run node --cpu-prof idle.mjs : ELU is over 96%

  • When there is a backpressured stream I also observe the same kind of behavior (reported ELU goes up, CPU stays the same). A reproduction is possible with this script :

elu-backpressure.mjs :

import net from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
import { performance } from 'node:perf_hooks';

if (!isMainThread) {
  // Worker thread used to fill the socket read by main thread
  net
    .createServer((sock) => {
      sock.write(Buffer.alloc(4 * 1024)); // > 1 KiB highWaterMark: reader stops
      parentPort.once('message', () => setTimeout(() => sock.write('x'), 900)); // 1 byte, 900 ms later
    })
    .listen(workerData, () => parentPort.postMessage('ready'));
} else {
  const bug = process.argv[2] !== 'nobug';
  const sockPath = path.join(os.tmpdir(), `elu-${process.pid}.sock`);
  const worker = new Worker(new URL(import.meta.url), { workerData: sockPath });

  worker.once('message', () => {
    const client = net.connect({ path: sockPath, readableHighWaterMark: 1024 });

    if (bug) {
      // start reading once, never consume: buffer fills, Node stops reading
      client.pause().read(0);
    } else {
      // data is consumed: Node keeps reading
      client.on('data', () => {});
    }

    setTimeout(() => {
      // let the setup settle, then measure 1 idle second
      const start = performance.eventLoopUtilization();
      worker.postMessage('go');
      setTimeout(() => {
        const elu = performance.eventLoopUtilization(start);
        console.log(`${bug ? 'bug  ' : 'nobug'} ELU over 1 s idle = ${(elu.utilization * 100).toFixed(1)}%`);
      }, 1000);
    }, 200);
  });
}

If you run node elu-backpressure.mjs, NodeJS interacts with a backpressured stream and shows 90% ELU
If you runnode elu-backpressure.mjs nobug, NodeJS interacts with a stream that is not backpressured and shows 0.1% ELU

How often does it reproduce? Is there a required condition?

My initial use case was to run Pyroscope continuous profiling on production. When enabling it I reproduce the Event Loop Utilization metrics bump from 10-15% to 50% every time.

What is the expected behavior? Why is that the expected behavior?

ELU increase should reflect the increase in CPU consumption

What do you see instead?

ELU reported by performance.eventLoopUtilization goes from 10-15% to 50% when enabling continuous profiling even if there is almost no CPU consumption increase.

Additional information

I opened an issue on libuv related to this problem : libuv/libuv#5373

There is also a pull request as this problem might be a bug in how event loop idle time is computed in libuv : libuv/libuv#5325. But it might seem this is actually a problem in how NodeJS (for the V8 profiler) and Datadog (for @datadog/pprof) use SIGPROF signals to run the profiling : libuv/libuv#5325 (comment)

This problem makes it impossible to use a continuous profiling tool while keeping a reliable Event Loop Utilization metrics 😬

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions