test: reduce flakiness of test-runner-output.mjs - #52146
Merged
Merged
Conversation
This commit is similar to nodejs#51952. When the system is under load it is possible for these timeout tests to become flaky. We work around that by using a much longer setTimeout() in the test so that it is not racing against the test's timeout. But, we have to unref() such a large timeout. And, because test timeouts do not currently keep the event loop alive, we use a different setTimeout() for that purpose. Fixes: nodejs#52139 Refs: nodejs#52140
nodejs-github-bot
added
needs-ci
aduh95
approved these changes
Mar 18, 2024
aduh95
added
fast-track
Contributor
|
Fast-track has been requested by @aduh95. Please 👍 to approve. |
github-actions
Bot
removed
the
request-ci
Collaborator
Collaborator
Collaborator
MoLow
approved these changes
Mar 19, 2024
MoLow
added
the
commit-queue
nodejs-github-bot
removed
the
commit-queue
Collaborator
|
Landed in 978d5a2 |
marco-ippolito
pushed a commit
that referenced
this pull request
May 2, 2024
This commit is similar to #51952. When the system is under load it is possible for these timeout tests to become flaky. We work around that by using a much longer setTimeout() in the test so that it is not racing against the test's timeout. But, we have to unref() such a large timeout. And, because test timeouts do not currently keep the event loop alive, we use a different setTimeout() for that purpose. Fixes: #52139 Refs: #52140 PR-URL: #52146 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Merged
marco-ippolito
pushed a commit
that referenced
this pull request
May 3, 2024
This commit is similar to #51952. When the system is under load it is possible for these timeout tests to become flaky. We work around that by using a much longer setTimeout() in the test so that it is not racing against the test's timeout. But, we have to unref() such a large timeout. And, because test timeouts do not currently keep the event loop alive, we use a different setTimeout() for that purpose. Fixes: #52139 Refs: #52140 PR-URL: #52146 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
tnunamak
added a commit
to PDP-Connect/data-connect
that referenced
this pull request
Sep 3, 2026
…r (Node test-runner false positive) Root-caused the largest remaining failure cluster (55 of 62 failures at last measurement, spanning 5 files): a documented, known Node.js test-runner limitation, not a bug in this repo's production code. nodejs/node#51381 ("spurious 'Promise resolution is still pending' when a callback test fails") and its follow-up discussion in nodejs/node#52025 describe the exact mechanism: the test runner uses an unref()'d internal timer for its own timeout bookkeeping. When test code ALSO uses an unref()'d timer racing against a deliberately-still-pending promise -- a completely valid, common pattern (e.g. "assert that operation A times out while operation B is left deliberately in-flight, to prove cancelling A doesn't affect B") -- the runner can flag that still-pending promise as abandoned before the real race actually resolves, well before either the test's own assertions or its explicit timeout value would suggest. Confirmed directly: shortening one such test's internal deadline by 20x still failed at the same speed, ruling out "the wait is too long" as the cause. Node's own core test suite documents the accepted workaround for this exact interaction (see the discussion linked above and nodejs/node#52146): a separate, ref()'d "keep the event loop alive" timer for the test's duration, cleared in t.after(), independent of whatever unref()'d timer the code under test uses. Applied this to the 5 affected files, in each file's existing shared per-test setup helper (freshDb()/setup()) where one exists, so every test in the file gets the same protection rather than patching call sites one at a time. Verified per-file in isolation (node --import tsx --test test/<file>.test.ts), each now fully green: controller-cancel-run.test.ts 5/5 (was 1/5), controller-browser- surface-leases.test.ts 39/39 (was 28/39), controller-phantom-active-run.test.ts 18/18 + 3 legitimately-skipped Postgres tests (was all 21 failing), run-generation- fencing.test.ts 5/5 (was 1/5), source-declaration-trust.test.ts 19/19 (was 5/19, after also fixing a second instance of the same shape in the same file -- "declaration retrieval bounds DNS work by the configured deadline" -- found once the first fix unblocked the rest of the file's cascade). Full suite dropped from 62 to 16 failures with zero regressions (confirmed via a before/after diff of the exact failing-test-name set). The remaining 16 are unrelated, already-tracked items (data-connect#53, data-connectors#67, the deliberately-deferred Signal connector rollout, and 2 pre-existing scanner findings) -- EXCEPT a handful of tests in source-declaration-trust.test.ts that still show this same failure signature specifically under the full suite's concurrent-worker execution model, despite passing 100% when that file runs standalone. Not yet root-caused whether this is a concurrency-specific variant of the same Node limitation or a separate interaction; noted for follow-up rather than blocking this fix, which is unambiguously a large net improvement either way. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit is similar to #51952. When the system is under load it is possible for these timeout tests to become flaky. We work around that by using a much longer
setTimeout()in the test so that it is not racing against the test's timeout. But, we have tounref()such a large timeout. And, because test timeouts do not currently keep the event loop alive, we use a differentsetTimeout()for that purpose.Fixes: #52139
Refs: #52140