flaky: parallel/test-dns

[jakecastelli]

Test

parallel/test-dns

Console output

not ok 233 parallel/test-dns
  ---
  duration_ms: 636.98900
  severity: fail
  exitcode: 1
  stack: |-
    (node:5488) [DEP0118] DeprecationWarning: The provided hostname "" is not a valid hostname, and is supported in the dns module solely for compatibility.
    (Use `node --trace-deprecation ...` to show where the warning was created)
    node:assert:126
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
    + actual - expected ... Lines skipped
    
      [
        {
          address: '1.2.3.4',
    +     ttl: 3333333332,
    -     ttl: 3333333333,
          type: 'A'
        },
        {
          address: '::42',
    +     ttl: 3333333332,
    -     ttl: 3333333333,
          type: 'AAAA'
    ...

nodejs/reliability#933

[joyeecheung]

It seems this started flaking some time around July 5 nodejs/reliability#911. cc @nodejs/dns any ideas about what might start breaking the test?

[targos]

Commits that were pushed between July 1 and July 5: https://github.com/nodejs/node/commits/main/?since=2024-07-01&until=2024-07-05

[joyeecheung]

It's difficult to grok the test, but it seems the answers were sent by a server that the test implements itself, so it should've been exactly the same and if not, there might be some integer precision issues going on?

[joyeecheung]

Another possibly related PR could be #53554 which happened a few weeks earlier than July 5

[joyeecheung]

Potentially related c-ares/c-ares#786 ?

[joyeecheung]

One potential way to find out is to open a PR that disables the cache and do a stress test CI with it...not sure if I'll get around to it but feel free to do it if someone has the time.

[richardlau]

I git bisected with

#!/usr/bin/env bash
cd /home/rlau/sandbox/github/node
./configure --verbose
make -j 88
tools/test.py --repeat=10000 -J parallel/test-dns

between e020dd8 and eb54c54 and bisect points to the c-ares update:

$ git bisect log
git bisect start
# status: waiting for both good and bad commits
# bad: [e020dd8610978f752ff014b3d5558a3fadee69c9] doc: correct peformance entry types
git bisect bad e020dd8610978f752ff014b3d5558a3fadee69c9
# status: waiting for good commit(s), bad commit known
# good: [eb54c54bf5f7c077ec094d27cc1d93cd318bde06] doc: recommend not using libuv node-api function
git bisect good eb54c54bf5f7c077ec094d27cc1d93cd318bde06
# bad: [96a614d63abeb8aa3b85f4ef82abeb7ae3d0819d] meta: make more bug-report information required
git bisect bad 96a614d63abeb8aa3b85f4ef82abeb7ae3d0819d
# bad: [33a6d1fe3a92d81df33024525fd3407744611afc] crypto: remove ERR_CRYPTO_SCRYPT_INVALID_PARAMETER
git bisect bad 33a6d1fe3a92d81df33024525fd3407744611afc
# bad: [c61ec3036c513931ecbba243f56f7600cfd77936] esm: remove unnecessary toNamespacedPath calls
git bisect bad c61ec3036c513931ecbba243f56f7600cfd77936
# bad: [b2916051943de836b3391d6f627b17c9392d03c5] doc: add esm example for os
git bisect bad b2916051943de836b3391d6f627b17c9392d03c5
# bad: [27f1306ee17b32db30ba9949d374482eb3b2c74c] meta: warnings bypass deprecation cycle
git bisect bad 27f1306ee17b32db30ba9949d374482eb3b2c74c
# bad: [eca806b54a2a43966ed79e4c7c74ee6f01ce8816] meta: prevent constant references to issues in versioning
git bisect bad eca806b54a2a43966ed79e4c7c74ee6f01ce8816
# bad: [73236d503a558984943ef1284586f9bf25464b13] esm: move hooks test with others
git bisect bad 73236d503a558984943ef1284586f9bf25464b13
# bad: [25719b11a0291b16399925b379945484244b80df] src: use `starts_with` in node_dotenv.cc
git bisect bad 25719b11a0291b16399925b379945484244b80df
# bad: [a0cb507ea67121392082795943a3dc00e5317b5d] deps: update c-ares to v1.31.0
git bisect bad a0cb507ea67121392082795943a3dc00e5317b5d
# first bad commit: [a0cb507ea67121392082795943a3dc00e5317b5d] deps: update c-ares to v1.31.0
$