Changes to ubuntu1804 sharedlibs containers?

[BethGriggs]

Ref: nodejs/node#30639 (comment)

I am seeing failures on ubuntu1804_sharedlibs_debug_x64 for current and previous v10.x releases. I'm curious if there have been any changes to these containers or host systems? Or perhaps a change to the Jenkins job configuration?

Containers I have seen the failures on:

• test-joyent-ubuntu1804_sharedlibs_container-x64-4
• test-joyent-ubuntu1804_sharedlibs_container-x64-5
• test-digitalocean-ubuntu1804_sharedlibs_container-x64-7

Potentially e915204? but I guess that wouldn't cover the Digital Ocean failures.

If possible, could I please be added to the "email when a job fails" for the node-daily-vN.x-staging branches? I can try and keep those green between releases/report issues accordingly.

[sam-github]

Emails to what mailing address? @BethGriggs you don't have config auth for jenkins?

[BethGriggs]

IBM email (listed in the Node.js README) please, and nope - I don't believe I have config access to any jobs in Node.js CI

[sam-github]

I added post-build email actions for node-daily-v8.x-staging and v10.x

[BethGriggs]

The following two CI runs built off of the same commit on the same day (26th November). The first one passes, the second fails. Since the first failing run (16252), I've not managed to get a passing CI for v10.x - it doesn't seem to be intermittent.

https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/16230/ ✅ - test-softlayer-ubuntu1804_sharedlibs_container-x64-2
https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/16252/ ❌ - test-joyent-ubuntu1804_sharedlibs_container-x64-4

I don't think the issue is with specific containers either, as I got subsequent failures on test-softlayer-ubuntu1804_sharedlibs_container-x64-2 on the same commit as above - https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu1804_sharedlibs_debug_x64/16452/

Historic release tags such as v10.15.0, v10.17.0 are also seeing the failures.

ping @nodejs/build, any ideas what could be up here?

[richardlau]

re. the sharedlib builds, I changed the treatment of flaky tests from skip to dontcare last week (nodejs/node#30629 (comment)) for the debug variant. I've changed it back to skip so @BethGriggs can run a test build for 10.x.

[BethGriggs]

Latest CI run with the swap back to skip is passing - https://ci.nodejs.org/job/node-test-commit-linux-containered/16463/

[richardlau]

Doing some digging and it looks like nodejs/node#25868 isn't on 10.x-staging and looks like it might explain why there are large numbers of "failures" seen on 10.x-staging with --flaky-tests=dontcare which aren't seen on master (i.e. the failures are tests marked as SLOW but the test runner treated the tests passing as a failure).

I've attached the lts-watch-v10.x label to nodejs/node#25868. If we want to run the flaky tests in debug mode on the CI we'll need nodejs/node#25868 to land on the release lines first (technically on 8.x too).

[richardlau]

With 8.x now End-of-Life and nodejs/node#25868 now on the 10.x branches we can probably reenable running the flaky tests in the debug builds (i.e. --flaky-tests=dontcare).

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.