Windows CI is not working

[Trott]

At the time of this writing, https://ci.nodejs.org/job/node-test-binary-windows-native-suites/ has 36 consecutive failures. The last successful run was 34 hours ago.

I ran git-clean-windows and windows-update-reboot with force reboot in the hopes it would fix things, but it looks like it did not. (And I may have even made it worse? If so, sorry!) Prior to my trying it, https://ci.nodejs.org/view/All/job/windows-update-reboot/2147/ had been running for 4 days waiting for a host that was offline due to low disk space. I put that host back online hoping that a reboot would fix things.

Refs: https://openjs-foundation.slack.com/archives/C03BJP63CH0/p1668101438259619

[Trott]

It looks like @joaocgreis is on the case and things might be back to working. They ran git-clean-windows about an hour ago, although I'm sure they did other things too but that's the thing I noticed. Anyway, now test-rackspace-win2012r2_vs2019-x64-1 is 50+ minutes into a build whereas previously it had failed after 2.6 seconds on the same task.

[Trott]

Alas node-test-binary-windows-native-suites and node-test-binary-windows-js-suites are still failing fast.

[Trott]

@nodejs/build @nodejs/platform-windows I know today is a holiday in a lot of places, so I'm not sure if people are available to take a look at this, but just in case.... (And I think Joao may already be looking at it, but I'm not 100% certain. I inferred that from seeing a CI job they started.)

[Trott]

I think the problem might be on ubuntu2204-x64-1 which those hosts use as a git server.

Caveat: I have no idea what I'm doing.

[Trott]

I think the problem might be on ubuntu2204-x64-1 which those hosts use as a git server.

Caveat: I have no idea what I'm doing.

It does seem like the permanently failing jobs are trying to check out things like refs/heads/jenkins-node-test-commit-windows-fanned-0237cff7ff552f2db40f2cc16178eb6ecfc53670-bin-win-vs2019-x86 when there isn't something with vs2019-x86 at the end. There's vs2019-arm64 and vs2019. I wonder if it's supposed to be using the latter, but not sure how/when this issue would have been introduced.

[Trott]

Ooh, there are x86 repos on there but the most recent one was created on November 9, which would seem to correspond with when this problem started. So I guess I need to go find the job that creates these repos and see if I can figure out why it stopped creating -x86 branches?

[Trott]

@sxa It appears that you did a Jenkins configuration change right around this time, adding an entry for Windows ARM stuff. Is there any chance this is what caused this issue?

[Trott]

@sxa It appears that you did a Jenkins configuration change right around this time, adding an entry for Windows ARM stuff. Is there any chance this is what caused this issue?

This change in particular removes an vs2019-x86 reference. Maybe you meant to remove the vs2017-x86 reference instead?

[Trott]

I added back win-vs2019-x86 and am running a fresh CI job. 🤞 https://ci.nodejs.org/job/node-test-commit-windows-fanned/51619/

[Trott]

Putting it back seems to have done the trick. The tests are running on x86 where before, the machines would fail trying to use git (I guess to fetch the compiled executable or something similar to that).