node-test-commit-osx is broken

[joyeecheung]

The macOS cluster is currently broken after the upgrade nodejs/Release#1106

After discussions on the slack channel, I've disabled node-test-commit-osx from node-test-commit for now so we are now relying on Github actions for macOS coverage. More background in https://openjs-foundation.slack.com/archives/C03BJP63CH0/p1752761029233379 and https://openjs-foundation.slack.com/archives/C03BJP63CH0/p1752768087218549

It needs to be enabled again from node-test-commit (unchecking disabled from the advanced dropdown, see https://wiki.jenkins-ci.org/JENKINS/59510168.html) when the cluster is back online. @ryanaslett is looking into it. Opening an issue to track it on GitHub as well.

[ryanaslett]

Update on the OSX Cluster:

The Ventura (MacOS13) ARM based Image that our agents are based on was incompatible with the newer hosts due to a bug in apple's hypervisor framework.

We'll have to rebuild the Images to be based on an upgraded version of the Ventura image. I will not be able to start work on that until next week (Tuesday), as I've got a vacation planned that puts me fully AFK. We do have a process to reconstruct the images, however, some manual coercion has happened to those images that had not yet been migrated to the packer builds to reconstruct the images, and now those images are unable to launch and we do not currently have access to the command history that would inform the steps required to get those images fully reconstructed.

MacStadium has agreed to running those images in one of their test labs so we can get access to that history, and that will happen on a call with them next Wednesday.

So, Tuesday, I'll rebuild the images as best I can, Wed I'll get all the confirmation if theres any settings I've missed with the ccache and we should be able to restore builds then.

[targos]

How did the session go?

[ryanaslett]

It went well, I was able to get all the info I needed to reconstruct the existing ARM images.

I've ran a preliminary test to see if the image works as before, and Im running into an issue with tap2junit.

Im unsure if this is a tap2junit issue, or an underlying issue with the image:

https://ci.nodejs.org/job/node-test-commit-osx/nodes=osx13-arm64/65946/console

1. 00:33:52.394 not ok 4221 es-module/test-http-imports fails.
2. Tap2junit cant seem to handle the sub test structure to display that fail.

[ryanaslett]

Okay, I've fixed the image, and have a working test build: https://ci.nodejs.org/job/node-test-commit-osx/65951/

Testing should work as far as i can tell now. The tap2junit issue was a red herring testing an old pr that wasn't passing, and encountering an error with tap2junit in its handling of nested tests.

I can reenable the osx testing portion if there are no objections.

Note: the release image has not been rebuilt, so osx releases are still offline until tomorrow.

[aduh95]

It seems releases are still blocked on osx13-arm64-release-tar and osx13-release-pkg. Is there something we can do to address this? Should I postpone the releases scheduled for today?

[ryanaslett]

Sorry, I didnt see this comment until now. If you havent already postponed the release, It probably should be. Im still working on the codesiging portion of the image and its somewhat fragile to test (the feedback loop is long).

[ryanaslett]

Hooray. notarytool is finally happy with all of the keychain mess.

I would really like to strip out secrets from baked images and not rely on storing them on the image, and also accessing them from the image non-interactively on an OS that presumes a human user.

@aduh95 releases should work now.

[ryanaslett]

Both node-test-commit and iojs-release are working again with the newly re-created images. I'll close this now and re-focus on the new images.