build: provide pre-built nodejs/node images or cache

[bnb]

Is your feature request related to a problem? Please describe.
Presently, making a small contribution to node.js core is particularly challenging. There are a number of contributing factors, but a primary one is simply having to build Node.js from scratch. For someone unfamiliar/new to Node.js, this can be particularly challenging compared to basically anything else in the JavaScript ecosystem.

Describe the solution you'd like
It would be a nice contributor experience enhancement to provide a pre-built image (containerd or Docker?) or some kind of cache that would help alleviate the pain of needing to build completely from scratch every single time.

One theoretical manifestation of how this could work:

• Nightly, build a Docker container.
• This Docker container includes all necessary tools / nice-to-haves for building Node.js.
• This Docker container builds Node.js.

Containers provide some nice benefits:

• We'd potentially be able to enhance many contributors' workflows.
• We'd enable a better experience through cloud-hosted developer environments like Codespaces that have the option to build from a container.
• We could include node-core-utils and potentially enhance it further with something like a contributing checklist to ease first-time-contributor or early-contributor burden, effectively automating what's codified in the PR guidance.

Or, we could do something in the vein of a cloud cache (in the vein of goma). While these require immense work, they are extremely useful and don't require you to run Docker. If there's more interest in this, I'm happy to see what resources I can pull from Microsoft (including potentially open sourcing some things) to enable it.

Either way, this is a build feature that seems to go in a positive direction that helps both new and existing contributors. Would love thoughts, if there are any.

Describe alternatives you've considered

Doing nothing: I mean this works but also retains an unnecessary bad experience.

[bnb]

Also, FWIW, I am happy to help make this happen however I can. I assume if we go down the route of images, the auto-generation will be the... harder part for me to meaningfully contribute to purely because I don't have access/am not familiar with the infrastructure we'd likely use to do that compute.

[targos]

The challenge I see with a Docker container is how to "connect" it to the user's GitHub credentials (so they can easily push to their fork after working on a change).

[bnb]

@targos yeah, that's a place I think the enhancement with a small CLI or other kind of helper would be beneficial.

[bnb]

I also think that's a much better problem to have than having to compile Node.js on your own computer 😅

[bnb]

I suppose I should ask, assuming we do images:

• are there any strong opinions on what needs to be included excluded outside of the most basic requirements?
• how would we build and publish nightly? do we want to Do It Ourselves or do we want to see if Actions will be "good enough"?

[mcollina]

I think we should really explore two options and see what is feasible:

1. have one docker image generated every night compiled with everything we need.

2. have a platform-specific (Linux/Intel, Mac/Intel, Mac/Apple, Win/Intel) archive created from the build/ folder that is donwloaded

A few stats:

• tar czf time: 30s
• tar xzf time: 5s
• .tar.gz size: 284MB

If we consider a 100MBit line, this bundle will download in around 20-30s.

[targos]

have one docker image generated every night compiled with everything we need.

I'm experimenting with this in https://github.com/targos/node-dev-docker

[targos]

how would we build and publish nightly? do we want to Do It Ourselves or do we want to see if Actions will be "good enough"?

GitHub actions might be enough, but I don't know where we would push the image. I would try with the GitHub container registry, but I don't know if it's designed to hold short-lived images (we don't want to keep all nightly builds in storage forever).

[targos]

have one docker image generated every night compiled with everything we need.

I'm experimenting with this in https://github.com/targos/node-dev-docker

After 41 minutes on my mac, the image is built. Its size is 2.86 GB

[mcollina]

After 41 minutes on my mac, the image is built. Its size is 2.86 GB

Compressed or uncompressed? Usually those are transferred in a compressed fashion.

[targos]

I guess uncompressed. It's the size reported in the Docker dashboard:

[richardlau]

You could always remove out. Although that would remove the compiled output, the ccache cache should still exist to speed up build times.

[targos]

I going to try again with --ninja because I don't like the output with the normal build and subsequent calls to make are not no-ops.

[mcollina]

You could always remove out. Although that would remove the compiled output, the ccache cache should still exist to speed up build times.

This might be really interesting.

[targos]

Still building, but FWIW the image is already 1.86 GB before compilation, so there's probably a lot of room for improvement (like removing the apt cache)