Support injecting multiple files into the executable

[mxschmitt]

What is the problem this feature will solve?

Projects which include multiple files or non-javascript files would benefit from it and would not have to bundle everything.

What is the feature you are proposing to solve the problem?

Support injecting multiple files, since right now its limited to inject a single file as per here.

What alternatives have you considered?

Bundling

---

Please add the single-executables label to it or suggest a better place to file this feature request.

[joyeecheung]

I am working on something along the lines. My current draft:

In the config file, we support a dictionary of assets, with ids that are later used in the API as keys, and path to the actual files as values:

{
  "assets": {
    "a.dat": "/path/to/a.dat",
    "b.txt": "/path/to/a.txt"
  }
}

At build time, we read these assets and add them to the preparation blob.

At runtime, users can get the assets back like this:

const { getAsset } = require('node:sea');
const a = getAsset('a.dat', 'buffer');  // Creates a copy
const b = getAsset('b.txt', 'utf-8');   // decode as an utf8 string, no copy

The API I have in mind is like this because I think it's probably not a good idea to return mutable data to users.

This can be a somewhat low level API as a first step - we can build VFS on top of this later (e.g. put some kind of archive as one of the assets)

Any thoughts?

@nodejs/single-executable-admins

[mxschmitt]

I think that would work for us, we would then create the VFS abstraction in the user land for now. Last-modified timestamp would be probably good to keep that stored inside as well. Not sure about rwx permissions / user permissions, makes probably not a lot of sense.

[tony-go]

Really great @joyeecheung 👏🏼

Looking forward to see the PR 🤩

[RaisinTen]

This sounds like a good first step for the VFS 👍

[joyeecheung]

I have a branch at https://github.com/joyeecheung/node/tree/sea-assets that contains an implementation of the proposal which is probably good to go, but I think I need to look into some of the SEA flaky tests issues before adding another SEA feature/test.

[robertsLando]

@joyeecheung thanks for all your efforts with nodejs SEA feature!

What about supporting also multiple js files? I mean for structured applications bundle them on a single file could be complex, is there any thougts about that?

[joyeecheung]

That should probably be layered on top of VFS, which can be layered on top of this lower-level thing.

[lukaslihotzki]

Exposing a native, dedicated API to read assets is a great idea. VFS patching can be done on top of this, possibly by including archives as assets. For this use case, it would make sense to (also) expose assets as Blob (not just as Buffer and String). Blobs can be sliced (and then be read asynchronously), which would allow reading specific files inside an archive on-demand. Also Blobs (and their slices) can be streamed, which would be nice when the archive is compressed.

Until such a native API is available, you can include (binary) files into a SEA by reading them with fs.readFileSync during snapshot creation (https://gist.github.com/lukaslihotzki/c60fef03d5a14d1c8723bc1251ede0ee).

[joyeecheung]

An interface that returns a Blob sounds like a good idea.