Audit build process for dependencies

[mhdawson]

Following the great work in #828 I think the next step is to audit any build steps related to dependencies and tools used in those build steps from a repeatability and supply chain perspective.

For dependencies that only contain a direct copy of the original source code for the dependencies and are completely built from that source during the Node.js make process there should be nothing to do other than to confirm they fall into that category.

For others, which have additional build steps we'll want to see what gets used/pulled in during those build steps, and see if we are comfortable that we can get the exact versions of the tools used in the future and that we have enough information that we complete the build steps if needed for some future Node.js release in a way that generates the same result. If we dynamically pull in tools we should also confirm we are confident that the source is trusted and not a potential source of a supply chain attack.

A good example of this is undici were there are WASM binaries that are build outside of the Node.js make step. These use dynamically installed packages in a docker container. Would we get the same version of those packages if we installed them a year from now? If not we might be introducing unexpected changes as we do maintenance releases.

Lets go through each of the dependencies and categorize them into

1. Original source - no issue, fully build in Node.js make nodejs
2. Build steps from original source - fully documented and repeatable
3. Build steps - needs work or unsure.

• acorn
• acorn-walk
• ada
• base64
• brotli
• cares
• cjs-module-lexer
• corepack
• googletest
• histogram
• icu-small
• llhttp
• minimatch
• nghttp2
• ngtcp2
• npm
• openssl
• postject
• simdutf
• undici
• uv
• vwasi
• v8
• zlib

Also note that we may need to not only look at the steps in the tools/dep_updaters directory but also look at the tools used to generate a new release of the dependency if what we copy over contains what would not be considered original source.

[mhdawson]

In terms of the dependencies which build WASM (undici, cjs-module-lexer, llhttp) some of them currently using a docker file to dynamically build a container and use apk to install packages. I have a feeling that what we get from the apk install may change over time.

We may be able to improve repeatability by building that docker image in advance and storing it in the GitHub container registry in the nodejs org. We could then use the same set of images across all projects were we build WASM binaries.

[marco-ippolito]

The main problem we have right now is that builds are not repeatable because we dont keep track of certain informations such as version of the tools we use to install, package locks, sources itself. (example: dependency github project is deleted)
I guess the first step is to identify how our dependencies are installed and what they contain.
We can check the following points for any dependency:

• Where we download the dependency from? (npm, github release, github source code, google source) this information should probably added in the maintaining dependency document
• Do we execute an install command? (npm install, make, configure) or we just copy files
• Is the dependency code equal to the source code of the project where we download from. Is it minified or changed during the release process?
• Are we aware of the tools required to install that dependency? for example if we do npm install do we keep track of npm version or lockfile?

Feel free to edit and add more question, improve etc...

[RafaelGSS]

I suggest using the same approach we used for automating dependencies. We could have a main track issue (possibly this one), and whenever we work on a specific version, we can create a separate issue to discuss it there.

[marco-ippolito]

Please have a look at this PR nodejs/node#49747

[marco-ippolito]

acorn-walk

• it's javascript
• It's in the acorn monorepo
• It goes through bundling with rollup pre release
• It is installed by downloading the tarball from npm registry (example: https://registry.npmjs.org/acorn-walk/-/acorn-walk-6.0.0.tgz) with npm pack and the content is copy pasted into /deps/acorn/acorn-walk

tools we use to install:

• npm: npm view acorn-walk dist-tags.latest to check latest upstream, npm pkg get version to get installed version, npm pack to download
• shasum
• perl
• tar

[marco-ippolito]

acorn

• it's javascript
• It's in the acorn monorepo
• It goes through bundling with rollup pre release
• It is installed by downloading the tarball from npm registry with npm pack
• we move the folder in deps/acorn/acorn
• we update /src/acorn_version.h file with the new version

tools we use to install:

• npm: npm view acorn dist-tags.latest to check latest upstream, npm pkg get version to get installed version, npm pack to download
• shasum
• perl
• tar

[github-actions]

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

[github-actions]

This issue has been inactive for 90 days. It will be closed in 14 days unless there is further activity or the stale label is taken off.

[mhdawson]

Work to improve the build process of deps that include WASM - #1236