`npm init neon --lib` #1014

dherman · 2024-01-07T19:17:01Z

Add to create-neon the ability to create Neon libraries with sensible zero-config defaults set up for pre-builds, including fully functional CI/CD.

Follow-up issues:

Example:

% npm init neon -y --lib example-project

Wrote to /Users/dherman/neon-XAqoAt/example-project/package.json:

{
  "name": "example-project",
  "exports": {
    ".": {
      "import": {
        "types": "./lib/index.d.mts",
        "default": "./lib/index.mjs"
      },
      "require": {
        "types": "./lib/index.d.cts",
        "default": "./lib/index.cjs"
      }
    }
  },
  "types": "./lib/index.d.cts",
  "main": "./lib/index.cjs",
  "files": [
    "lib/**/*.?({c,m}){t,j}s"
  ],
  "scripts": {
    "test": "tsc && cargo test",
    "cargo-build": "tsc && cargo build --message-format=json > cargo.log",
    "cross-build": "tsc && cross build --message-format=json > cross.log",
    "postcargo-build": "neon dist < cargo.log",
    "postcross-build": "neon dist -m /target < cross.log",
    "debug": "npm run cargo-build --",
    "build": "npm run cargo-build -- --release",
    "cross": "npm run cross-build -- --release",
    "prepack": "tsc && neon update",
    "version": "neon bump --binaries platforms && git add .",
    "release": "gh workflow run release.yml -f dryrun=false -f version=patch",
    "dryrun": "gh workflow run publish.yml -f dryrun=true"
  },
  "neon": {
    "type": "library",
    "org": "@example-project",
    "platforms": {},
    "load": "./ts/load.cts"
  },
  "devDependencies": {
    "@neon-rs/cli": "^0.1.68",
    "@tsconfig/node18": "^18.2.2",
    "@types/node": "^20.11.16",
    "typescript": "^5.3.3"
  },
  "dependencies": {
    "@neon-rs/load": "^0.1.68"
  },
  "version": "1.0.0",
  "keywords": [],
  "author": "",
  "license": "MIT",
  "description": ""
}


✨ Created Neon project `example-project`. Happy 🦀 hacking! ✨
% ls example-project
Cargo.toml	README.md	package.json	platforms	src		ts		tsconfig.json

dherman · 2024-03-27T04:44:29Z

pkgs/create-neon/dev/expect.ts

@@ -11,7 +11,7 @@ function readChunks(input: Readable): Readable {
  });
  input.on("close", () => {
    output.write(decoder.end());
-    output.destroy();
+    output.end();


The .destroy() call happened to work in the past but wasn't the right method to use, and it broke in more recent versions of Node.

…mport the JSON data with static types

…eleases

- generates necessary boilerplate to be a portable Node library implemented in Rust - uses TS by default (can be disabled) - generates GitHub Actions boilerplate by default (can be disabled) to make publishing push-button

- README improveements * copy edits * coalesce directory layout docs into a table - build.yml takes ref instead of branch - setup action shares common setup routines - scripts contain a `"release"` script instead of `"tag"`, which simply dispatches release.yml

…emplate references packageSpec not package

dherman · 2024-03-28T04:24:21Z

An example repo generated by this PR: https://github.com/dherman/create-neon-manual-test-project

kjvalencik · 2024-03-28T13:00:38Z

Thanks so much for the example repo! That's really helpful for the review. ❤️

pkgs/create-neon/data/templates/.gitignore.hbs

pkgs/create-neon/data/templates/.npmignore.hbs

pkgs/create-neon/data/templates/manifest/scripts.json.hbs

pkgs/create-neon/data/templates/README.md.hbs

pkgs/create-neon/data/templates/ts/index.mts.hbs

pkgs/create-neon/data/templates/ci/github/comments.yml.hbs

pkgs/create-neon/data/templates/ci/github/release.yml.hbs

pkgs/create-neon/data/templates/ci/github/setup.yml.hbs

pkgs/create-neon/data/templates/ci/github/.env.hbs

fix typo in build.yml description Co-authored-by: K.J. Valencik <[email protected]>

…libraries

…ries

- no need for `fetch-depth: 0` when checking out git - fix GH_TOKEN for `gh` cli to use the built-in ${{ github.token }} - add `--silent` to `npm pack` to work around npm/cli#7354

pkgs/create-neon/data/templates/ts/index.cts.hbs

- use a wrapper type instead of a string subtype as the example wrapper logic, for clarity - use `declare module` for a better suggested style of interacting with the addon (to avoid the any type)

- cfg matrix should have .script: string not .cross: boolean - prepack should include `tsc &&` for TS projects

kjvalencik · 2024-04-22T16:32:11Z

pkgs/create-neon/README.md

+
+### Creating a Portable Library
+
+To create a portable npm library with pre-built binaries:


Nit, it might not be obvious to a potential user what "portable" means.

Great point--I added some explanatory text.

kjvalencik · 2024-04-22T16:41:06Z

pkgs/create-neon/data/versions.json

+    "neonPublish": "v0.4.1",
+    "dotenv": "v1",
+    "ghRelease": "v1",
+    "releaseDownloader": "v1.7"


Does GitHub offer any protections on these tags changing? They didn't used to, but I'm not sure if they added it.

If not, can we use explicit SHA on unofficial actions?

I divided the versions.json metadata into two categories, one for verified and one for unverified actions:

"actions": { "verified": { "checkout": "v3", "githubScript": "v7", "setupNode": "v3", "setupRust": "v1", "cargoInstall": "v2", "neonBuild": "v0.9", "neonPublish": "v0.4.1" }, "unverified": { "dotenv": { "tag": "v1.1.2", "sha": "d1cd55661714e830a6e26f608f81d36e23424fed" }, "ghRelease": { "tag": "v2.0.4", "sha": "9d7c94cfd0a1f3ed45544c887983e9fa900f0564" }, "releaseDownloader": { "tag": "v1.10", "sha": "c39a3b234af58f0cf85888573d361fb6fa281534" } } }

And I changed the template to generate an explicit SHA reference with a comment noting the version tag, e.g.:

- name: Set Environment Variables uses: falti/dotenv-action@d1cd55661714e830a6e26f608f81d36e23424fed # v1.1.2

…ny unverified providers

kjvalencik

This is fantastic! I'm super excited. 🎉

dherman changed the title ~~npm init neon --lib~~ [WIP] npm init neon --lib Jan 7, 2024

dherman commented Mar 27, 2024

View reviewed changes

dherman changed the title ~~[WIP] npm init neon --lib~~ npm init neon --lib Mar 27, 2024

David Herman and others added 13 commits March 27, 2024 09:58

Use tsconfig/node18 as base TS config

2b6476b

generate more modern ES, which unfortunately means we can't as easy i…

6633fb5

…mport the JSON data with static types

consolidate Versions type handling

f3be15e

get the tests passing again with modern TS and modern node versions

d79a4a6

single quotes for imports

aa6ce0f

simplify unnecessary type test that I added during debugging

bdf24a9

add top-level comment explaining the versions.ts wrapper module

827016b

WIP

7804e07

rename "release" to "remote-publish" to avoid ambiguity with GitHub R…

23585f1

…eleases

Initial version of npm init neon --lib

3555443

- generates necessary boilerplate to be a portable Node library implemented in Rust - uses TS by default (can be disabled) - generates GitHub Actions boilerplate by default (can be disabled) to make publishing push-button

move hbs delegate into GitHub CI plugin

7460c7c

update versions

00394f8

dherman force-pushed the create-neon-lib branch from 97bf5c1 to 00394f8 Compare March 27, 2024 17:02

npm run prettier

88e8bb7

dherman marked this pull request as ready for review March 27, 2024 17:27

dherman added 4 commits March 27, 2024 19:03

bugfix: PackageSpec needs version (default is "0.1.0") and manifest t…

74fe46a

…emplate references packageSpec not package

add a newline to the manifest

2a68374

diagnostics

e54a1de

build fix

4b301e3

kjvalencik reviewed Apr 1, 2024

View reviewed changes

dherman and others added 4 commits April 7, 2024 12:58

export * from './index.cts'

d995950

better ways to extract node and rust versions

b2e0535

bugfix: steps.build.outputs.filename => steps.pack.outputs.filename

86cfc2f

Update pkgs/create-neon/data/templates/ci/github/build.yml.hbs

4741f2a

fix typo in build.yml description Co-authored-by: K.J. Valencik <[email protected]>

dherman added 12 commits April 14, 2024 13:16

complete the node v20 changes

24e8898

generate the right README details for the more nested repo layout of …

a36e170

…libraries

fix the conditional logic for the directory layout in README

66148cf

create-neon now generates the nested directory structure for TS libra…

d27655d

…ries

command-line option documentation in the main README

c8b804e

tee all the GITHUB_OUTPUT for simple diagnostics

4bbafb2

no default since it's required

4ec45b3

a few more GHA details:

eb6ac15

- no need for `fetch-depth: 0` when checking out git - fix GH_TOKEN for `gh` cli to use the built-in ${{ github.token }} - add `--silent` to `npm pack` to work around npm/cli#7354

replace the magic comments with a magic label

d3aa32c

remove comments.yml.hbs from templates

8de9b14

comment copy-edits

1150668

eliminate two more magic feathers

36a002f

kjvalencik reviewed Apr 15, 2024

View reviewed changes

pkgs/create-neon/data/templates/ts/index.cts.hbs Outdated Show resolved Hide resolved

dherman added 9 commits April 15, 2024 09:54

slight improvements to the generated index.cts boilerplate:

e6eef73

- use a wrapper type instead of a string subtype as the example wrapper logic, for clarity - use `declare module` for a better suggested style of interacting with the addon (to avoid the any type)

different hello world for libraries vs simple projects

970aff8

add some explanatory comments to the generated code

8830100

oops, copy-pasta bug

a7168c0

missed one {{#$}}

e2b3792

id: tag

8c59954

bugfixes:

8d68793

- cfg matrix should have .script: string not .cross: boolean - prepack should include `tsc &&` for TS projects

more detailed instructions for setting up npm tokens

a24ab89

bump @neon-rs/manifest so saving manifests includes a final newline

b2c69e0

kjvalencik requested changes Apr 22, 2024

View reviewed changes

dherman added 3 commits May 5, 2024 22:00

use the new more ergonomic CLI syntax

4091edf

harden the use of third-party GH actions by using explicit SHAs for a…

895361e

…ny unverified providers

explain what "portable library" means in the README

4ecd9b1

kjvalencik approved these changes May 6, 2024

View reviewed changes

fix failing test for --yes

fd212c1

dherman merged commit 62440e2 into main May 6, 2024
9 checks passed

dherman deleted the create-neon-lib branch May 6, 2024 18:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`npm init neon --lib` #1014

`npm init neon --lib` #1014

dherman commented Jan 7, 2024 •

edited

Loading

dherman Mar 27, 2024

dherman commented Mar 28, 2024

kjvalencik commented Mar 28, 2024

kjvalencik Apr 22, 2024

dherman May 6, 2024

kjvalencik Apr 22, 2024

dherman May 6, 2024 •

edited

Loading

kjvalencik left a comment


		### Creating a Portable Library

		To create a portable npm library with pre-built binaries:

npm init neon --lib #1014

npm init neon --lib #1014

Conversation

dherman commented Jan 7, 2024 • edited Loading

dherman Mar 27, 2024

Choose a reason for hiding this comment

dherman commented Mar 28, 2024

kjvalencik commented Mar 28, 2024

kjvalencik Apr 22, 2024

Choose a reason for hiding this comment

dherman May 6, 2024

Choose a reason for hiding this comment

kjvalencik Apr 22, 2024

Choose a reason for hiding this comment

dherman May 6, 2024 • edited Loading

Choose a reason for hiding this comment

kjvalencik left a comment

Choose a reason for hiding this comment

`npm init neon --lib` #1014

`npm init neon --lib` #1014

dherman commented Jan 7, 2024 •

edited

Loading

dherman May 6, 2024 •

edited

Loading