[BUG] Directories and files named '*' are excluded from packages in npm 6.12.1 or newer

[maxwellgerber]

What

Directories and files named * are not present in the final artifact in npm 6.12.1 or newer.

May be related to #1048

Steps To Repro

A full reproduction is available at https://github.com/maxwellgerber/npm-bug-repro

The format of my project is as follows:

|-- index.js
|-- package.json
`-- lib/
    |-- other/
    |   `-- *
    `-- */
        `-- file.js

It appears that in versions of NPM > 6.12.0, the lib/other/* textfile
and the 'lib/*/file.js' files will not be added to the final artifact
when running npm publish. This is bad - since the files will be present
during CI, it is very easy to have a green build and release broken artifacts.

The contents of the final artifact are evaluated below using npm pack.

Adding entries manually to the files array in package.json does not
fix the issue.

  ...
  "files": ["lib/other/*"],
  ...

Environment

$ node -v
v12.16.0
mgerber@mgerber-ltm ~/work/npm-bug-repro
$ uname
Darwin

Behavior for NPM v6.12.0

mgerber@mgerber-ltm ~/work/npm-bug-repro
$ npm -v
6.12.0
mgerber@mgerber-ltm ~/work/npm-bug-repro
$ npm pack --dry-run
npm notice
npm notice 📦  [email protected]
npm notice === Tarball Contents ===
npm notice 25B   lib/other/*
npm notice 57B   lib/*/file.js
npm notice 61B   index.js
npm notice 273B  package.json
npm notice 2.7kB README.md
npm notice === Tarball Details ===
npm notice name:          npm-bug-repro
npm notice version:       1.0.0
npm notice filename:      npm-bug-repro-1.0.0.tgz
npm notice package size:  1.2 kB
npm notice unpacked size: 3.1 kB
npm notice shasum:        87c035e2eda0c570f202fab45bc2c8188f674290
npm notice integrity:     sha512-U2rzmNTh+E/zE[...]/Nu84fGo8oDog==
npm notice total files:   5
npm notice
npm-bug-repro-1.0.0.tgz

Behavior for NPM v6.12.1

mgerber@mgerber-ltm ~/work/npm-bug-repro
$ npm -v
6.12.1
mgerber@mgerber-ltm ~/work/npm-bug-repro
$ npm pack --dry-run
npm notice
npm notice 📦  [email protected]
npm notice === Tarball Contents ===
npm notice 61B   index.js
npm notice 273B  package.json
npm notice 1.1kB README.md
npm notice === Tarball Details ===
npm notice name:          npm-bug-repro
npm notice version:       1.0.0
npm notice filename:      npm-bug-repro-1.0.0.tgz
npm notice package size:  877 B
npm notice unpacked size: 1.5 kB
npm notice shasum:        fb794f492e66b9606c6c33b6e243021cbc4561df
npm notice integrity:     sha512-CwTMtMdn9u/HA[...]m2Ix2mp88A0Bw==
npm notice total files:   3
npm notice
npm-bug-repro-1.0.0.tgz

Who

• n/a

References

• n/a

[isaacs]

Files named * result in artifacts that are impossible to install on Windows, and cause issues on other platforms as well, iirc. (Docker, I believe? I don't recall exactly.) In rare cases, the presence of * in filenames can thwart other unix tools that expect it to be a glob, though of course that's much less likely to be an issue in your node_modules folder.

They were removed starting with this change in npm-packlist.

If you do want your tool to depend on * characters in filenames, and you don't care about Windows support, then you can of course create these files in your test setup or ship them a different way. You just can't publish them in an npm package, because it breaks downstream users.

Sorry for the inconvenience. I don't like it either. In a perfect world, Windows would have moved to a fully posix compliant file system long ago.

[maxwellgerber]

It would be fantastic for NPM to throw a loud error when creating the pack list, instead of silently dropping files. That would give package developers a clear message of what is and isn't allowed. It would also prevent CI releases from shipping broken packages.

Should I open this feature request in the CLI project or in npm-packlist directly?

Awesome, I see you've already mentioned it in npm/npm-packlist#32.
Is there a ticket I can track against?