Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] npx resolving wrong binary when using a workspace #6765

Closed
2 tasks done
niklasholm opened this issue Sep 5, 2023 · 5 comments · Fixed by #6973
Closed
2 tasks done

[BUG] npx resolving wrong binary when using a workspace #6765

niklasholm opened this issue Sep 5, 2023 · 5 comments · Fixed by #6973
Assignees
Labels
Bug thing that needs fixing Priority 1 high priority issue Release 9.x work is associated with a specific npm 9 release

Comments

@niklasholm
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

This issue exists in the latest npm version

  • I am using the latest npm

Current Behavior

When using a multi-workspace monorepo, and the same package is present in both the root and a workspace, but with different versions, npx without a --package or --call argument will resolve the package installed in the root rather than the one in the workspace when using the --workspace option.

Expected Behavior

npx should resolve and execute the package installed in the specified workspace if present.

Steps To Reproduce

  1. Setup steps (output omitted)
$ npm init --yes
...
$ npm init -w app --yes
...
$ npm install --save-dev 'eslint@^8'
...
$ npm install -w app --save-dev 'eslint@^7'
...
  1. Check versions of installed packages without npx
$ ./node_modules/.bin/eslint --version
v8.48.0
$ ./app/node_modules/.bin/eslint --version
v7.32.0
  1. Run the same command with npx
$ npx --no -- eslint --version
v8.48.0   # Correct
$ npx --no -w app -- eslint --version
v8.48.0   # Incorrect!
$ npx --no -w app --package=eslint -- eslint --version
v7.32.0   # Correct
$ npx --no -w app -c 'eslint --version'
v7.32.0   # Correct

As shown above when using the --package and/or --call forms of the command it resolves the correct version of eslint but not with only positional arguments, even though they should be equivalent since the package name and binary name is the same,

Environment

  • npm: 10.0.0
  • Node.js: 18.17.1
  • OS Name: Windows 10 22H2
  • System Model Name: ThinkPad T480 20L5
  • npm config:
; "builtin" config from C:\Users\nikho\AppData\Roaming\npm\node_modules\npm\npmrc

prefix = "C:\\Users\\nikho\\AppData\\Roaming\\npm"

; node bin location = C:\Program Files\nodejs\node.exe
; node version = v18.17.1
; npm local prefix = C:\Users\nikho\npx-test
; npm version = 10.0.0
; cwd = C:\Users\nikho\npx-test
; HOME = C:\Users\nikho
; Run `npm config ls -l` to show all defaults.
@niklasholm niklasholm added Bug thing that needs fixing Needs Triage needs review for next steps Release 9.x work is associated with a specific npm 9 release labels Sep 5, 2023
@lukekarrys lukekarrys added Priority 1 high priority issue and removed Needs Triage needs review for next steps Release 10.x labels Sep 8, 2023
@lukekarrys
Copy link
Contributor

Just a note that I confirmed this bug on both npm 9 and 10.

@wraithgar wraithgar assigned wraithgar and siemhesda and unassigned wraithgar Sep 20, 2023
@siemhesda
Copy link
Contributor

The behavior you described does not seem to be a bug in npx, looks more like an intended design based on how npx resolves and executes packages.

When you provide the package name explicitly using the --package option, npx knows exactly which package to execute, and it doesn't rely on the local context or the package.json file to determine the package.

If you want to ensure that a specific version of a package is used consistently, you can always use the --package option as you did in your example:

$ npx --no -w app --package=eslint -- eslint --version

This way, you explicitly specify the package to use, and you have control over the version. This can help ensure consistent behavior, especially if you have multiple versions of a package installed in different locations.

I don't think it is a bug but rather a feature of npx to provide flexibility in resolving and executing packages. Using the --package option is a reliable way to ensure a specific version is used.

npx will try to execute the locally installed eslint package, and it will look at the bin field in the package.json file of the project to determine which binary to run. If there is no locally installed eslint package, it will fall back to using the globally installed eslint package.

However, when you run npx with --package, it explicitly specifies which package to use, and it doesn't rely on the locally installed package.

@niklasholm
Copy link
Author

niklasholm commented Oct 12, 2023

@siemhesda Your explanation does not make any sense when considering the documentation, which states:

To run a binary other than the named binary, specify one or more --package options, which will prevent npm from inferring the package from the first command argument.

Implying that when --package is omitted, the package name is inferred from the command name, which is in this case ("eslint") is identical to the package name, so determining the package name to be "eslint" should be trivial, and --package should make no difference, especially in regards to which locally installed version of "eslint" is used (workspace or workspace root).

I'm not sure the situation you're describing is relevant since you're talking about locally vs globally installed, whereas this issue is about locally installed in workspace vs workspace root.

If this is indeed intentional behavior, then the documentation should be updated to reflect this. I challenge you to explain this logic in such a way that it makes sense to the regular user, because I cannot. If I've missed something, please help me out and point to the relevant paragraph for me.

@wraithgar
Copy link
Member

#6973 fixes this and another bug found while working on this one. npm will now look in the workspace for both the bin entry and the installed dependencies.

@niklasholm
Copy link
Author

@wraithgar Thank you!

wraithgar added a commit that referenced this issue Nov 14, 2023
ide pushed a commit to expo/expo that referenced this issue Mar 26, 2024
…epos (#27779)

# Why

If you have a monorepo in which you have different versions of
typescript installed you may see that `expo-module build plugin` fails
because of unknown TS compiler options. I think it happens because wrong
version of `tsc` picked up by `npx`.

Related GitHub issues:
npm/cli#6765

# How

As far as I understand the root cause of this issue is that if we simply
run `npx tsc ...` npm does not know `tsc` binary belongs to the
`typescript` package and it will simply try to resolve it from the root
workspace's `node_modules/.bin` where it will find the wrong version of
`tsc` because of dependency hoisting. It should start looking up `tsc`
in the workspace's `node_modules/.bin` (from where we're executing
`expo-module build plugin`) but that is not the case for some reason. We
can only achieve this by giving a hint to `npx` that `tsc` belongs to
the package called `typescript`. In this case it will start the binary
lookup from the workspace's `node_modules/.bin` folder and use the
correct version to build our module plugin.

# Test Plan

1. Create a Yarn v2 Berry monorepo.
2. Create a module (`packages/cool-module`) by following the official
tutorial **with a config plugin**
(https://docs.expo.dev/modules/config-plugin-and-native-module-tutorial/)
3. Add `typescript@^4.5.5` to your workspace root as a dev dependency
(or add you could have multiple other workspaces that depend on
`typescript@^4.5.5`, e.g. `packages/my-api`)
4. At this point you should see that the 4.5.5 `tsc` is installed in
your root `node_modules` and 5.3.0 `tsc` is installed in your
`packages/cool-module/node_modules` folder.
5. If you try to run `cd packages/cool-module && yarn run build plugin`
you should receive TypeScript config errors since
`packages/cool-module/plugin/tsconfig.json` inherits configuration
options that are unknown to 4.5.5 `tsc`. (e.g. `"lib": ["es2023"]`). At
this point you know something is wrong since `expo-module-scripts`
depends on `typescript": "^5.1.3"` that should be aware of `es2023` and
`node16` module resolution logic.
6. Go to your module workspace folder (`cd packages/cool-module`) and
execute the same command that is executed by `expo-module build plugin`
under the hood: `yarn exec -- npx tsc --version`. It should give the
**mismatched** `4.5.5` version.
7. Stay in the module workspace folder (`packages/cool-module`) and
execute now the following command: `yarn exec -- npx
--package=typescript tsc --version`. It should give the **correct**
`5.3.0` version.

# Checklist

- [x] Documentation is up to date to reflect these changes (eg:
https://docs.expo.dev and README.md).
- [x] Conforms with the [Documentation Writing Style
Guide](https://github.com/expo/expo/blob/main/guides/Expo%20Documentation%20Writing%20Style%20Guide.md)
- [x] This diff will work correctly for `npx expo prebuild` & EAS Build
(eg: updated a module plugin).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing Priority 1 high priority issue Release 9.x work is associated with a specific npm 9 release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants