fix: workspace incorrectly resolves node_modules path during script execution on windows (fixes #4564)

[nonara]

Background Summary

The issue arises from the way Yarn handles symlink paths for executable files in node_modules.

When executables like rimraf.cmd are run from symlinked directories (like those under /node_modules/@pkg/a/node_modules/.bin), the paths resolve incorrectly relative to the repository root.

This problem is caused because Yarn adds the symlink path instead of the real, resolved path to the PATH environment variable. As a result, relative paths calculated in scripts point incorrectly, leading to failure in locating the correct files (e.g., prisma\build\index.js).

More Detail: #4564 (comment)

Severity & Notes

This issue has been a pervasive one, having been first reported over 7 years ago. I've run into it multiple times and the severity is such that it entirely cripples complex projects.

It appears to also be present in Yarn 3.

I have a fix submitted, but I'm also submitting multiple solution proposals. I'm happy to discuss these with the team and do whatever is most suitable.

I noticed that some patches are still being accepted for Yarn 1, which are not strictly security related, so I'm hopeful we can get this through as well.

With that said, I am also willing to address it in Yarn 3+ if need-be to get this merged, however, I'd love to get this merged in Yarn 1, as many of us still use it.

Issue

• Yarn workspace incorrectly resolves node_modules path during script execution of workspace dependencies #4564

Solutions

Solution 1 : Modify execute-lifecycle-script.js

This would be the most ideal IMO, assuming there are no side-effects.

A simple change is to edit makeEnv to the following:

  // Add node_modules .bin folders to the PATH
  for (const registryFolder of config.registryFolders) {
    const binFolder = path.join(registryFolder, '.bin');
    if (config.workspacesEnabled && config.workspaceRootFolder) {
      pathParts.unshift(path.join(config.workspaceRootFolder, binFolder));
    }
    pathParts.unshift(path.join(config.linkFolder, binFolder));

    // We resolve the path here before adding to PATH
    const realBinFolder = await fs.realpath(path.join(cwd, binFolder));
    pathParts.unshift(realBinFolder);
  }

Solution 2 : Modify cmd files

We can resolve symlinks in the generated command files.

Here is an example:

@echo off
SETLOCAL
FOR /f "tokens=*" %%i IN ('fsutil reparsepoint query "%~dp0" ^| find "Print Name"') DO SET RealPath=%%i
SET RealPath=%RealPath:*Print Name: =%

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe"  "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs" %*
  echo "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs"
) ELSE (
  node  "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs" %*
  echo "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs"
)
ENDLOCAL

We'd want to ensure that this works with Win XP and up, but I believe it will.

If not, we can also do something more simple, like using cd to the path.

Questions for the Yarn Team

Questions I'd have for the Yarn team are:

1. Solution 1
   a. Would it be better to just use realpath at a higher level?
   b. Is there any potential downside to modifying it here?

Failing tests

Looks like I'm getting some failures:

    error Couldn't find package "is-core-module" on the "npm" registry.

This seems to be happening locally and on the latest master (before my changes) as well. Not sure what's going on here, as that package exists on npmjs registry, but if anyone has any guidance, I'm happy to make adjustments to ensure all passes.

[nonara]

Hi @BYK! I saw you had this issue assigned before.

If you're not free, would you happen to know who would be best to talk with? I'm happy to do whatever is necessary to get this pushed through.

I'm assuming it should be fairly straightforward in that we know that output .cmd files use relative paths and therefore using non-realpath would cause failure. I mentioned above, but this does seem to also affect v3.

I can't imagine a case where using the symlink path for the output env PATH would be more useful, but I wanted to check with the team first.

Appreciate you taking the time to read and point me in the right direction!

[nonara]

We could narrow this by checking to see if os is windows first and/or stat to see if it's a symlink.

[nonara]

Notes

NPX

Looks like npx is still failing in the same way. I'm assuming they're using the same logic and may need a PR as well.

Alternatively, we can just go with the other route and make the cmd file resolve the real path, but I'd assume upstream fixes are best. Will wait on hearing back from the Yarn team.

Commands

Looks like we need to also cover commands like yarn prisma db pull where prisma is a known command.

Presumably, this is in lib/cli/commands/run.js

I will hold on working further until I get a response from the team on what approach they prefer

[nonara]

FYI team —

Here is a workaround I created with tests that handles this issue by transforming the generated .cmd files. It uses powershell in a much simpler alteration of the resulting file:

https://github.com/nonara/yarn-fix-bin-cmds

Final cmd file looks something like this:

@echo off
SETLOCAL
FOR /F "delims=" %%I IN ('powershell -Command "(Get-Item -Path '%~dp0').Target"') DO SET "RealPath=%%I"
IF "%RealPath%"=="" SET "RealPath=%~dp0"

@IF EXIST "%RealPath%\node.exe" (
  "%RealPath%\node.exe"  "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs" %*
) ELSE (
  node  "%RealPath%\..\..\..\node_modules\rimraf\dist\esm\bin.mjs" %*
)
ENDLOCAL

If this approach makes sense, I can revise this PR to do similar. FWIW, I believe this method should be compatible with Windows 7+. If need be, for compatibility, we can add a failsafe to make it execute using %~dp0 as a fallback in the absence of powershell