Frogbot PR scan does not work with yarn using private repo

[nerijusk]

Describe the bug

Frogbot seems to not pass token to yarn for private repo access when scanning pull requests. Here's the error:

Fetch step
  ➤ YN0041: │ @PRIVATE_ORG/PRIVATE_REPO@npm:1.1.3::__archiveUrl=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40PRIVATE_ORG%2FPRIVATE_REPO%2F1.1.3%2F097e2a3f34c9d1a8f4dce81ff234314d3a22dee5: Invalid authentication (as an anonymous user)

GitHub workflow used to launch frogbot PR scan:

name: Frogbot Scan for Pull Request
on:
  pull_request_target:
    types: [opened, synchronize]
permissions:
  pull-requests: write
  contents: read
jobs:
  invoke-frogbot-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Invoke Frogbot scan
        uses: jfrog/frogbot@v2
        env:
          JF_URL: 'https://PRIVATE_ARTIFACTORY/'
          JF_ACCESS_TOKEN: ${{ secrets.JFROG_TOKEN }}
          JF_GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
          JF_WATCHES: 'ospo-licenses-unblocking-critical-high,es-core-cve-unblocking'
          JF_INCLUDE_ALL_VULNERABILITIES: 'FALSE'
          JF_FAIL: 'TRUE'

.yarnrc.yml content:

compressionLevel: mixed
enableGlobalCache: false
nodeLinker: node-modules
npmScopes:
    PRIVATE_ORG:
        npmAuthToken: ${NPM_TOKEN-}
        npmRegistryServer: 'https://npm.pkg.github.com'

yarnPath: .yarn/releases/yarn-4.6.0.cjs

.frogbot/frogbot-config.yml content:

- params:
      git:
          repoName: PRIVATE_GH_REPO
          branches:
              - main
      scan:
          projects:
              - installCommand: yarn install
                workingDirs:
                    - '.'

However, if I check out frogbot repo and build the tool locally and then execute with additional environment variables for owner and repo names, PR number, it works then. It looks as something is lost when frobgbot is run through the GitHub action. I've reproduced successful local run in GitHub workflow. It's strange, but this works:

name: Frogbot Scan for Pull Request
on:
  pull_request_target:
    types: [opened, synchronize]
permissions:
  pull-requests: write
  contents: read
jobs:
  invoke-frogbot-scan:
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v4
        with:
          ref: ${{ github.event.pull_request.head.ref }}
      - name: yarn
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
        run: |
          yarn install
      - name: Invoke Frogbot scan
        env:
          JF_URL: 'https://PRIVATE_ARTIFACTORY/'
          JF_ACCESS_TOKEN: ${{ secrets.JFROG_TOKEN }}
          JF_GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
          JF_WATCHES: 'ospo-licenses-unblocking-critical-high,es-core-cve-unblocking'
          JF_INCLUDE_ALL_VULNERABILITIES: 'FALSE'
          JF_FAIL: 'TRUE'
          JF_GIT_PROVIDER: 'github'
          JF_GIT_OWNER: 'PRIVATE_ORG'
          JF_GIT_REPO: 'PRIVATE_GH_REPO'
          JF_GIT_PULL_REQUEST_ID: ${{ github.event.pull_request.number }}
        run: |
          GWD=$(mktemp -d)
          cd ${GWD}
          git clone https://github.com/jfrog/frogbot.git
          cd frogbot
          ./buildscripts/build.sh
          ./frogbot scan-pull-request

The strangest part is that without checkout and yarn install steps it gets the same auth error as using frogbot action. Probably something is set/populated in GitHub runner with those steps. Also, if I do checkout and yarn install before using the GitHub action, it too gets the auth error.

And, of course, it is inefficient every time checking out and building the tool. I'd rather use the GitHub action.

Current behavior

Frogbot PR scan fails when yarn tries to fetch packages from private repo.

Reproduction steps

No response

Expected behavior

No response

JFrog Frogbot version

v2 of GitHub action

Package manager info

yarn

Git provider

GitHub

JFrog Frogbot configuration yaml file

No response

Operating system type and version

Ubuntu 24.04

JFrog Xray version

No response

[nerijusk]

After a number of experiments we've found out that installCommand: yarn install in .frogbot/frogbot-config.yml file was the reason of the failures for our Frogbot pull request scans. Since we commit lock files (package-lock.json and yarn.lock) to the repository and they're up to date, we don't really need to run yarn install before the scan. Of course, this might differ for others. For us this works now:

- params:
      git:
          repoName: PRIVATE_GH_REPO
          branches:
              - main
      scan:
          projects:
              - workingDirs:
                    - '.'

Leaving the bug open for Frogbot maintainers to decide if they want to close or to fix it so it works with yarn install and private package repository.

[eranturgeman]

Hello @nerijusk and thank you for using Frogbot!
You are correct- there is no reason for the yarn install command to be executed and if we detect any lockfiles (in your case yarn.lock) the install command should not be triggered. If it does run, it means we did not find the lock file.

1. Can you please tell me where is the lock file located related to the root?
2. What is the auth method you use (username + password or Access token)?
3. if you are scanning the root, why do you specify it in the yml file? If you only need the definitions in the last frogbot-config.yml you sent I see no reason to use it since there are no special settings there (and the file is not mandatory). However, when specifying working dirs you miss our auto-detection mechanism (which is not triggered when working dirs are provided). This mechanism resolves the project's structure and finds all descriptors. If this mechanism is not used Frogbot assumes the lock file is directly under the root of the provided working dir. If this is not the case - this is why the 'install' command is being executed

[nerijusk]

Hello @nerijusk and thank you for using Frogbot! You are correct- there is no reason for the yarn install command to be executed and if we detect any lockfiles (in your case yarn.lock) the install command should not be triggered. If it does run, it means we did not find the lock file.

1. Can you please tell me where is the lock file located related to the root?

2. What is the auth method you use (username + password or Access token)?

3. if you are scanning the root, why do you specify it in the yml file? If you only need the definitions in the last frogbot-config.yml you sent I see no reason to use it since there are no special settings there (and the file is not mandatory). However, when specifying working dirs you miss our auto-detection mechanism (which is not triggered when working dirs are provided). This mechanism resolves the project's structure and finds all descriptors. If this mechanism is not used Frogbot assumes the lock file is directly under the root of the provided working dir. If this is not the case - this is why the 'install' command is being executed

1. The lock file is in the root of repository.
2. All auth uses tokens in our case.
3. You mean we can ditch the .frogbot/frogbot-config.yml file entirely?

[nerijusk]

I did not mention, but we use yarn workspaces. Just tried to remove Frogbot config file and it correctly assumed the top level is yarn, but incorrectly detected that subfolders use npm, whereas we only have top level yarn.lock file for all. So, even if the config file does list pretty much defaults, we still need it for the scan to work correctly.

Thanks for the questions - they've been helpful in understanding what our config should be.