upload-sarif@v1 : master does not match /^refs\/(heads|pull|tags)\/.*$/.

[totogtr]

Hi,

I'm facing an issue uploading a SARIF file using upload-sarif@v1 on a self-hosted runner.

Here is my code (in a composite action) :

    - name: Scan image
      id: scan
      uses: sysdiglabs/scan-action@v3
      with:
       [...]

    - uses: github/codeql-action/upload-sarif@v1
      with:
        sarif_file: ${{ steps.scan.outputs.sarifReport }}

This works great when I push / PR to a branch called develop but an issue is happening when I apply the same composite action to master branch. It looks like for some reason the upload-sarif@v1 action doesn't call the API correctly ?

Here are the logs on develop (omitting env variables) :

Run github/codeql-action/upload-sarif@v1
  with:
    sarif_file: ./sarif.json
    checkout_path: /home/azureuser/actions-runner-2/runner/_work/my-app/my-app
    token: ***
    matrix: {
    "node-version": "16.3.0",
    "env": {
      "name": "development"
    }
  }
  env:
    [...]
Uploading results
  Processing sarif files: ["./sarif.json"]
  Uploading results
  Successfully uploaded results

I can see the uploaded results on the security tab of the github repo.

Here are the logs on master:

  Processing sarif files: ["./sarif.json"]
  Uploading results
  Error: Invalid request.
  
  master does not match /^refs\/(heads|pull|tags)\/.*$/.
  RequestError [HttpError]: Invalid request.
  
  master does not match /^refs\/(heads|pull|tags)\/.*$/.
      at /home/azureuser/actions-runner-2/runner/_work/_actions/github/codeql-action/v1/node_modules/@octokit/request/dist-node/index.js:66:23
      at processTicksAndRejections (internal/process/task_queues.js:93:5)
      at async Job.doExecute (/home/azureuser/actions-runner-2/runner/_work/_actions/github/codeql-action/v1/node_modules/bottleneck/light.js:405:18) {
    name: 'HttpError',
    status: 422,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',
      connection: 'close',
      'content-length': '141',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Thu, 28 Oct 2021 06:12:21 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'GitHub.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': '0805:5EAA:1CCB18D:34AE888:617A3F44',
      'x-ratelimit-limit': '500',
      'x-ratelimit-remaining': '490',
      'x-ratelimit-reset': '1635404509',
      'x-ratelimit-resource': 'code_scanning_upload',
      'x-ratelimit-used': '10',
      'x-xss-protection': '0'
    },
    request: {
      method: 'PUT',
      url: 'https://api.github.com/repos/MYORG/my-app/code-scanning/analysis',
      headers: {
        accept: 'application/vnd.github.v3+json',
        'user-agent': 'CodeQL-Action/1.0.20 octokit-core.js/3.1.2 Node.js/12.13.1 (linux; x64)',
        authorization: 'token [REDACTED]',
        'content-type': 'application/json; charset=utf-8'
      },

...

The API doc mention that ref is Required. The full Git reference, formatted as refs/heads/<branch name>,refs/pull/<number>/merge, or refs/pull/<number>/head. : https://docs.github.com/en/rest/reference/code-scanning#upload-an-analysis-as-sarif-data

Using this action we have no control other what the action sends to the API so it is not possible in our end to enforce the full git reference.

Is this a known issue ? Any reason why I am facing here in a master branch but not on other branches ?

[aeisenberg]

The ref that the upload action passes is grabbed from the GITHUB_REF environment variable (there's some extra checking for edge cases, but it's mostly just that). So, if the action is uploading an invalid ref it's most likely that the environment variable is not pointing to the correct thing.

You mentioned that you are using a composite action. It's unlikely, but it's possible that the environment variable is getting mangled there? Something like this would work:

Can you try something for me? Before the - uses: github/codeql-action/upload-sarif@v1 step, can you add a new step that outputs all of your environment variables? And then please add a comment with all the relevant variables.

      - name: Dump environment
        run: env

If my guess is right, then we should figure out why the variable is being mangled. But as a quick workaround, you could explicitly set the variable to prepend the /refs/heads before running the upload action.

[totogtr]

Thanks for your quick reply.

Indeed you are right, it seems that the GITHUB_REF is not populated correctly in some case with the composite action.
I've added the dump environment, here is what I get when I trigger my workflow (that include the composite action)

Workflow triggered manually on master branch :

Run env
[...]
_=/usr/bin/env
LANG=C.UTF-8
GITHUB_API_URL=https://api.github.com
INVOCATION_ID=d3a73578ae00455485a825122b0fd5b7
GITHUB_ACTION_PATH=/home/azureuser/actions-runner/_work/my-app/my-app/./.github/workflows/composite-actions/build-angular
GITHUB_RUN_ATTEMPT=1
RUNNER_TOOL_CACHE=/home/azureuser/actions-runner/_work/_tool
RUNNER_TRACKING_ID=github_70b2dd0a-5967-4c58-a9bc-2e098972648b
GITHUB_ACTIONS=true
CI=true
USER=azureuser
GITHUB_HEAD_REF=
JAVA_VERSION=1.8
GITHUB_ACTOR=totogtr
GITHUB_ACTION_REF=
GITHUB_ACTION=build-frontend-composite
***
HOME=/home/azureuser
GITHUB_ACTION_REPOSITORY=
RUNNER_TEMP=/home/azureuser/actions-runner/_work/_temp
GITHUB_RETENTION_DAYS=90
GITHUB_REF=refs/heads/master
GITHUB_SHA=27e7b763b118340b19087429c812202a1b41b590
GITHUB_RUN_ID=1398173530
GITHUB_SERVER_URL=https://github.com
GITHUB_EVENT_PATH=/home/azureuser/actions-runner/_work/_temp/_github_workflow/event.json
RUNNER_OS=Linux
GITHUB_BASE_REF=
SHELL=/bin/bash
RUNNER_NAME=gh-runner-001
SHLVL=2
GITHUB_EVENT_NAME=workflow_dispatch
LOGNAME=azureuser
GITHUB_RUN_NUMBER=37
GITHUB_WORKFLOW=Release Deploy on Staging
[...]
Run github/codeql-action/upload-sarif@v1
Uploading results

GITHUB_REF contains refs/head and the upload-sarif works fine.

However when the same workflow is triggered by a PR closed from develop to master, I get :

Run env
GITHUB_ACTIONS=true
CI=true
USER=azureuser
GITHUB_HEAD_REF=develop
JAVA_VERSION=1.8
GITHUB_ACTOR=totogtr
GITHUB_ACTION_REF=
GITHUB_ACTION=build-frontend-composite
***
HOME=/home/azureuser
GITHUB_ACTION_REPOSITORY=
RUNNER_TEMP=/home/azureuser/actions-runner/_work/_temp
GITHUB_ENV=/home/azureuser/actions-runner/_work/_temp/_runner_file_commands/set_env_55403f45-1513-4886-946f-2855d3a9f213
GITHUB_REF=master
GITHUB_SHA=5f59b912f8f81fab26fb27113db047d5669ff530
GITHUB_RUN_ID=1398237546
GITHUB_SERVER_URL=https://github.com
GITHUB_EVENT_PATH=/home/azureuser/actions-runner/_work/_temp/_github_workflow/event.json
GITHUB_GRAPHQL_URL=https://api.github.com/graphql
RUNNER_OS=Linux
GITHUB_BASE_REF=master
SHELL=/bin/bash
ACR_ADMIN_USER=crrefappweu001
GITHUB_PATH=/home/azureuser/actions-runner/_work/_temp/_runner_file_commands/add_path_55403f45-1513-4886-946f-2855d3a9f213
GITHUB_JOB=build-frontend
GITHUB_EVENT_NAME=pull_request
Run github/codeql-action/upload-sarif@v1
RequestError [HttpError]: Invalid request.

master does not match /^refs\/(heads|pull|tags)\/.*$/.

Here the GITHUB_REF is only master and then it fails.

So do you think "something" in the composite action change the value of the GITHUB_BASE_REF if the workflow using it has GITHUB_EVENT_NAME set as pull_request instead of workflow_dispatch

In my use-case, this workflow could be either triggered by a PR or manually. I'll add a step to check and explicity set the /refs/heads if it's missing.

[lenyfernandez]

Hi @totogtr and @aeisenberg I'm facing the same issue. Were you able to explicitly set the /refs/heads ? I could not seem to explicitly set GITHUB_REF to any value. Thanks!

[aeisenberg]

I haven't heard back to hear if the solution works, but you could add a step like this in your composite action before uploading:

     - name: Update GITHUB_REF
        run: |
              if [[ ! "$GITHUB_REF" =~ ^\/refs\/heads\/.*$ ]]; then
               echo "/refs/heads/$GITHUB_REF" >> "$GITHUB_ENV"
              fi
      - name: Check GITHUB_REF
         run: echo "$GITHUB_REF"

I haven't tried this out directly, but in theory, this should update the GITHUB_REF env variable if it doesn't already start with /refs/heads/.

EDIT: This will not work. GitHub Actions refuses to allow you to edit any env variable prefixed with GITHUB_. It is a security flaw.

[adityasharad]

@totogtr would you be able to share the full Actions workflow file, so that we can see the different events your workflow is running on? And also the full contents of your composite action if possible.

[totogtr]

@adityasharad Hi, sorry I cannot share the full action, some as client data in it.

I've just hit this issue again today (1st time since my last message !)

Is there anything in #889 that would help us fix it?

[aeisenberg]

Yes, the upload-sarif action now supports a ref and sha input. So, you can specify the values you want using the with property:

    - uses: github/codeql-action/upload-sarif@v1
      with:
        ref: '${{ ref }}'
        sha: '${{ sha }}'

You need to specify both of them together.