CodeQL with custom build run autobuild in analyze step

[JGiola]

I'm running CodeQL against a golang project that use the 1.21.1 version.
I've noticed that in the latest iteration the custom build step seems to be ignored during the analyze step and the autobuild script is launched even if I've already build it.
I can't understand if I'm doing something wrong, but when I was using the default golang version the autobuild script was not running...

The problem for me is that the default make command that the autobuild script is launching is not building anything but is the command for running the linter and the test suites, so is waisting time downloading things and running tests that are not inherent to the code quality step.

You can see the script launched here. And the workflow for codeql is the following:

env:
  GO_VERSION: 1.21.1
  GORELEASER_VERSION: v1.21.2

jobs:
  codeql:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
    steps:
    - name: Checkout repository
      uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 # v4.1.0
      with:
        show-progress: false
    - name: Setup Golang
      uses: actions/setup-go@93397bea11091df50f3d7e59dc26a7711a8bcfbe # v4.1.0
      with:
        go-version: ${{ env.GO_VERSION }}
        cache: false
    - name: Setup Goreleaser
      uses: goreleaser/goreleaser-action@7ec5c2b0c6cdda6e8bbb49444bc797dd33d74dd8 # v5.0.0
      with:
        version: ${{ env.GORELEASER_VERSION }}
        install-only: true
    - name: Initialize CodeQL
      uses: github/codeql-action/init@ddccb873888234080b77e9bc2d4764d5ccaaccf9 # v2.21.9
      with:
        languages: go
    - name: Run Build
      run: make build
    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@ddccb873888234080b77e9bc2d4764d5ccaaccf9 # v2.21.9

[mbg]

Hi @JGiola! 👋🏻 Thank you for reporting this problem. There's a bug with tracing custom Go 1.21 builds on Linux that we are aware of and have a fix in progress that should get released soon.

In the meantime, you should be able to get your custom build to work by implementing the fix manually, e.g. by changing your "Run Build" step like this:

    - name: Run Build
      run: |
        mkdir -p ./.temp-go-fix/
        echo "#!/bin/bash\n\nexec $(which go) \"$@\"" > ./.temp-go-fix/go
        chmod +x ./.temp-go-fix/go
        export PATH=./.temp-go-fix:$PATH

        make build

Alternatively, you could temporarily run your CodeQL workflow on Windows or macOS, or downgrade to Go 1.20.

[JGiola]

Unfortunately, I cannot downgrade go, but if the fix is poised to be released soon I can wait or move to macOS in case. I was worried that I was doing something wrong 😅

Thank you for your response

[mbg]

Hi @JGiola 👋🏻

Just to let you know, we have just released the new version of the CodeQL Action which should fix this problem. If you have pinned v2 of the Action, then the new release should automatically get used next time you run your CodeQL workflow.

Let me know if this has fixed the problem for you!

[JGiola]

No, it not seems to have been fixed for me, you can see here https://github.com/mia-platform/miactl/actions/runs/6456636776/job/17526523110 with the 1.22.1 version of the action inside the analyze step an autobuild is triggered.

[mbg]

Thank you for giving that a shot and I am sorry it still doesn't work for you. We have done some testing and discussed this internally. Our new version of the Action does indeed fix the problem we were aware of, but your particular configuration has brought a different issue to our attention where tracing of custom builds is broken and our fix does not work.

The previous problem we addressed is that go now ships as a statically-linked binary on Linux. The CodeQL tracer works fine with dynamically-linked binaries out-of-the-box, but tracing statically-linked binaries currently requires extra work. Until we have better support for that, the workaround implemented in the Action wraps the go binary in a script so that we effectively have a dynamically-linked go that our tracer can observe being executed.

The problem in your case is that goreleaser is also a statically-linked binary and that it is what invokes go in your workflow. It turns out that this particular scenario of a statically-linked binary invoking our wrapper script which invokes the actual, statically-linked go binary is again not observable by our tracer.

We are looking into ways of fixing this as well as more permanent improvements to our ability to trace statically-linked binaries, but I can't promise that this will be a quick fix. Unsatisfactorily, your options are mostly as before until we have a proper fix:

• Run the workflow on macOS or Windows
• Downgrade to Go 1.20 (but I appreciate you have ruled this out)
• Not use goreleaser and instead invoke go directly from your Makefile
• A temporary workaround where replacing our wrapper script with a small wrapper program resolves the issue (let me know if you would be keen on trying this and I can provide details of what to do)

[JGiola]

For various reasons I can only try the first road to run the workflow on macOS instead of the linux worker.

It can be reasonable to open an issue on the actions repository to ask to add a configuration where we can provide a custom make command for the auto builder? Even if is not so "auto" if I have to say what command to launch instead of make without options...

[leungster]

I'm running into this same issue but on GHE 3.10.3 with codeql-action 2.20.3. I tried the workaround wrapper script but autobuild is still triggering and printing Attempting to automatically build go code which is building my entire repo.

mkdir -p ./.temp-go-fix/
echo "#!/bin/bash\n\nexec $(which go) \"\$@\"" > ./.temp-go-fix/go
chmod +x ./.temp-go-fix/go
export PATH=./.temp-go-fix:$PATH
        
go build ./some-path/

[smowton]

One possible cause is if go build ./some-path/ call doesn't actually lead to any go files getting compiled. Could you share the log/ subdirectory of the database after the go build command is run but before finalization so that I can check if that's happening?

[smowton]

Ah, or, if I try something similar locally I see cannot run executable found relative to current directory. I believe Go's packages.Load is looking up go in the path so it can run go list, but os.Exec chokes on a relative path. That's arguably a bug in packages#Load not making the path relative before trying to Exec it, but in general a relative dir on the PATH is fragile since processes might change their working directory and expect the path to continue to identify the same working set of programs. Therefore I suggest that instead of adding ./.temp-go-fix to your PATH, you add $(readlink -f ./.temp-go-fix/).

[leungster]

That seems to have fixed it. Thank you.