Confusing CodeQL Error: Go files were found but not processed

[zliang-akamai]

Error: Go files were found but not processed

We are seeing a weird error complaining not all Go files were processed and the error also appears on the default CodeQL scan configuration.

We have some Go files with build tags for testing purposes, for example: https://github.com/linode/terraform-provider-linode/blob/dev/linode/stackscripts/framework_models_unit_test.go#L1, which are the files not included in the scan, but I think it should not trigger an error because generally go build won't build test files (correct me if I'm wrong on this).

Is there any way to get more detailed error message or ignore the error from scanning status reporting?

And I think the correct behavior is to ignore test files not being built in the action.

[aibaars]

That does indeed look a bit confusing. The error originates from extractor.go#L127-L136

Looking at the last lines of the Autobuild step I notice a No packages found. line:

 [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done extracting packages.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Installing dependencies using `go get -v ./...` in `tools`.
  [2024-09-27 04:23:23] [build-stderr] go: warning: "./..." matched no packages
  [2024-09-27 04:23:23] [build-stderr] go: warning: "./..." matched no packages
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Running extractor command '/opt/hostedtoolcache/CodeQL/2.19.0/x64/codeql/go/tools/linux64/go-extractor [./...]' from directory 'tools'.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Build flags: ''; patterns: './...'
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Running packages.Load.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done running packages.Load.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 No packages found.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Extracting universe scope.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done extracting universe scope.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Running go list to resolve package and module directories.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done running go list deps: resolved 0 packages.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done processing dependencies.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Starting to extract packages.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Done extracting packages.
  [2024-09-27 04:23:23] [build-stderr] 2024/09/27 04:23:23 Success: extraction succeeded for all 2 discovered project(s).

Strangely enough, the summary lines in the log right before Go files were found but not processed suggest that CodeQL scanned 386 out 545 files. You might want to look at the "status" page . It should have a download link for a CSV file with information about which files were analyzed.

CodeQL scanned 386 out of 545 Go files in this invocation. Check the status page for overall coverage information: https://github.com/linode/terraform-provider-linode/security/code-scanning/tools/CodeQL/status/
Analysis produced the following diagnostic information:
Go files were found but not processed (1 result)
    * [Specify a custom build command](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-the-codeql-workflow-for-compiled-languages) that includes one or more `go build` commands to build the `.go` files to be analyzed.

[mbg]

Hi @zliang-akamai

Is there any way to get more detailed error message or ignore the error from scanning status reporting?

@aibaars' suggestion for getting more information makes sense. You can download the CSV file to see which files were extracted and which ones weren't to see whether any that you care about weren't extracted.

And I think the correct behavior is to ignore test files not being built in the action.

We do not extract test files by default, which is indeed deliberate.

---

The error you are seeing is ultimately just to warn you that there are .go source files in your repository, but that some of them weren't extracted. That may be deliberate (as with test files) or not, but you should check whether you are happy with the set of files that were extracted.

[zliang-akamai]

Hi @mbg and @aibaars,
Thanks for the response! I inspected the file and found all the not extracted files are test files. See attachment below.

code-scanning-files-extracted.csv

I think it is working as expected (test files aren't extracted by default), but I am not sure why it's still reporting the error, which is showing up in the status page... and I would like to get rid of the error from the status page if it indeed worked as expected.

[mbg]

Hi again @zliang-akamai 👋🏻

Great to hear that all the files you were expecting to have been extracted were extracted, and thank you for checking!

We agree that it would be better if this error message did not show up on the tool status page in a case like this and we are working on some improvements in this area.