x/tools/go/packages: high CPU usage while processing cgo file

[RadhiFadlillah]

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN="/home/radhi/Go/bin"
GOCACHE="/home/radhi/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/radhi/Go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build763115998=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This issue is related with vscode-go#2598. While working on cgo project, both gocode and gopls will use huge amount of CPU when making autocomplete suggestion. It can be seen here for gocode and here for gopls.

After looking around, I found out the reason for this is because both gocode and gopls uses LoadSyntax mode, which make go/packages run go list with -compiled flag set to true, which lead to high CPU usage in cgo project.

What did you expect to see?

The cgo file can be parsed as light as the normal Go code.

What did you see instead?

The process uses huge amount of CPU.

[bcmills]

CC @matloob @ianthehat

[bcmills]

I'm going to assume for now that this is something that can be addressed within go/packages, but please let me and @jayconrod know if this is looking like a bug in go list -compiled instead.

[matloob]

When requesting source in go/packages, we always return the cgo-processed source code. That's where we chose to draw the abstraction boundary. So I don't think there's too much we can do from the go/packages side. We're going to have to run go list -compiled, and once go list runs, go/packages does minimal work to copy the outputs of go/packages into the Packages struct it returns in a Load call (since we don't need to do parsing/typechecking/etc).

The one strange thing is that it's my understanding that the LSP gets the package information only once (unless new packages are opened) so there shouldn't be consistent high cpu usage.

[stamblerre]

gopls will run go/packages whenever imports are changed, so it could be happening more frequently.

[segevfiner]

Probably related if not duplicate of #29427