Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go mod: migrating from dep to go modules #949

Merged
merged 1 commit into from
Mar 21, 2019
Merged

go mod: migrating from dep to go modules #949

merged 1 commit into from
Mar 21, 2019

Conversation

vaamarnath
Copy link
Contributor

  • Dependencies are vendored before building using go mod vendor.
  • During go build and go test, the vendored dependencies are
    used by setting GOFLAGS=-mod=vendor.

Fixes #598

Signed-off-by: Amarnath [email protected]

@vaamarnath
go mod: migrating from dep to go modules
dbe534e 
- Dependencies are vendored before building using `go mod vendor`.
- During `go build` and `go test`, the vendored dependencies are
  used by setting `GOFLAGS=-mod=vendor`.

Signed-off-by: Amarnath <[email protected]>
@vaamarnath
Copy link
Contributor Author

staticcheck is running out of memory after this change. Locally, when I run staticcheck, this is what I see

staticcheck \
        -checks all,-ST1003 \
        github.com/heptio/contour/cmd/contour github.com/heptio/contour/internal/contour github.com/heptio/contour/internal/dag github.com/heptio/contour/internal/debug github.com/heptio/contour/internal/e2e github.com/heptio/contour/internal/envoy github.com/heptio/contour/internal/grpc github.com/heptio/contour/internal/httpsvc github.com/heptio/contour/internal/k8s github.com/heptio/contour/internal/metrics

fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xa1d6b6, 0x16)
        /usr/local/go/src/runtime/panic.go:617 +0x72
runtime.sysMap(0xc390000000, 0xa4000000, 0xed84b8)
        /usr/local/go/src/runtime/mem_linux.go:170 +0xc7
runtime.(*mheap).sysAlloc(0xec0140, 0xa2894000, 0xec0150, 0x5144a)
        /usr/local/go/src/runtime/malloc.go:633 +0x1cd
runtime.(*mheap).grow(0xec0140, 0x5144a, 0x0)
        /usr/local/go/src/runtime/mheap.go:1232 +0x42
runtime.(*mheap).allocSpanLocked(0xec0140, 0x5144a, 0xed84c8, 0x0)
        /usr/local/go/src/runtime/mheap.go:1150 +0x3a7
runtime.(*mheap).alloc_m(0xec0140, 0x5144a, 0x100, 0x0)
        /usr/local/go/src/runtime/mheap.go:977 +0xc2
runtime.(*mheap).alloc.func1()
        /usr/local/go/src/runtime/mheap.go:1048 +0x4c
runtime.(*mheap).alloc(0xec0140, 0x5144a, 0x7f0f45010100, 0x428180)
        /usr/local/go/src/runtime/mheap.go:1047 +0x8a
runtime.largeAlloc(0xa2894000, 0x450001, 0xc053004000)
        /usr/local/go/src/runtime/malloc.go:1055 +0x99
runtime.mallocgc.func1()
        /usr/local/go/src/runtime/malloc.go:950 +0x46
runtime.systemstack(0x0)
        /usr/local/go/src/runtime/asm_amd64.s:351 +0x66
runtime.mstart()
        /usr/local/go/src/runtime/proc.go:1153

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc0d902f648 sp=0xc0d902f640 pc=0x458bc0
runtime.mallocgc(0xa2894000, 0x9d6360, 0xc0de924001, 0x45c329a0)
        /usr/local/go/src/runtime/malloc.go:949 +0x872 fp=0xc0d902f6e8 sp=0xc0d902f648 pc=0x40daf2
runtime.growslice(0x9d6360, 0xc1277d0000, 0xabb904, 0xabb93b, 0x190167a, 0xabb904, 0xb37cec, 0xabb904)
        /usr/local/go/src/runtime/slice.go:181 +0x1e6 fp=0xc0d902f750 sp=0xc0d902f6e8 pc=0x4444d6
honnef.co/go/tools/lint/lintutil.Lint(0xc0000e8720, 0x4, 0x6, 0xc0000201d0, 0xa, 0xa, 0xc0d902fe40, 0x97f2a0, 0x94a820, 0xf00000001, ...)
        /home/amarnath/go/pkg/mod/honnef.co/go/[email protected]/lint/lintutil/util.go:287 +0x56f fp=0xc0d902fa78 sp=0xc0d902f750 pc=0x6e221f
honnef.co/go/tools/lint/lintutil.ProcessFlagSet(0xc0000e8720, 0x4, 0x6, 0xc0000e86c0)
        /home/amarnath/go/pkg/mod/honnef.co/go/[email protected]/lint/lintutil/util.go:178 +0x9aa fp=0xc0d902ff00 sp=0xc0d902fa78 pc=0x6e12aa
main.main()
        /home/amarnath/go/pkg/mod/honnef.co/go/[email protected]/cmd/staticcheck/staticcheck.go:29 +0x2d0 fp=0xc0d902ff98 sp=0xc0d902ff00 pc=0x8f8ab0
runtime.main()
        /usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc0d902ffe0 sp=0xc0d902ff98 pc=0x42fdbc
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc0d902ffe8 sp=0xc0d902ffe0 pc=0x45ac91
Makefile:59: recipe for target 'staticcheck' failed

This might be related to dominikh/go-tools#419 which appears to be a similar issue.

@davecheney
Copy link
Contributor

davecheney commented Mar 18, 2019 via email

@vaamarnath
Copy link
Contributor Author

No, I am using 64bit build of Go.

➜  contour git:(migrate-to-go-mod) ✗ which go
/usr/local/go/bin/go
➜  contour git:(migrate-to-go-mod) ✗ file /usr/local/go/bin/go
/usr/local/go/bin//go: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, not stripped
➜  contour git:(migrate-to-go-mod) ✗ file ~/go/bin/staticcheck
/home/amarnath/go/bin/staticcheck: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, not stripped

@davecheney
Copy link
Contributor

davecheney commented Mar 18, 2019 via email

@davecheney
Copy link
Contributor

@vaamarnath thanks for working on this. Why do we need to use -mod vendor mode all the way through the makefile and build tools?

@vaamarnath
Copy link
Contributor Author

My thought process was to make use of the vendor/ directory for all builds and tests available at the root of the repository.

Quoting from https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away

In brief, to use vendoring with modules:

  • go mod vendor resets the main module's vendor directory to include all packages needed to build and test all of the module's packages based on the state of the go.mod files and Go source code.
  • By default, go commands like go build ignore the vendor directory when in module mode.
  • The -mod=vendor flag (e.g., go build -mod=vendor) instructs the go commands to use the main module's top-level vendor directory to satisfy dependencies. The go commands in this mode therefore ignore the dependency descriptions in go.mod and assume that the vendor directory holds the correct copies of dependencies. Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored.
  • Some people will want to routinely opt-in to vendoring by setting a GOFLAGS=-mod=vendor environment variable.

From what I understand, it avoids go build and go test from downloading the dependencies and to re-use what's available in vendor/. https://tip.golang.org/cmd/go/#hdr-Modules_and_vendoring

@davecheney Is this a wrong approach to take here?

@davecheney davecheney merged commit dbe534e into projectcontour:master Mar 21, 2019
@davecheney
Copy link
Contributor

@vaamarnath my understanding is that wiki page is non canonical, it is community maintained and non authoratative. The goal of switching to using go modules is to switch away from vendor entirely. I don't see the value in using modules to continue working in a vendor mode

@davecheney davecheney added this to the 0.11.0 milestone Mar 21, 2019
@vaamarnath vaamarnath deleted the migrate-to-go-mod branch March 21, 2019 14:11
@vaamarnath
Copy link
Contributor Author

@davecheney TIL that I should not rely on that wiki page. Thanks for pointing out. Can you point to the best documentation for go modules that I should read upon? Big thanks in advance!

@davecheney
Copy link
Contributor

davecheney commented Mar 21, 2019 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.11.0
Development

Successfully merging this pull request may close these issues.
2 participants
@vaamarnath @davecheney