build fails with golang 1.16.0

[giulianozor]

Hello,
I am trying to build by project with golang 1.16beta1 1.16.0 (using the alpine based docker image) and I get the following error:

buffalo build -v
DEBU[2021-01-05T07:39:43Z] Step: e1807c46
DEBU[2021-01-05T07:39:43Z] Chdir: /test/xxx
DEBU[2021-01-05T07:39:43Z] File: /test/xxx/main.go
DEBU[2021-01-05T07:39:43Z] File: /test/xxx/a/aa.go
DEBU[2021-01-05T07:39:43Z] File: /test/xxx/buffalo_build_main.go
DEBU[2021-01-05T07:39:43Z] File: /test/xxx/migrations/.pop-tmp.md
DEBU[2021-01-05T07:39:43Z] Exec: go build -v -tags development -o bin/xxx
go: updates to go.mod needed; try 'go mod tidy' first
DEBU[2021-01-05T07:39:43Z] Rollback: main.go
DEBU[2021-01-05T07:39:43Z] File: /test/xxx/main.go
DEBU[2021-01-05T07:39:43Z] Delete: a/aa.go
DEBU[2021-01-05T07:39:43Z] Delete: buffalo_build_main.go
DEBU[2021-01-05T07:39:43Z] Delete: migrations/.pop-tmp.md
DEBU[2021-01-05T07:39:43Z] Exec: go mod tidy
Usage:
  buffalo build [flags]

Aliases:
  build, b, bill, install

Flags:
      --clean-assets               will delete public/assets before calling webpack
      --dry-run                    runs the build 'dry'
      --environment string         set the environment for the binary (default "development")
  -e, --extract-assets             extract the assets and put them in a distinct archive
  -h, --help                       help for build
      --ldflags string             set any ldflags to be passed to the go build
      --mod string                 -mod flag for go build
  -o, --output string              set the name of the binary
  -k, --skip-assets                skip running webpack and building assets
      --skip-build-deps            skip building dependencies
      --skip-template-validation   skip validating templates
  -s, --static                     build a static binary using  --ldflags '-linkmode external -extldflags "-static"'
  -t, --tags string                compile with specific build tags
  -v, --verbose                    print debugging information

ERRO[0000] Error: exit status 1

A strange thing is that echoing $? returns 255, not 1

Running go mod tidy does not solve the issue.
Running buffalo dev works fine
The project builds fine with golang 1.15.5 and the same buffalo versio (0.16.18)

What can I check to provide more useful infos/solve this ?

test repo: https://github.com/giulianozor/buffalotest

thanks

Update: running go get -u, buffalo fix buffalo update does not fix the issue
Update 2: if I run buffalo build --dry-run it seems to complete without error, but obviously no binary is produced

[giulianozor]

Update: still present in 0.16.21

[giulianozor]

Update: still present with go 1.16rc1

[paganotoni]

hey @giulianozor 👋,

It seems the project needs to run go mod tidy. I see the following message in the logs:

go: updates to go.mod needed; try 'go mod tidy' first

could you try running go mod tidy and seeing if that helps.

[giulianozor]

Hi @paganotoni
running go mod tidy seems to have no effect. I get no errors from the command, but the buffalo build still fails with the same error :(

[giulianozor]

test repo: https://github.com/giulianozor/buffalotest

[giulianozor]

Update: with go 1.16.0 I get some hints about installing some missing libraries.
After

        go get -u github.com/gobuffalo/buffalo
        go get -u github.com/gobuffalo/buffalo/render
        go get -u github.com/markbates/grift/grift
  	go get -u github.com/gobuffalo/flect
  	go get -u github.com/gobuffalo/packr/v2
  	go get -u github.com/gobuffalo/pop/v5
  	go get -u github.com/gobuffalo/packr/v2
  	go get -u github.com/gobuffalo/packr/v2/file/resolver
  	go get -u github.com/gobuffalo/buffalo/binding

buffalo build runs fine. If I use go install xxx I get the same error

[saurori]

I'm also seeing the same errors on go 1.16. It's not clear why the buffalo command needs these dependencies. The current Dockerfile does a go get on pop here.

Also, since the Dockerfile uses golang:latest, does that mean the image will break (since 1.16 is latest)?

[AlexMercedCoder]

I was just writing a tutorial after updating go and running into the same issue, created a fresh project to see if maybe I broke something along the way and no luck.

I did a go get on the libraries it asked me too and ran go mod tidy and it reduced the errors a bit but it still wants me to import actions and models and no quite sure how do that successfully.

I tried

module project1

go 1.16

require (
	github.com/gobuffalo/buffalo-pop/v2 v2.3.0 // indirect
	github.com/gobuffalo/envy v1.9.0 // indirect
	github.com/gobuffalo/pop/v5 v5.3.0 // indirect
	github.com/gobuffalo/validate/v3 v3.1.0 // indirect
	github.com/gofrs/uuid v3.2.0+incompatible // indirect
	github.com/rs/cors v1.7.0 // indirect
    project1/actions v0.0.0
    project1/models v0.0.0
)

but that didn't work I get this from buffalo dev

buffalo dev
buffalo: 2021/03/04 13:05:25 === Rebuild on: :start: ===
buffalo: 2021/03/04 13:05:25 === Running: go build -v -tags development -o tmp/project1-build  (PID: 494942) ===
go: project1/[email protected]: missing go.sum entry; to add it:
        go mod download project1/actions
buffalo: 2021/03/04 13:05:25 === Error! ===
buffalo: 2021/03/04 13:05:25 === exit status 1
go: project1/[email protected]: missing go.sum entry; to add it:
        go mod download project1/actions
 ===

then I run go mod tidy

go: project1/[email protected]: malformed module path "project1/actions": missing dot in first path element

When I create a fresh project and run buffalo dev I get this

buffalo dev
buffalo: 2021/03/04 13:07:18 === Rebuild on: :start: ===
buffalo: 2021/03/04 13:07:18 === Running: go build -v -tags development -o tmp/project2-build  (PID: 495212) ===
go: project2/actions: package github.com/gobuffalo/envy imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/[email protected]
go: project2/models: package github.com/gobuffalo/envy imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/[email protected]
go: project2/models: package github.com/gobuffalo/pop/v5 imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/pop/[email protected]
buffalo: 2021/03/04 13:07:18 === Error! ===
buffalo: 2021/03/04 13:07:18 === exit status 1
go: project2/actions: package github.com/gobuffalo/envy imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/[email protected]
go: project2/models: package github.com/gobuffalo/envy imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/[email protected]
go: project2/models: package github.com/gobuffalo/pop/v5 imported from implicitly required module; to add missing requirements, run:
        go get github.com/gobuffalo/pop/[email protected]
 ===

[saurori]

EDIT: better solution is to import the dep which forces go.mod to add as direct dep: import _ "github.com/gobuffalo/flect"

For anyone still stuck on this, I was able to get a build working with Go 1.16. My main problem was this error:

 > [builder 22/22] RUN buffalo build -o /apps/myapp:                                                                                                                              
#30 31.26 go: github.com/saurori/myapp/a: package github.com/gobuffalo/flect imported from implicitly required module; to add missing requirements, run:               
#30 31.26       go get github.com/gobuffalo/[email protected] 

I'm not exactly sure what is causing the error but adding go get github.com/gobuffalo/flect right before buffalo build solves the problem.

Here's the working build stage of my Dockerfile:

FROM golang:1.16 as builder

ENV GOPROXY=https://proxy.golang.org

RUN go version

RUN apt-get update \
    && apt-get install -y -q build-essential

# Installing Node 16
RUN curl -sL https://deb.nodesource.com/setup_16.x | bash
RUN apt-get update && apt-get install nodejs

# Installing yarn
RUN npm install -g --no-progress yarn \
    && yarn config set yarn-offline-mirror /npm-packages-offline-cache \
    && yarn config set yarn-offline-mirror-pruning true

# Installing buffalo binary
RUN curl -sf https://gobinaries.com/gobuffalo/buffalo/[email protected] | sh
RUN go get github.com/gobuffalo/buffalo-pop/v2
RUN buffalo version

RUN mkdir -p /apps/myapp
WORKDIR /apps/myapp

ENV GO_ENV=production

# this will cache the npm install step, unless package.json changes
ADD package.json .
ADD .yarnrc.yml .
ADD .yarn /apps/myapp/.yarn
ADD yarn.lock .
RUN yarn install
# Copy the Go Modules manifests
COPY go.mod go.mod
COPY go.sum go.sum
# cache deps before building and copying source so that we don't need to re-download as much
# and so that source changes don't invalidate our downloaded layer
RUN go mod download

ADD . .
RUN go get github.com/gobuffalo/flect
RUN buffalo build -o /apps/myapp

[fasmat]

I made a small discovery when trying to figure out where the issue comes from.

buffalo build creates an additional package called a in the projects root. This package contains a single go file aa.go with an import to github.com/gobuffalo/flect. The template for the file can be found here: https://github.com/gobuffalo/buffalo/blob/master/genny/build/templates/a/aa.go.plush

When I prevent the buffalo command line tool from deleting this file after generating it (by e.g. removing the write dependency on the a directory) the build completes without any issues and github.com/gobuffalo/flect v0.2.2 is added to go.mod.

So the issue introduced in 1.16 seems to be that go build detects a dependency on the flect package but when it tries to resolve this dependency the source file has already been deleted again, where 1.15 had no issues with.

[giulianozor]

@fasmat
I tested v.0.16.25 and unfortunately I get the same error (goland 1.16.5 and 1.17beta1)

go: pro: package github.com/markbates/grift/grift imported from implicitly required module; to add missing requirements, run:
	go get github.com/markbates/grift/[email protected]
go: pro/a: package github.com/gobuffalo/flect imported from implicitly required module; to add missing requirements, run:
	go get github.com/gobuffalo/[email protected]
go: pro/a: package github.com/gobuffalo/packr/v2 imported from implicitly required module; to add missing requirements, run:
	go get github.com/gobuffalo/packr/[email protected]
go: pro/packrd: package github.com/gobuffalo/packr/v2 imported from implicitly required module; to add missing requirements, run:
	go get github.com/gobuffalo/packr/[email protected]
go: pro/packrd: package github.com/gobuffalo/packr/v2/file/resolver imported from implicitly required module; to add missing requirements, run:
	go get github.com/gobuffalo/packr/v2/file/[email protected]

I have buffalo 0.16.25 installed and also doucle checked the go.mod version is the same

thanks

[fasmat]

@giulianozor seems like my previous fix doesn't always work. I tried a more reliable fix here: #2119. Can you try this?

[paganotoni]

@fasmat I just released v0.16.26-beta to confirm if its working. If it does we can then release v0.16.26

[fasmat]

@paganotoni It seems to be working. I have two projects using gobuffalo, both build now with Go 1.16 without issues.

@giulianozor can you confirm as well?

[paganotoni]

Great news! Thanks @fasmat. Will wait for @giulianozor to close the issue and release v0.16.26

[fasmat]

@giulianozor I cloned your test repository https://github.com/giulianozor/buffalotest . With the following Dockerfile it fails to build with the known errors:

FROM golang:1.16-alpine
ENV ADDR=0.0.0.0
ENV PORT=3000
WORKDIR /test
RUN apk add git --no-cache
COPY . .
RUN go get github.com/gobuffalo/buffalo/[email protected]
RUN go mod download
RUN go mod tidy
RUN buffalo build

If I replace line 7 with RUN go get github.com/gobuffalo/buffalo/[email protected] the build completes successfully.

[paganotoni]

Awesome. Releasing v0.16.26 and closing this issue after.

[paganotoni]

v0.16.26 released. Thanks @fasmat for taking care of this issue.