can't load package when building

[prologic]

$ fn build
Running prebuild command: docker run --rm -v /Users/prologic/faas-test:/go/src/github.com/x/y -w /go/src/github.com/x/y iron/go:dev go build -o func
can't load package: package github.com/x/y: no buildable Go source files in /go/src/github.com/x/y
error running docker build: exit status 1

Pretty much followed the README, created a directory, copied the sample.

[prologic]

And... I think I know why:

$ fn build -v
building /Users/prologic/faas-test/func.yaml
Running prebuild command: docker run --rm -v /Users/prologic/faas-test:/go/src/github.com/x/y -w /go/src/github.com/x/y iron/go:dev go build -o func
can't load package: package github.com/x/y: no buildable Go source files in /go/src/github.com/x/y
error running docker build: exit status 1

fb build makes a really silly assumption. It assumes the Docker host is local to where you run fn from. This is almost never the case for full production systems. All my Docker nodes are remote to me.

[c0ze]

Yes, fn tool is used to build a container which hosts your local function/application.

It doesn't really make sense to use a remote docker host with fn build, because your local files will need to be copied there as well. Therefore the recommended way to provision and deploy your function is to use a container registry (Dockerhub, or ECR, or your own). The flow should be something like this :

• init & develop in your local
• build
• push the container to some CR
• pull & run at your production node which is running the functions container (you can do this via docker remote host)

If you have an idea for an alternative flow, please let us know !

Hmm, thinking about this, (and checking the documentation for docker build) it seems it is possible to specify a git repository. In that case, it should be possible to build via docker remote API if the project is in a repo.

[prologic]

Hmm ... AFAIK docker build itself copies the contents of the cwd into what's called a "context", shoves that to the remote api, unpacks it, then runs the build instructions in the Dockerfile I feel like fb needs to have the same/similar capability -- either that or re-use the docker build api somehow? James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Wed, Aug 9, 2017 at 11:11 PM, Arda Karaduman ***@***.***> wrote: Yes, fn tool is used to build a container which hosts your local function/application. It doesn't really make sense to use a remote docker host with fn build, because your local files will need to be copied there as well. Therefore the recommended way to provision and deploy your function is to use a container registry (Dockerhub, or ECR, or your own). The flow should be something like this : - init & develop in your local - build - push the container to some CR - pull & run at your production node which is running the functions container (you can do this via docker remote host) If you have an idea for an alternative flow, please let us know ! Hmm, thinking about this, (and checking the documentation for docker build <https://docs.docker.com/engine/reference/commandline/build/#usage>) it seems it is possible to specify a git repository. In that case, it should be possible to build via docker remote API if the project is in a repo. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#631 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-iSYaZfXK_-2oA4V2830sSNmM2f1ks5sWp74gaJpZM4OsEnW> .

[c0ze]

It seems you are right, thank you for the clarification !
I'm checking the fn source code, it seems fn tool is executing docker as a shell command.

functions/fn/common.go

Line 95 in d8871d1

cmd := exec.Command("docker", "build", "-t", ff.FullName(), ".")

The proper solution to this would be to re write fn tool using docker golang bindings, but as a short term solution, I am wondering if you could get this to work by passing DOCKER_HOST to fn build ? ie

DOCKER_HOST={your docker host here} fn build

[prologic]

I'll try that... James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Thu, Aug 10, 2017 at 1:07 AM, Arda Karaduman ***@***.***> wrote: It seems you are right, thank you for the clarification ! I'm checking the fn source code, it seems fn tool is executing docker as a shell command. https://github.com/iron-io/functions/blob/d8871d1562c0e30fcf43f7e02b19a7 06f366eafb/fn/common.go#L95 The proper solution to this would be to re write fn tool using docker golang bindings, but as a short term solution, I am wondering if you could get this to work by passing DOCKER_HOST to fn build ? ie DOCKER_HOST={your docker host here} fn build — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#631 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-nnL3Dn248RgzQo96UzmeLJW7ayzks5sWrpEgaJpZM4OsEnW> .

[prologic]

Yeah that doesn't work I'm afraid :/

$ DOCKER_HOST=tcp://10.0.0.10:2376 fn build -v
building /Users/prologic/hello-faas/func.yaml
Running prebuild command: docker run --rm -v /Users/prologic/hello-faas:/go/src/github.com/x/y -w /go/src/github.com/x/y iron/go:dev go build -o func
can't load package: package github.com/x/y: no buildable Go source files in /go/src/github.com/x/y
error running docker build: exit status 1

I think the main issue is the lack of using the Docker build API itself and sending context (contents of the working directory) up to the remote host as part of the image build. IIUC fn build should really be wrapping docker build right? Using the API proper will probably fix this.

[prologic]

Meanwhile my only work-around is to spin up a Docker machine on my Mac which is unideal as its woefully underpowered :) (tiny 11" Macbook) -- hence the whole reason I have an entire cluster of machiens more grunty!

[c0ze]

Hello again,

I've checked into this issue, it seems the problem is with

https://github.com/iron-io/functions/blob/master/fn/langs/go.go#L29

which tries to mount your local dir to the remote host via the -v flag.

moby/moby#4023

which doesn't work. As a work-around, you can create your own Dockerfile, which would prevent this prebuild step.

I'm linking an example project which has a known working Dockerfile for remote building.

https://bitbucket.org/coze/query/src/dff81a3ab0a39f1c586efe394a377a382065ed0d/?at=custom-dockerfile

This one uses glide for dependency management, but feel free to use another one if you like.

PS: you also need the .dockerignore to prevent your local vendor directory being copied over to the host.

[prologic]

I see! I'll check this out and report back. Do we still want to keep this issue open to track making appropriate changes to the build step so it doesn't assume a local Docker engine? James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Mon, Aug 14, 2017 at 3:20 AM, Arda Karaduman ***@***.***> wrote: Hello again, I've checked into this issue, it seems the problem is with https://github.com/iron-io/functions/blob/master/fn/langs/go.go#L29 which tries to mount your local dir to the remote host via the -v flag. moby/moby#4023 <moby/moby#4023> which doesn't work. As a work-around, you can create your own Dockerfile, which would prevent this prebuild step. I'm linking an example project which has a known working Dockerfile for remote building. https://bitbucket.org/coze/query/src/dff81a3ab0a39f1c586efe394a377a 382065ed0d/?at=custom-dockerfile This one uses glide <https://glide.sh/> for dependency management, but feel free to use another one if you like. PS: you also need the .dockerignore to prevent your local vendor directory being copied over to the host. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#631 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-oNoudybU6P6bLtivgh-8HVF2VJeks5sYB9ugaJpZM4OsEnW> .

[c0ze]

I created a new issue for that. Please close this one if the Dockerfile solves your problem.

[prologic]

Seems to work:

prologic@Jamess-MacBook
Wed Aug 16 21:58:10
~/hello-faas
 0
$ fn build
Building image prologic/hello-faas:0.0.1
Sending build context to Docker daemon  4.096kB
Step 1/8 : FROM golang:alpine
 ---> 310e63753884
Step 2/8 : ENTRYPOINT /func
 ---> Running in 0582d30bb55a
 ---> dbb47fc13237
Removing intermediate container 0582d30bb55a
Step 3/8 : RUN apk add --update git &&     rm -rf /var/cache/apk/*
 ---> Running in 717a21df83e0
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/5) Installing libssh2 (1.7.0-r2)
(2/5) Installing libcurl (7.55.0-r0)
(3/5) Installing expat (2.2.0-r1)
(4/5) Installing pcre (8.39-r0)
(5/5) Installing git (2.11.3-r0)
Executing busybox-1.25.1-r0.trigger
OK: 23 MiB in 17 packages
 ---> 7c2b194a58f2
Removing intermediate container 717a21df83e0
Step 4/8 : RUN mkdir -p /go/src
 ---> Running in d47c377284d1
 ---> 96746ddf0dea
Removing intermediate container d47c377284d1
Step 5/8 : WORKDIR /go/src
 ---> ac4dcefcf6ed
Removing intermediate container 9367fd70cbe8
Step 6/8 : COPY . /go/src
 ---> 1bbf130cc55e
Removing intermediate container cf4dfd502619
Step 7/8 : RUN go get -v -d
 ---> Running in 08e16dabdf8a
 ---> 621aa64f1887
Removing intermediate container 08e16dabdf8a
Step 8/8 : RUN go build -o /func .
 ---> Running in 12c3b6bf8dc7
 ---> b43cefdc021c
Removing intermediate container 12c3b6bf8dc7
Successfully built b43cefdc021c
Successfully tagged prologic/hello-faas:0.0.1
Function prologic/hello-faas:0.0.1 built successfully.

Contents of Dockerfile and tree output:

prologic@Jamess-MacBook
Wed Aug 16 21:58:26
~/hello-faas
 0
$ tree
.
|-- Dockerfile
|-- func.go
`-- func.yaml

0 directories, 3 files

prologic@Jamess-MacBook
Wed Aug 16 22:00:34
~/hello-faas
 0
$ cat Dockerfile
FROM golang:alpine

ENTRYPOINT ["/func"]

RUN \
    apk add --update git && \
    rm -rf /var/cache/apk/*

RUN mkdir -p /go/src
WORKDIR /go/src

COPY . /go/src

RUN go get -v -d
RUN go build -o /func .

[prologic]

I believe it did. Closing...