Reduce number of layers and provide an Alpine variant #1132

robbyoconnor · 2018-06-22T04:21:24Z

This refactors the dockerfile to use less layers and properly clean up after itself.

I have provided a new Dockerfile named Dockerfile.slim.build -- which can be tagged as alpine-based. Additionally, I did as much as I could do with the debian-based image.

Fixes #1130

robbyoconnor · 2018-06-22T04:49:11Z

I cleaned things up so it's not so bad...it follows most best practices and takes advantage of the cache -- putting stuff that does not change at the top

robbyoconnor · 2018-06-22T04:51:18Z

@drnic have a peek -- there's not much more I can do to clean this up short of switching the base image to alpine

robbyoconnor · 2018-06-22T05:35:11Z

My changes:

$ docker history b
IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
dc74742c0694        About a minute ago   /bin/sh -c ls -la /npm-packages-offline-cache   0B                  
18c611619c65        About a minute ago   /bin/sh -c #(nop) WORKDIR /go/src               0B                  
a5ef2e9491a2        About a minute ago   /bin/sh -c buffalo version                      0B                  
ef32583bc89d        About a minute ago   /bin/sh -c grep -v '{{' ./generators/assets/…   236MB               
337697b4bbb8        3 minutes ago        /bin/sh -c go get -tags "sqlite" -v -t githu…   46MB                
583ef3a54014        4 minutes ago        /bin/sh -c #(nop) ADD dir:9f01e5c84884feddf5…   15.5MB              
aa0ad9ee5b35        5 minutes ago        /bin/sh -c #(nop) WORKDIR /go/src/github.com…   0B                  
221100d7d0d3        5 minutes ago        /bin/sh -c rm -rfv $BP && mkdir -p $BP          0B                  
7e93a9db0a42        5 minutes ago        /bin/sh -c npm install -g --no-progress yarn…   5.42MB              
a2e7b8a0207f        5 minutes ago        /bin/sh -c go get -u github.com/golang/dep/c…   403MB               
2d651a63b064        11 minutes ago       /bin/sh -c curl -sL https://deb.nodesource.c…   355MB               
746d185d5c28        14 minutes ago       /bin/sh -c go version && go get -v -u github…   13MB                
77eddce36712        15 minutes ago       /bin/sh -c #(nop)  ENV BP=/go/src/github.com…   0B                  
46399fedab3e        15 minutes ago       /bin/sh -c #(nop)  EXPOSE 3000                  0B                  
52057de6c8d0        13 days ago          /bin/sh -c #(nop) WORKDIR /go                   0B                  
<missing>           13 days ago          /bin/sh -c mkdir -p "$GOPATH/src" "$GOPATH/b…   0B                  
<missing>           13 days ago          /bin/sh -c #(nop)  ENV PATH=/go/bin:/usr/loc…   0B                  
<missing>           13 days ago          /bin/sh -c #(nop)  ENV GOPATH=/go               0B                  
<missing>           13 days ago          /bin/sh -c set -eux;   dpkgArch="$(dpkg --pr…   359MB               
<missing>           13 days ago          /bin/sh -c #(nop)  ENV GOLANG_VERSION=1.10.3    0B                  
<missing>           13 days ago          /bin/sh -c apt-get update && apt-get install…   162MB               
<missing>           2 weeks ago          /bin/sh -c apt-get update && apt-get install…   142MB               
<missing>           2 weeks ago          /bin/sh -c set -ex;  if ! command -v gpg > /…   7.8MB               
<missing>           2 weeks ago          /bin/sh -c apt-get update && apt-get install…   23.2MB              
<missing>           7 weeks ago          /bin/sh -c #(nop)  CMD ["bash"]                 0B                  
<missing>           7 weeks ago          /bin/sh -c #(nop) ADD file:9572fdb59dfbb9b03…   101MB

Current image prior

$ docker history gobuffalo/buffalo:development 
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
52d952a9e7d1        4 hours ago         /bin/sh -c #(nop)  EXPOSE 3000/tcp              0B                  
<missing>           4 hours ago         /bin/sh -c ls -la /npm-packages-offline-cache   0B                  
<missing>           4 hours ago         /bin/sh -c #(nop) WORKDIR /go/src               0B                  
<missing>           4 hours ago         /bin/sh -c buffalo version                      0B                  
<missing>           4 hours ago         /bin/sh -c yarn install --no-progress           236MB               
<missing>           4 hours ago         /bin/sh -c grep -v '{{' ./generators/assets/…   1.03kB              
<missing>           4 hours ago         /bin/sh -c make install                         36.8MB              
<missing>           4 hours ago         /bin/sh -c go get -tags "sqlite" -v -t githu…   35.5MB              
<missing>           4 hours ago         /bin/sh -c #(nop) ADD dir:f28ff8a944bc7dded1…   943kB               
<missing>           4 hours ago         /bin/sh -c #(nop) WORKDIR /go/src/github.com…   0B                  
<missing>           4 hours ago         /bin/sh -c mkdir -p $BP                         0B                  
<missing>           4 hours ago         /bin/sh -c rm -rfv $BP                          0B                  
<missing>           4 hours ago         /bin/sh -c #(nop)  ENV BP=/go/src/github.com…   0B                  
<missing>           4 hours ago         /bin/sh -c yarn config set yarn-offline-mirr…   169B                
<missing>           4 hours ago         /bin/sh -c yarn config set yarn-offline-mirr…   136B                
<missing>           4 hours ago         /bin/sh -c npm install -g --no-progress yarn    5.42MB              
<missing>           4 hours ago         /bin/sh -c gometalinter --install               163MB               
<missing>           4 hours ago         /bin/sh -c go get -u github.com/alecthomas/g…   20.6MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u golang.org/x/vgo        20.3MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gorilla/s…   159kB               
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   862kB               
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   5.58MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   431kB               
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   4.03MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   4.46MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/mattn/go-…   48.1MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gobuffalo…   62.3MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gobuffalo…   27.5MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gobuffalo…   761kB               
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gobuffalo…   590kB               
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/gobuffalo…   1.03MB              
<missing>           4 hours ago         /bin/sh -c go get -u github.com/golang/dep/c…   44MB                
<missing>           4 hours ago         /bin/sh -c apt-get install -y vim               31MB                
<missing>           4 hours ago         /bin/sh -c apt-get install -y -q mysql-client   75.2MB              
<missing>           4 hours ago         /bin/sh -c apt-get install -y postgresql pos…   153MB               
<missing>           4 hours ago         /bin/sh -c wget -q https://www.postgresql.or…   8.14kB              
<missing>           4 hours ago         /bin/sh -c sh -c 'echo "deb http://apt.postg…   63B                 
<missing>           4 hours ago         /bin/sh -c apt-get install -y sqlite3 libsql…   4.93MB              
<missing>           4 hours ago         /bin/sh -c apt-get install -y build-essentia…   62.8MB              
<missing>           4 hours ago         /bin/sh -c curl -sL https://deb.nodesource.c…   29.5MB              
<missing>           4 hours ago         /bin/sh -c apt-get update                       16.2MB              
<missing>           4 hours ago         /bin/sh -c go get -v -u github.com/markbates…   13MB                
<missing>           4 hours ago         /bin/sh -c go version                           0B                  
<missing>           13 days ago         /bin/sh -c #(nop) WORKDIR /go                   0B                  
<missing>           13 days ago         /bin/sh -c mkdir -p "$GOPATH/src" "$GOPATH/b…   0B                  
<missing>           13 days ago         /bin/sh -c #(nop)  ENV PATH=/go/bin:/usr/loc…   0B                  
<missing>           13 days ago         /bin/sh -c #(nop)  ENV GOPATH=/go               0B                  
<missing>           13 days ago         /bin/sh -c set -eux;   dpkgArch="$(dpkg --pr…   359MB               
<missing>           13 days ago         /bin/sh -c #(nop)  ENV GOLANG_VERSION=1.10.3    0B                  
<missing>           13 days ago         /bin/sh -c apt-get update && apt-get install…   162MB               
<missing>           2 weeks ago         /bin/sh -c apt-get update && apt-get install…   142MB               
<missing>           2 weeks ago         /bin/sh -c set -ex;  if ! command -v gpg > /…   7.8MB               
<missing>           2 weeks ago         /bin/sh -c apt-get update && apt-get install…   23.2MB              
<missing>           7 weeks ago         /bin/sh -c #(nop)  CMD ["bash"]                 0B                  
<missing>           7 weeks ago         /bin/sh -c #(nop) ADD file:9572fdb59dfbb9b03…   101MB

As you can see - this is a significant reduction in the number of layers -- while the size may not have changed, the number of layers it needs to download should speed things up.

drnic · 2018-06-22T05:42:05Z

Well that is a pity that we didn't get a nice size improvement from that technique. Nice work though!

robbyoconnor · 2018-06-22T05:54:28Z

Not giving up -- I'm cobbling together an alpine variant...so stay tuned...let's see...

…

On 06/22/2018 01:42 AM, Dr Nic Williams wrote: Well that is a pity that we didn't get a nice size improvement from that technique. Nice work though! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1132 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABaMKUSEqVlYvgHMMMtO-vKZ44vFJTgks5t_IOvgaJpZM4UzHxS>.

robbyoconnor · 2018-06-22T06:20:12Z

Mine with Alpine as a base

$ docker history b
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
c655460632db        11 seconds ago      /bin/sh -c ls -la /npm-packages-offline-cache   0B                  
fe1b7211168e        14 seconds ago      /bin/sh -c #(nop) WORKDIR /go/src               0B                  
e304e84b5e1b        16 seconds ago      /bin/sh -c buffalo version                      0B                  
3552e0373ca0        23 seconds ago      /bin/sh -c grep -v '{{' ./generators/assets/…   236MB               
4fe063735be9        2 minutes ago       /bin/sh -c go get -tags "sqlite" -v -t githu…   46.2MB              
1af15fdbf451        3 minutes ago       /bin/sh -c #(nop) ADD dir:e4971030524eeee83e…   15.5MB              
1ac6f34cfffc        3 minutes ago       /bin/sh -c #(nop) WORKDIR /go/src/github.com…   0B                  
7c37c2f93f83        3 minutes ago       /bin/sh -c rm -rfv $BP && mkdir -p $BP          0B                  
8e06ff759a59        3 minutes ago       /bin/sh -c npm install -g --no-progress yarn…   5.42MB              
7ef51cfacccb        3 minutes ago       /bin/sh -c go get -u github.com/golang/dep/c…   399MB               
c8866a0e57a0        8 minutes ago       /bin/sh -c go version && go get -v -u github…   13MB                
84191dcc5dbc        9 minutes ago       /bin/sh -c apk add --no-cache --upgrade apk-…   294MB               
4500a1cbf7fe        20 minutes ago      /bin/sh -c #(nop)  ENV BP=/go/src/github.com…   0B                  
5fe4e258c746        20 minutes ago      /bin/sh -c #(nop)  EXPOSE 3000                  0B                  
c4b5d89b27f4        13 days ago         /bin/sh -c #(nop) WORKDIR /go                   0B                  
<missing>           13 days ago         /bin/sh -c mkdir -p "$GOPATH/src" "$GOPATH/b…   0B                  
<missing>           13 days ago         /bin/sh -c #(nop)  ENV PATH=/go/bin:/usr/loc…   0B                  
<missing>           13 days ago         /bin/sh -c #(nop)  ENV GOPATH=/go               0B                  
<missing>           13 days ago         /bin/sh -c set -eux;  apk add --no-cache --v…   371MB               
<missing>           13 days ago         /bin/sh -c #(nop)  ENV GOLANG_VERSION=1.10.3    0B                  
<missing>           5 months ago        /bin/sh -c [ ! -e /etc/nsswitch.conf ] && ec…   17B                 
<missing>           5 months ago        /bin/sh -c apk add --no-cache   ca-certifica…   555kB               
<missing>           5 months ago        /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B                  
<missing>           5 months ago        /bin/sh -c #(nop) ADD file:093f0723fa46f6cdb…   4.15MB

Final resulting size: 1.38GB

robbyoconnor · 2018-06-22T06:41:41Z

Sorry for the barrage of comments: Final resulting size:

The Alpine-based image is 1.38GB
The debian-based image is 1.87 GB, which wasn't a significant drop, but the number of layers have been sigificantly reduced.

Make base image clean up after itself better and reduce layers Size unfortunately could not be reduced in any significant way.

robbyoconnor mentioned this pull request Jun 22, 2018

Huge base image for Dockerfile #1130

Closed

robbyoconnor force-pushed the tame-dockerfile branch from fcf1fb0 to 782d834 Compare June 22, 2018 04:26

robbyoconnor changed the title ~~tame the dockerfile~~ [WIP] tame the dockerfile Jun 22, 2018

robbyoconnor force-pushed the tame-dockerfile branch 2 times, most recently from 8c372fc to a66c0e0 Compare June 22, 2018 04:37

robbyoconnor changed the title ~~[WIP] tame the dockerfile~~ [RDY] tame the dockerfile Jun 22, 2018

robbyoconnor force-pushed the tame-dockerfile branch 2 times, most recently from ac9ad30 to db36d0d Compare June 22, 2018 05:22

robbyoconnor changed the title ~~[RDY] tame the dockerfile~~ Make base image clean up after itself better and reduce layers Jun 22, 2018

robbyoconnor force-pushed the tame-dockerfile branch from db36d0d to 7141699 Compare June 22, 2018 05:23

robbyoconnor force-pushed the tame-dockerfile branch from 7141699 to cc59dad Compare June 22, 2018 06:10

robbyoconnor changed the title ~~Make base image clean up after itself better and reduce layers~~ Use Alpine Linux to get a slimmer base image Jun 22, 2018

robbyoconnor force-pushed the tame-dockerfile branch from cc59dad to 8c0a24f Compare June 22, 2018 06:36

robbyoconnor changed the title ~~Use Alpine Linux to get a slimmer base image~~ Reduce number of layers and provide an Alpine variant Jun 22, 2018

robbyoconnor force-pushed the tame-dockerfile branch from 2c6f232 to 8c0a24f Compare June 22, 2018 18:36

Dockerfile improvements: Add an alpine variant and refactor

00030e1

Make base image clean up after itself better and reduce layers Size unfortunately could not be reduced in any significant way.

robbyoconnor force-pushed the tame-dockerfile branch from 8c0a24f to 00030e1 Compare June 22, 2018 18:37

duckbrain approved these changes Jun 28, 2018

View reviewed changes

Merge branch 'development' into tame-dockerfile

1900fa6

markbates added this to the v0.12.4 milestone Jun 30, 2018

markbates approved these changes Jun 30, 2018

View reviewed changes

markbates merged commit 0e29956 into gobuffalo:development Jun 30, 2018

robbyoconnor deleted the tame-dockerfile branch June 30, 2018 18:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce number of layers and provide an Alpine variant #1132

Reduce number of layers and provide an Alpine variant #1132

robbyoconnor commented Jun 22, 2018 •

edited

Loading

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018

drnic commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018 via email

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018

Reduce number of layers and provide an Alpine variant #1132

Reduce number of layers and provide an Alpine variant #1132

Conversation

robbyoconnor commented Jun 22, 2018 • edited Loading

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018

My changes:

Current image prior

drnic commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018 via email

robbyoconnor commented Jun 22, 2018

Mine with Alpine as a base

robbyoconnor commented Jun 22, 2018

robbyoconnor commented Jun 22, 2018 •

edited

Loading