Build improvements

[errordeveloper]

Description

Basic summary:

• streamlined CircleCI configuration
• store dependency cache in the build image
• 2 minute speed-up of CircleCI builds
• Fix automate tagging of build image  #1145
• Fix build cache misses #1202

See proposal document within for more details.

Checklist

• Code compiles correctly (i.e make build)
• All unit tests passing (i.e. make test)
• Manually tested

[gemagomez]

I have a more generic question for the analysis doc. When you separate build from test, how are you going to know that you are testing the right image when it comes to downloading it and testing?

[errordeveloper]

I have a more generic question for the analysis doc. When you separate build from test, how are you going to know that you are testing the right image when it comes to downloading it and testing?

It's configured explicitly in the CircleCI config at the moment, and for any given revision it can be determined based on git tree hashes. You can read Makefile.docker to find out how it works, if comments are not clear there - please let me know, I'm not very keen on duplicating the details in another file. In short, we have .build_image_manifest that tracks git tree hashes of each of the files that determine what goes into an image, that file is checked in and we use git ls-tree --full-tree @ -- .build_image_manifest to get the git tree hash of that file and use that as image tag.
So, given image tag you can use git show to find out what was in .build_image_manifest, and use git show again for each of the hashes that file returns.

E.g. given weaveworks/eksctl-build:b0aa27c30e3f533cdc861a471a0f01f624c7f3d5:

$ git show b0aa27c30e3f533cdc861a471a0f01f624c7f3d5
100644 blob 47cb6bf90db7b43772d0a77f4082d8552d0724fc    Dockerfile
100644 blob 5a627d2f0babe888166ab54168f5d11cdaa11f8e    go.mod
100644 blob 68e10db9f70d4e1d0e0159dba3cdb57669be64c0    go.sum
100755 blob 180a7df24779458c0bb748261c6dc4ed78185c86    install-build-deps.sh
$ git show 47cb6bf90db7b43772d0a77f4082d8552d0724fc | head -5
# Make sure to run the following commands after changes to this file are made:
# `make -f Makefile.docker update-build-image-manifest && make -f Makefile.docker push-build-image`

# This digest corresponds to golang:1.12.9-alpine3.10
FROM golang@sha256:e0660b4f1e68e0d408420acb874b396fc6dd25e7c1d03ad36e7d6d1155a4dff6 AS base
$

This explicitly precludes that image was built and published before tests run, which was already required previously, but not explicitly.

[errordeveloper]

cc @stefanprodan

[errordeveloper]

@marccarre I've updated the proposal doc, and rebased this - could you please take another look?

[marccarre]

Nit: do you know if there is a way to factorise docker://weaveworks/eksctl-build:19ff80e010d2fdb7be3c44fc6491e13395a682ad?

[errordeveloper]

Not sure. For now make -f Makefile.docker update-build-image-manifest handles updating it here and in .circleci/config.yml.

[errordeveloper]

Some of the fields in github configs accept shell syntax (like foo-$(echo 123)), but that didn't work here.

[marccarre]

Nit: do you know if there is a way to factorise BUILD_IMAGE: weaveworks/eksctl-build:19ff80e010d2fdb7be3c44fc6491e13395a682ad in order to have it defined at a higher-level in the YAML file?

[marccarre]

This looks reasonable to me. The only way is forward. And we can always refine later.