Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: more fixes for CircleCI cross-build setup #20499

Closed
wants to merge 11 commits into from
Closed

build: more fixes for CircleCI cross-build setup #20499

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
32fe8d2
build: try upgrading Docker instead of installing buildx plugin
danxmoran Jan 12, 2021
e1b3e96
build: enable jobs for debugging
danxmoran Jan 12, 2021
0e0e733
build: try an older ubuntu version
danxmoran Jan 12, 2021
209aee5
build: restart docker service after upgrading
danxmoran Jan 12, 2021
e3c4fd8
build: back to 20.04
danxmoran Jan 12, 2021
f14084a
build: try installing more qemu support
danxmoran Jan 12, 2021
053d6a8
build: try running build jobs in-order
danxmoran Jan 12, 2021
18b7fe0
build: don't share GOCACHE archives between release job types
danxmoran Jan 12, 2021
8d7689e
build: reset steps of build workflow
danxmoran Jan 12, 2021
0bfb11a
build: run build and cross_build simultaneously
danxmoran Jan 12, 2021
f0c05db
build: remove 1 emulator image, and uninstall system emulators
danxmoran Jan 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 30 additions & 27 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ workflows:
requires:
- godeps
- jsdeps
- cross_build:
requires:
- godeps
- jsdeps
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am torn between running this on every PR (catches problems early, but slow) and only running it on master. Thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better I think to weed out problems early.

- e2e:
requires:
- build
Expand All @@ -41,13 +45,6 @@ workflows:
branches:
only:
- /^(?!pull\/).*$/
- cross_build:
requires:
- build
filters:
branches:
only:
- master
- litmus_integration:
requires:
- build
Expand Down Expand Up @@ -166,6 +163,7 @@ commands:
sudo apt-get install -y --no-install-recommends \
cmake \
gnupg \
jq \
libssl-dev \
libxml2-dev \
llvm-dev \
Expand Down Expand Up @@ -218,25 +216,36 @@ commands:
echo 'export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=/usr/local/musl-cross/bin/aarch64-unknown-linux-musl-gcc' >> $BASH_ENV
echo 'export CARGO_TARGET_X86_64_APPLE_DARWIN_LINKER=/usr/local/osxcross/target/bin/x86_64-apple-darwin15-clang' >> $BASH_ENV
- run:
name: Install Docker cross-build kit and emulators
environment:
DOCKER_BUILDX_VERSION: 0.5.1
name: Install updated Docker
command: |
set -euo pipefail
export BUILDKIT_PROGRESS=plain
export DOCKER_BUILDKIT=1
export DOCKER_CLI_EXPERIMENTAL=enabled
echo 'export BUILDKIT_PROGRESS=plain' >> $BASH_ENV
echo 'export DOCKER_BUILDKIT=1' >> $BASH_ENV
echo 'export DOCKER_CLI_EXPERIMENTAL=enabled' >> $BASH_ENV

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
sudo apt-get install -y docker-ce docker-ce-cli containerd.io
sudo service docker restart
- run:
name: Set up Docker cross-builder
command: |
# Uninstall any emulators provided by the system.
emulators=($(docker run --rm --privileged tonistiigi/binfmt:latest) | jq -r .emulators[])
for e in ${emulators[@]}; do
docker run --rm --privileged tonistiigi/binfmt:latest --uninstall ${e}
done

# Install the QEMU emulators we need to cross-build.
docker run --rm --privileged tonistiigi/binfmt:latest --install arm64
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

mkdir -p ${HOME}/.docker/cli-plugins
curl -sfL -o ${HOME}/.docker/cli-plugins/docker-buildx \
https://github.com/docker/buildx/releases/download/v${DOCKER_BUILDX_VERSION}/buildx-v${DOCKER_BUILDX_VERSION}.linux-amd64
chmod a+x ${HOME}/.docker/cli-plugins/docker-buildx
docker buildx install
echo 'export DOCKER_CLI_EXPERIMENTAL=enabled' >> $BASH_ENV
docker run --rm --privileged tonistiigi/binfmt:latest --install all

# Create a new buildx context using the freshly-installed emulators.
docker buildx create --name cross-builder --platform linux/amd64,linux/arm64
docker buildx create --name cross-builder
docker buildx use --default cross-builder
docker buildx inspect --bootstrap
- run:
Expand Down Expand Up @@ -571,8 +580,6 @@ jobs:
keys:
- influxdb-cross-build-{{ .Branch }}-{{ .Revision }}
- influxdb-cross-build-{{ .Branch }}-
- influxdb-build-{{ .Branch }}-{{ .Revision }}
- influxdb-build-{{ .Branch }}-
- restore_cache:
name: Restore Yarn Cache
keys:
Expand Down Expand Up @@ -606,11 +613,7 @@ jobs:
name: Restore GOCACHE
keys:
- influxdb-nightly-{{ .Branch }}-{{ .Revision }}
- influxdb-cross-build-{{ .Branch }}-{{ .Revision }}
- influxdb-nightly-{{ .Branch }}-
- influxdb-cross-build-{{ .Branch }}-
- influxdb-build-{{ .Branch }}-{{ .Revision }}
- influxdb-build-{{ .Branch }}-
- restore_cache:
name: Restore Yarn Cache
keys:
Expand Down