Introduce Github Actions CI workflow #3339

siggy · 2019-08-28T19:27:15Z

The existing Travis CI setup requires additional integrations and
permissions with Github, and also lacks some flexibility around job
dependency management.

Introduce a new CI workflow based on Github Actions. This initial
workflow performs the same CI work that Travis does, and will iniitially
run in parallel:

Go unit tests
JS unit tests
Go lint
Validate Go deps
Integration tests (deep, upgrade, helm)

Signed-off-by: Andrew Seigner [email protected]

olix0r

@siggy This looks awesome! To confirm: we'll continue to rely on Travis for artifact publishing until we change that process?

siggy · 2019-08-28T20:10:58Z

@olix0r Correct re: artifact publishing. For the moment Travis will continue to handle that. I'd like to get Github Actions in place just for pull request CI, then later take on artifact publishing and release automation.

grampelberg

Really just a bunch of questions, this looks great. Super easy to understand what's going on and I love the kind/remote docker implementation.

grampelberg · 2019-08-28T20:08:25Z

.github/workflows/workflow.yml

+    - name: Checkout code
+      uses: actions/checkout@v1
+    - name: ENV
+      run: env | sort


Wouldn't this leak secrets?

The actions UI obscures them. Also that line was for debugging, I've removed it.

grampelberg · 2019-08-28T20:11:34Z

.github/workflows/workflow.yml

+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v1
+    - name: Yarn setup


If we wanted to get rid of this step, would we need to use an image with yarn already installed? Build cache it? Copy files from a tarball?

Yeah any of those approaches would work. We run this command today in Travis, so mostly looking to match that setup before optimizing.

grampelberg · 2019-08-28T20:12:57Z

.github/workflows/workflow.yml

+      run: |
+        export PATH="$HOME/.yarn/bin:$PATH"
+        export NODE_ENV=test
+        bin/web


I assume this just came over from the existing tooling, but should it be more explicit what's being done? Maybe bin/web setup && bin/web test? I don't think a build is required and it might speed things up some.

Yeah that's correct this is cribbed from Travis. I'm totally down to optimize further, but for this PR I mostly want to get it into the pipeline as-is.

grampelberg · 2019-08-28T20:15:28Z

.github/workflows/workflow.yml

+        DOCKER_HOST_PRIVATE_KEY: ${{ secrets.DOCKER_HOST_PRIVATE_KEY }}
+      run: |
+        mkdir -p ~/.ssh/
+        echo "$DOCKER_HOST_PRIVATE_KEY" > ~/.ssh/id_rsa


This'll put the full key in the logs, right?

I don't think so?
https://github.com/linkerd/linkerd2/pull/3339/checks?check_run_id=206118982#step:3:2

Oh, I was using a Makefile and that auto-expanded this for reasons. Ignore me!

fwiw, steps within a job shares the same workspace filesystem. If we ever need to add -x, we can use shell to write the secrets to files on the filesystem, and refer them (e.g. scp -i) in subsequent steps.

grampelberg · 2019-08-28T20:16:05Z

.github/workflows/workflow.yml

+        DOCKER_HOST_PRIVATE_KEY: ${{ secrets.DOCKER_HOST_PRIVATE_KEY }}
+      run: |
+        mkdir -p ~/.ssh/
+        echo "$DOCKER_HOST_PRIVATE_KEY" > ~/.ssh/id_rsa


Same question as above.

https://github.com/linkerd/linkerd2/pull/3339/checks?check_run_id=206118982#step:3:2

grampelberg · 2019-08-28T20:28:12Z

.github/workflows/workflow.yml

+      env:
+        DOCKER_ADDRESS: ${{ secrets.DOCKER_ADDRESS }}
+        DOCKER_HOST_PRIVATE_KEY: ${{ secrets.DOCKER_HOST_PRIVATE_KEY }}
+      run: |


Is there a way to make this step more common? Is there a sandbox shared between all the steps?

Yeah, it's unfortunate we're doing this Docker SSH setup step across 4 different jobs. Each job runs in a separate VM so not great shared option.

From a performance perspective, this step only takes 1 second.

From a maintenance perspective, I agree it's less than ideal. We solved this in Travis using YAML aliases, but this is not supported in Github Actions.

grampelberg · 2019-08-28T20:28:44Z

.github/workflows/workflow.yml

+      run: |
+        TAG="$(CI_FORCE_CLEAN=1 bin/root-tag)"
+        export KIND_CLUSTER=github-$TAG-${{ matrix.integration_test }}
+        bin/kind delete cluster --name=$KIND_CLUSTER


If the kind cluster doesn't exist, does this succeed? Or, does it even matter?

If the cluster doesn't exist, this command will fail. This is probably fine because if we've gotten this far and the cluster does not exist, a previous job must have failed.

l5d-bot · 2019-08-28T20:48:45Z

Integration test results for 57503f8: success 🎉
Log output: https://gist.github.com/3e6d41f156c88efe0149478d6343bb61

alpeb · 2019-08-29T14:24:28Z

.github/workflows/workflow.yml

+      env:
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |
+        echo "$GITCOOKIE_SH" | bash


We also had this in Travis, but I've never known what it is for...

Good question! It's to mitigate rate-limiting when pulling go dependencies:
golang/go#12933 (comment)

The good news is we should be able to remove this when Go 1.13 lands, as Google will run a module mirror:
https://proxy.golang.org/

alpeb · 2019-08-29T14:37:53Z

.github/workflows/workflow.yml

+  kind_setup:
+    strategy:
+      matrix:
+        integration_test: [deep, upgrade, helm]


alpeb

This looks awesome 🥇

A few ideas about dependencies:

Can we have go_unit_tests and js_unit_tests depend on the success of validate_go_deps and go_lint (these being fast, it shouldn't augment too much the overall time)?
docker_build could also depend on go_unit_tests and js_unit_tests .

On the other hand, do you know how this interacts with forks?

siggy · 2019-08-29T17:56:20Z

@alpeb Re: Dependencies between jobs, I'd like to avoid introducing dependencies unless a job actually depends on a previous job. The goal is to minimize the total time for all jobs to complete, and also make clear what jobs depends on other jobs. While validate_go_deps takes only ~8s, go_lint takes ~4m, and go_unit_tests takes ~5m.

Fortunately the Github PR and Actions UIs provide feedback when individual jobs fail, prior to the full workflow completing, so the user gets feedback asap.

For reference, right now the critical path is:
[docker_build | kind_setup] -> kind_integration -> kind_cleanup

.github/workflows/workflow.yml

ihcsim · 2019-08-29T18:11:06Z

.github/workflows/workflow.yml

+        DOCKER_ADDRESS: ${{ secrets.DOCKER_ADDRESS }}
+        DOCKER_HOST: ssh://github@${{ secrets.DOCKER_ADDRESS }}
+        GITCOOKIE_SH: ${{ secrets.GITCOOKIE_SH }}
+      run: |


I'm curious about the error reporting in run. So if e.g., the scp command fail, does run terminate immediately? And does the UI show which shell command fail? Or do we need -e? Setting -x is probably not a good idea here, because of the secret env var.

Fail fast is on by default:
https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstepsrunshell

I believe the UI highlights which command fails.

ihcsim · 2019-08-29T18:14:01Z

.github/workflows/workflow.yml

+        DOCKER_HOST_PRIVATE_KEY: ${{ secrets.DOCKER_HOST_PRIVATE_KEY }}
+      run: |
+        mkdir -p ~/.ssh/
+        echo "$DOCKER_HOST_PRIVATE_KEY" > ~/.ssh/id_rsa


fwiw, steps within a job shares the same workspace filesystem. If we ever need to add -x, we can use shell to write the secrets to files on the filesystem, and refer them (e.g. scp -i) in subsequent steps.

siggy · 2019-08-29T18:51:49Z

@ihcsim Agree good to highlight the same workspace filesystem feature between steps in a job. We are taking advantage of this feature in the Docker SSH setup, but the more, the better.

ihcsim

👍

siggy · 2019-08-29T21:35:22Z

Filed related issues in the actions/checkout repo and Github Community forums:
actions/checkout#27
https://github.521000.bestmunity/t5/GitHub-API-Development-and/Github-Actions-Inconsistent-repo-checkouts-across-jobs/td-p/30258

Sort of relates to this issue in actions/checkout:
actions/checkout#15

The existing Travis CI setup requires additional integrations and permissions with Github, and also lacks some flexibility around job dependency management. Introduce a new CI workflow based on Github Actions. This initial workflow performs the same CI work that Travis does, and will iniitially run in parallel: - Go unit tests - JS unit tests - Go lint - Validate Go deps - Integration tests (deep, upgrade, helm) Signed-off-by: Andrew Seigner <[email protected]>

Signed-off-by: Andrew Seigner <[email protected]>

PR #3339 introduced a GitHub Actions CI workflow. Add a badge to the top of README.md to report status of the CI workflow. Signed-off-by: Andrew Seigner <[email protected]>

PR #3339 introduced a GitHub Actions CI workflow. Booting 6 clusters simultaneously (3x Github Actions + 3x Travis) exhibits some transient failures. Retry kind cluster creation once Retry log reading from integration k8s clusters once Add kind cluster creation debug logging Add a badge to the top of README.md to report status of the CI workflow. Signed-off-by: Andrew Seigner <[email protected]>

PR #3339 introduced a GitHub Actions CI workflow. Booting 6 clusters simultaneously (3x Github Actions + 3x Travis) exhibits some transient failures. Implement fixes in GitHub Actions and integration tests to address kind cluster creation and testing: - Retry kind cluster creation once. - Retry log reading from integration k8s clusters once. - Add kind cluster creation debug logging. - Add a GitHub Actions status badge to the top of README.md Signed-off-by: Andrew Seigner <[email protected]>

PR #3339 introduced a GitHub Actions CI workflow. Booting 6 clusters simultaneously (3x Github Actions + 3x Travis) exhibits some transient failures. Implement fixes in GitHub Actions and integration tests to address kind cluster creation and testing: - Retry kind cluster creation once. - Retry log reading from integration k8s clusters once. - Add kind cluster creation debug logging. - Add a GitHub Actions status badge to the top of `README.md`. Signed-off-by: Andrew Seigner <[email protected]>

PR #3339 introduced a GitHub Actions CI workflow. Booting 6 clusters simultaneously (3x Github Actions + 3x Travis) exhibits some transient failures. Implement fixes in GitHub Actions and integration tests to address kind cluster creation and testing: - Retry kind cluster creation once. - Retry log reading from integration k8s clusters once. - Add kind cluster creation debug logging. - Add a GitHub Actions status badge to top of `README.md`. Signed-off-by: Andrew Seigner <[email protected]>

siggy added the area/test label Aug 28, 2019

siggy self-assigned this Aug 28, 2019

siggy force-pushed the siggy/action-time branch from 1e213cc to 6bd874c Compare August 28, 2019 19:51

siggy requested review from alpeb, ihcsim, olix0r and grampelberg August 28, 2019 19:56

olix0r reviewed Aug 28, 2019

View reviewed changes

grampelberg reviewed Aug 28, 2019

View reviewed changes

alpeb reviewed Aug 29, 2019

View reviewed changes

.github/workflows/workflow.yml

kind_setup:

strategy:

matrix:

integration_test: [deep, upgrade, helm]

Copy link

Member

alpeb Aug 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎

alpeb reviewed Aug 29, 2019

View reviewed changes

ihcsim reviewed Aug 29, 2019

View reviewed changes

ihcsim approved these changes Aug 29, 2019

View reviewed changes

siggy force-pushed the siggy/action-time branch 2 times, most recently from 172a3e7 to e859815 Compare August 29, 2019 22:14

siggy added 4 commits September 3, 2019 16:24

remove debug output

d89eff0

Signed-off-by: Andrew Seigner <[email protected]>

ivan feedback: kind --wait

e1f4343

Signed-off-by: Andrew Seigner <[email protected]>

always run all cluster cleanup jobs in matrix

d3c0efe

Signed-off-by: Andrew Seigner <[email protected]>

siggy force-pushed the siggy/action-time branch from e859815 to d3c0efe Compare September 3, 2019 23:24

siggy merged commit 4f71b52 into master Sep 4, 2019

siggy deleted the siggy/action-time branch September 4, 2019 00:11

siggy added a commit that referenced this pull request Sep 4, 2019

Add GitHub Actions CI badge to README

8ef0534

PR #3339 introduced a GitHub Actions CI workflow. Add a badge to the top of README.md to report status of the CI workflow. Signed-off-by: Andrew Seigner <[email protected]>

siggy mentioned this pull request Sep 4, 2019

GitHub Actions, kind, integration test logs fixes #3372

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Github Actions CI workflow #3339

Introduce Github Actions CI workflow #3339

siggy commented Aug 28, 2019

olix0r left a comment

siggy commented Aug 28, 2019

grampelberg left a comment

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 29, 2019

ihcsim Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

grampelberg Aug 28, 2019

siggy Aug 29, 2019

l5d-bot commented Aug 28, 2019

alpeb Aug 29, 2019

siggy Aug 29, 2019

alpeb Aug 29, 2019

alpeb left a comment

siggy commented Aug 29, 2019

ihcsim Aug 29, 2019 •

edited

Loading

siggy Aug 29, 2019

ihcsim Aug 29, 2019

siggy commented Aug 29, 2019

ihcsim left a comment

siggy commented Aug 29, 2019 •

edited

Loading

Introduce Github Actions CI workflow #3339

Introduce Github Actions CI workflow #3339

Conversation

siggy commented Aug 28, 2019

olix0r left a comment

Choose a reason for hiding this comment

siggy commented Aug 28, 2019

grampelberg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

l5d-bot commented Aug 28, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alpeb left a comment

Choose a reason for hiding this comment

siggy commented Aug 29, 2019

ihcsim Aug 29, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siggy commented Aug 29, 2019

ihcsim left a comment

Choose a reason for hiding this comment

siggy commented Aug 29, 2019 • edited Loading

ihcsim Aug 29, 2019 •

edited

Loading

siggy commented Aug 29, 2019 •

edited

Loading