Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): Delete mainnet and testnet disks separately #6582

Merged
merged 4 commits into from
Apr 28, 2023

Conversation

teor2345
Copy link
Contributor

@teor2345 teor2345 commented Apr 27, 2023

Motivation

In PR #6581, I started creating testnet cached state disks.

But the current disk cleanup job will treat them like mainnet disks, causing CI bugs like:

  • CI will take 3 days to run if all the recent disks are testnet, and all the mainnet disks are deleted
  • CI will fail if all the mainnet disks are deleted in the middle of a job

This is part of #6367.

To avoid these bugs, I temporarily disabled the disk deletion workflow in GitHub Actions.

Solution

Delete the mainnet and testnet disks separately, so each network always keeps a few disks.

Review

This is an urgent fix because it could stop all PRs merging for 3 days.

Here's a manual run of this workflow from this branch, after PR #6581 generated some testnet disks:
https://github.com/ZcashFoundation/zebra/actions/runs/4824829115

Reviewer Checklist

  • Will the PR name make sense to users?
    • Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
  • Are the PR labels correct?
  • Does the code do what the ticket and PR says?
    • Does it change concurrent code, unsafe code, or consensus rules?
  • How do you know it works? Does it have tests?

Follow Up Work

Re-enable disk deletion by turning this action back on:
https://github.com/ZcashFoundation/zebra/actions/workflows/delete-gcp-resources.yml

@teor2345 teor2345 added C-bug Category: This is a bug A-devops Area: Pipelines, CI/CD and Dockerfiles P-High 🔥 I-integration-fail Continuous integration fails, including build and test failures labels Apr 27, 2023
@teor2345 teor2345 self-assigned this Apr 27, 2023
@teor2345 teor2345 requested a review from a team as a code owner April 27, 2023 01:04
@teor2345 teor2345 requested review from dconnolly and removed request for a team April 27, 2023 01:04
@github-actions github-actions bot added the C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG label Apr 27, 2023
@teor2345 teor2345 added the I-cost Zebra infrastructure costs label Apr 27, 2023
@teor2345 teor2345 mentioned this pull request Apr 27, 2023
8 tasks
Co-authored-by: Deirdre Connolly <[email protected]>
@teor2345 teor2345 requested a review from dconnolly April 27, 2023 05:54
@teor2345
Copy link
Contributor Author

Anyone can re-approve this PR, Deirdre has already reviewed it.

Copy link
Member

@gustavovalverde gustavovalverde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

mergify bot added a commit that referenced this pull request Apr 28, 2023
@mergify mergify bot merged commit 1f639ff into main Apr 28, 2023
@mergify mergify bot deleted the ci-delete-testnet-separately branch April 28, 2023 02:26
@oxarbitrage oxarbitrage mentioned this pull request May 9, 2023
38 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles C-bug Category: This is a bug C-trivial Category: A trivial change that is not worth mentioning in the CHANGELOG I-cost Zebra infrastructure costs I-integration-fail Continuous integration fails, including build and test failures
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants