Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEAM #1177- Add script to cleanup task definitions in GHA workflow #2204

Merged
merged 2 commits into from
Dec 26, 2024

Conversation

coreycarvalho
Copy link

@coreycarvalho coreycarvalho commented Dec 26, 2024

Description

This PR introduces a GHA workflow that deregisters all api and celery task definitions except the most recent 10 (for each family). The workflow runs on a cron, every Sunday at 00:00 UTC. There is also a workflow dispatch option that contains a dry run for testing purposes.

issue TEAM 1177

How Has This Been Tested?

As of right now, only dry run has been tested. I am going to test this now without dry run, and I am expecting it to fail on API limits being hit in AWS. Failed workflows on API limits are acceptable for now while we churn through the thousands of task definitions we have to deregister.

Once we churn through those, running this weekly should not trigger API limits.

Dry run: https://github.com/department-of-veterans-affairs/notification-api/actions/runs/12505543172/job/34889065650#step:4:130

Current workflow run deleting task definitions: https://github.com/department-of-veterans-affairs/notification-api/actions/runs/12505695741

Checklist

  • I have assigned myself to this PR
  • PR has an appropriate title: #9999 - What the thing does
  • PR has a detailed description, including links to specific documentation
  • I have added the appropriate labels to the PR.
  • I did not remove any parts of the template, such as checkboxes even if they are not used
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to any documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works. Testing guidelines
  • I have ensured the latest main is merged into my branch and all checks are green prior to review
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • The ticket was moved into the DEV test column when I began testing this change

@coreycarvalho coreycarvalho force-pushed the 1177-task-def-cleanup branch 5 times, most recently from 31b19a1 to 0ca4b98 Compare December 26, 2024 15:13
@coreycarvalho coreycarvalho marked this pull request as ready for review December 26, 2024 15:39
@coreycarvalho coreycarvalho requested a review from a team as a code owner December 26, 2024 15:39
@coreycarvalho coreycarvalho self-assigned this Dec 26, 2024
@coreycarvalho coreycarvalho changed the title Add script to cleanup task definitions in GHA workflow TEAM #1177- Add script to cleanup task definitions in GHA workflow Dec 26, 2024
k-macmillan
k-macmillan previously approved these changes Dec 26, 2024
@coreycarvalho
Copy link
Author

Latest commit adds both jitter and exponential backoff so that we don't hit rate limits in AWS.

@coreycarvalho coreycarvalho merged commit 402e961 into main Dec 26, 2024
14 checks passed
@coreycarvalho coreycarvalho deleted the 1177-task-def-cleanup branch December 26, 2024 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants