Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cache): file cache cleanup #20061

Merged
merged 2 commits into from
Feb 1, 2023
Merged

feat(cache): file cache cleanup #20061

merged 2 commits into from
Feb 1, 2023

Conversation

rarkins
Copy link
Collaborator

@rarkins rarkins commented Jan 27, 2023

Changes

Checks file cache for expired items at the end of a run.

Context

Closes #13732

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

@rarkins rarkins requested review from secustor and viceice January 27, 2023 06:28
@rarkins
Copy link
Collaborator Author

rarkins commented Jan 27, 2023

I'll add tests once gotten positive feedback on the implementation and whether it should be an opt-in flag at first. I think it seems safe enough seeing as it's wrapped in a try/catch

Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be very time consuming for very big caches

lib/util/cache/package/file.ts Outdated Show resolved Hide resolved
lib/util/cache/package/file.ts Outdated Show resolved Hide resolved
@rarkins
Copy link
Collaborator Author

rarkins commented Jan 28, 2023

It's at the end of the run and it needs to be cleaned up sometime

@rarkins rarkins requested a review from viceice January 29, 2023 07:25
@rarkins
Copy link
Collaborator Author

rarkins commented Jan 29, 2023

can be very time consuming for very big caches

I think it needs to be done though. And it cleans at the end of a Renovate run.

Worst case:

  • It times out or they kill it manually
  • It will have cleaned out X entries by that time
  • It will have less to do next time

We're also releasing it in a major so if it somehow makes it impossible to use for some, they can hold off upgrading and raise an issue (or just wipe their cache)

@rarkins rarkins marked this pull request as ready for review January 29, 2023 07:28
@rarkins rarkins changed the base branch from main to v35 January 29, 2023 07:28
Copy link
Member

@viceice viceice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something wrong with your merge. too many changes here

@rarkins rarkins force-pushed the feat/13732-cacache-cleanup branch from e7cda43 to 80dbc79 Compare January 31, 2023 10:26
@rarkins rarkins requested a review from viceice January 31, 2023 10:26
@rarkins
Copy link
Collaborator Author

rarkins commented Jan 31, 2023

Will add tests

@rarkins rarkins marked this pull request as draft January 31, 2023 10:53
@rarkins rarkins marked this pull request as ready for review February 1, 2023 11:47
@rarkins rarkins merged commit 6347e0f into v35 Feb 1, 2023
@rarkins rarkins deleted the feat/13732-cacache-cleanup branch February 1, 2023 11:48
rarkins added a commit that referenced this pull request Feb 4, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 10, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 17, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 20, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 23, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 25, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
rarkins added a commit that referenced this pull request Feb 27, 2023
Checks file cache for expired items at the end of a run. Non-breaking change but it may result in some long cleanup jobs for any bots which have been left to populate their package cache for a long time.

Closes #13732
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide mechanism or documentation for cache cleanup
2 participants