Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: bring back function set_cache_capacity #17196

Merged
merged 7 commits into from
Jan 8, 2025
Merged

chore: bring back function set_cache_capacity #17196

merged 7 commits into from
Jan 8, 2025

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Jan 7, 2025 •
edited
Loading

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

bring back function system$set_cache_capacity (#16016)

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-chore this PR only has small changes that no need to record, like coding styles. label Jan 7, 2025
@dantengsky dantengsky changed the title chore:bring back function set_cache_capacity chore: bring back function set_cache_capacity Jan 8, 2025
@dantengsky dantengsky force-pushed the chore-bring-back-cache-capacity branch from 7e4c12e to c64143c Compare January 8, 2025 06:51
@dantengsky dantengsky force-pushed the chore-bring-back-cache-capacity branch from c64143c to 8f81869 Compare January 8, 2025 07:42
@dantengsky dantengsky force-pushed the chore-bring-back-cache-capacity branch from 8f81869 to 7580cd9 Compare January 8, 2025 07:57
@dantengsky dantengsky added the ci-benchmark Benchmark: run all test label Jan 8, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

Docker Image for PR

  • tag: pr-17196-ff26c0d-1736325363

note: this image tag is only available for internal use,
please check the internal doc for more details.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

ClickBench Report

@dantengsky dantengsky added ci-benchmark Benchmark: run all test and removed ci-benchmark Benchmark: run all test labels Jan 8, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

Docker Image for PR

  • tag: pr-17196-53413c6-1736332879

note: this image tag is only available for internal use,
please check the internal doc for more details.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 8, 2025

ClickBench Report

@dantengsky dantengsky marked this pull request as ready for review January 8, 2025 13:55
@dantengsky dantengsky added this pull request to the merge queue Jan 8, 2025
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Jan 8, 2025
@BohuTANG BohuTANG merged commit fa209f4 into databendlabs:main Jan 8, 2025
90 checks passed
@wubx
Copy link
Member

wubx commented Jan 9, 2025

Is there a docs for this function?

BTW:
Use system$set_cache_capacity set parameters is persistent?

@BohuTANG
Copy link
Member

BohuTANG commented Jan 9, 2025

Is there a docs for this function?

BTW: Use system$set_cache_capacity set parameters is persistent?

This is used by internal not for user.

@dantengsky
Copy link
Member Author

Is there a docs for this function?

BTW: Use system$set_cache_capacity set parameters is persistent?

This function is mainly designed to help cloud operators, or in private deployment scenarios, find the right cache capacity for a given workload: balancing execution performance, query soft OOM, and spill.

Once the right cache capacity is found, still need to adjust the cache size in the config to make the capacity setting persistent, ensuring it remains effective during warehouse suspend and resume operations.

dantengsky added a commit to dantengsky/fuse-query that referenced this pull request Jan 21, 2025
@dantengsky
chore: bring back function set_cache_capacity (databendlabs#17196)
9333dd7 
* chore:bring back func `set_cache_capacity`

* test cases

* revert config file

* tweak test case

* fix: use usize as bytes capacity

* tweak lock scope

* rename vars/methods of cache mgr
@dantengsky dantengsky mentioned this pull request Jan 21, 2025
Merged
11 tasks
dantengsky added a commit that referenced this pull request Jan 22, 2025
@dantengsky
chore: backport set cache capacity to rel/v1.2.636-rc8.2 (#17343)
63cf08b 
* chore: bring back function `set_cache_capacity` (#17196)

* chore:bring back func `set_cache_capacity`

* test cases

* revert config file

* tweak test case

* fix: use usize as bytes capacity

* tweak lock scope

* rename vars/methods of cache mgr

* resovle pick conflicts

* cargo fmt

* tweak audit.toml, bypass RUSTSEC-2025-0001

* pin patched opendal to tag, instead of commit hash

* taplo fmt

* tweak golden file of ut test_caches_table
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SkyFan2002 SkyFan2002 SkyFan2002 approved these changes

@zhang2014 zhang2014 zhang2014 approved these changes

Assignees
No one assigned
Labels
ci-benchmark Benchmark: run all test pr-chore this PR only has small changes that no need to record, like coding styles.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@dantengsky @wubx @BohuTANG @zhang2014 @SkyFan2002