Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use entry size as weight in block and meta cache, export cache capacity config #1512

Merged
merged 3 commits into from
Apr 1, 2022

Conversation

MrCroxx
Copy link
Contributor

@MrCroxx MrCroxx commented Apr 1, 2022

What's changed and what's your intention?

  • Use cache entry size as weight in block cache and meta cache instead entry count.
  • Export sstable block cache and meta cache config as:
storage.block_cache_capacity
storage.meta_cache_capacity

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@MrCroxx MrCroxx requested review from twocode, hzxa21 and BugenZhao April 1, 2022 04:50
@MrCroxx MrCroxx self-assigned this Apr 1, 2022
@codecov
Copy link

codecov bot commented Apr 1, 2022

Codecov Report

Merging #1512 (67ef6ae) into main (7d7a4b3) will increase coverage by 0.01%.
The diff coverage is 94.66%.

@@             Coverage Diff              @@
##               main    #1512      +/-   ##
============================================
+ Coverage     70.45%   70.47%   +0.01%     
  Complexity     2766     2766              
============================================
  Files          1031     1031              
  Lines         90539    90607      +68     
  Branches       1790     1790              
============================================
+ Hits          63793    63856      +63     
- Misses        25855    25860       +5     
  Partials        891      891              
Flag Coverage Δ
java 61.01% <ø> (ø)
rust 72.48% <94.66%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
rust/bench/ss_bench/main.rs 0.96% <0.00%> (-0.02%) ⬇️
rust/storage/src/store_impl.rs 6.15% <0.00%> (-0.20%) ⬇️
rust/common/src/config.rs 77.06% <100.00%> (+1.81%) ⬆️
rust/storage/src/hummock/block_cache.rs 88.23% <100.00%> (+1.13%) ⬆️
rust/storage/src/hummock/iterator/test_utils.rs 96.82% <100.00%> (+0.10%) ⬆️
...src/hummock/shared_buffer/shared_buffer_manager.rs 97.15% <100.00%> (+0.01%) ⬆️
rust/storage/src/hummock/snapshot_tests.rs 95.75% <100.00%> (+0.13%) ⬆️
rust/storage/src/hummock/sstable/mod.rs 93.15% <100.00%> (+1.41%) ⬆️
rust/storage/src/hummock/sstable_store.rs 87.30% <100.00%> (+1.33%) ⬆️
rust/storage/src/hummock/test_utils.rs 80.95% <100.00%> (+0.46%) ⬆️
... and 2 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

Copy link
Contributor

@twocode twocode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the rest. Cool!

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -52,6 +52,8 @@ pub fn mock_sstable_store_with_object_store(object_store: ObjectStoreRef) -> Sst
object_store,
path,
Arc::new(StateStoreMetrics::unused()),
64 << 20,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may use default::block_cache_capacity() here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or default_config_for_test().block_cache_capacity. 🤣

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The differences between these two configs are big. Let's maintain them separately for now. I think we still need to refactor the test utils later.

@MrCroxx MrCroxx enabled auto-merge (squash) April 1, 2022 06:21
@MrCroxx MrCroxx merged commit 4d11fe4 into main Apr 1, 2022
@MrCroxx MrCroxx deleted the xx/cache-config branch April 1, 2022 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants