[Enhancement] Adjust calculation method for page cache mem usage metrics #37740

zaorangyang · 2023-12-25T11:54:35Z

Why I'm doing:
The memory usage of the page cache tracked by MemTracker may exceed the maximum available memory of the page cache. This could be problematic for users.

What I'm doing:
Make the size tracked by the memory allocator as the size occupied in the page cache. The new strategy results in starrocks_be_storage_page_cache_mem_bytes being slightly larger than starrocks_be_page_cache_capacity, as starrocks_be_page_cache_capacity does not include the metadata of the page cache itself.

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

…etrics Signed-off-by: Zaorang Yang <[email protected]>

Signed-off-by: Zaorang Yang <[email protected]>

be/src/util/lru_cache.h

github-actions · 2024-01-02T09:32:53Z

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2024-01-02T09:33:32Z

[BE Incremental Coverage Report]

✅ pass : 9 / 9 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	src/util/lru_cache.cpp	9	9	100.00%	[]

github-actions · 2024-01-03T08:23:21Z

@Mergifyio backport branch-3.2

github-actions · 2024-01-03T08:23:23Z

@Mergifyio backport branch-3.1

github-actions · 2024-01-03T08:23:25Z

@Mergifyio backport branch-3.0

mergify · 2024-01-03T08:23:25Z

backport branch-3.2

✅ Backports have been created

#38338 [Enhancement] Adjust calculation method for page cache mem usage metrics (backport #37740) has been created for branch branch-3.2

mergify · 2024-01-03T08:23:26Z

backport branch-3.1

✅ Backports have been created

#38339 [Enhancement] Adjust calculation method for page cache mem usage metrics (backport #37740) has been created for branch branch-3.1

github-actions · 2024-01-03T08:23:26Z

@Mergifyio backport branch-2.5

mergify · 2024-01-03T08:23:29Z

backport branch-3.0

✅ Backports have been created

#38340 [Enhancement] Adjust calculation method for page cache mem usage metrics (backport #37740) has been created for branch branch-3.0 but encountered conflicts

mergify · 2024-01-03T08:23:31Z

backport branch-2.5

✅ Backports have been created

#38342 [Enhancement] Adjust calculation method for page cache mem usage metrics (backport #37740) has been created for branch branch-2.5 but encountered conflicts

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9)

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9) # Conflicts: # be/src/util/lru_cache.cpp

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9)

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9) # Conflicts: # be/src/util/lru_cache.cpp

…ics (backport #37740) (#38339) Co-authored-by: zaorangyang <[email protected]>

…ics (backport #37740) (#38338) Co-authored-by: zaorangyang <[email protected]>

…ics (StarRocks#37740) Signed-off-by: Zaorang Yang <[email protected]>

…ics (backport #37740) (#38638) Signed-off-by: Zaorang Yang <[email protected]>

…ics (backport #37740) (#38342) Co-authored-by: zaorangyang <[email protected]> Co-authored-by: wyb <[email protected]>

…ics (backport #37740) (#38340) Co-authored-by: zaorangyang <[email protected]> Co-authored-by: wyb <[email protected]>

[Enhancement] Adjust calculation method for page cache memory usage m…

caedab4

…etrics Signed-off-by: Zaorang Yang <[email protected]>

github-actions bot added the 3.2 label Dec 25, 2023

mergify bot assigned zaorangyang Dec 25, 2023

github-actions bot added 3.1 3.0 2.5 labels Dec 25, 2023

zaorangyang added 3 commits December 26, 2023 13:50

Update

46127db

Signed-off-by: Zaorang Yang <[email protected]>

Update

ac9d703

Signed-off-by: Zaorang Yang <[email protected]>

Format

45334b2

Signed-off-by: Zaorang Yang <[email protected]>

chaoyli reviewed Dec 26, 2023

View reviewed changes

be/src/util/lru_cache.h Show resolved Hide resolved

trueeyu approved these changes Dec 28, 2023

View reviewed changes

chaoyli approved these changes Dec 29, 2023

View reviewed changes

trueeyu merged commit 65b0fa9 into StarRocks:main Jan 3, 2024
44 checks passed

github-actions bot removed the 3.2 label Jan 3, 2024

github-actions bot removed the 3.1 label Jan 3, 2024

github-actions bot removed 3.0 2.5 labels Jan 3, 2024

mergify bot pushed a commit that referenced this pull request Jan 3, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

a0f8a91

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9)

mergify bot pushed a commit that referenced this pull request Jan 3, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

f3f6b8b

…ics (#37740) Signed-off-by: Zaorang Yang <[email protected]> (cherry picked from commit 65b0fa9)

mergify bot mentioned this pull request Jan 3, 2024

[Enhancement] Adjust calculation method for page cache mem usage metrics (backport #37740) #38342

Merged

18 tasks

wanpengfei-git pushed a commit that referenced this pull request Jan 3, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

5b4fc2e

…ics (backport #37740) (#38339) Co-authored-by: zaorangyang <[email protected]>

wanpengfei-git pushed a commit that referenced this pull request Jan 3, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

4bbabec

…ics (backport #37740) (#38338) Co-authored-by: zaorangyang <[email protected]>

github-actions bot added 3.1-merged 3.2-merged labels Jan 3, 2024

zaorangyang added a commit to zaorangyang/starrocks that referenced this pull request Jan 8, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

b720020

…ics (StarRocks#37740) Signed-off-by: Zaorang Yang <[email protected]>

trueeyu pushed a commit that referenced this pull request Jan 8, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

07e0888

…ics (backport #37740) (#38638) Signed-off-by: Zaorang Yang <[email protected]>

github-actions bot added the 2.5-merged label Jan 8, 2024

wanpengfei-git pushed a commit that referenced this pull request Jan 31, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

e7b693c

…ics (backport #37740) (#38342) Co-authored-by: zaorangyang <[email protected]> Co-authored-by: wyb <[email protected]>

wanpengfei-git pushed a commit that referenced this pull request Jan 31, 2024

[Enhancement] Adjust calculation method for page cache mem usage metr…

d828bbb

…ics (backport #37740) (#38340) Co-authored-by: zaorangyang <[email protected]> Co-authored-by: wyb <[email protected]>

github-actions bot added the 3.0-merged label Jan 31, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement] Adjust calculation method for page cache mem usage metrics #37740

[Enhancement] Adjust calculation method for page cache mem usage metrics #37740

zaorangyang commented Dec 25, 2023 •

edited

Loading

github-actions bot commented Jan 2, 2024

github-actions bot commented Jan 2, 2024

github-actions bot commented Jan 3, 2024

github-actions bot commented Jan 3, 2024

github-actions bot commented Jan 3, 2024

mergify bot commented Jan 3, 2024 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading

github-actions bot commented Jan 3, 2024

mergify bot commented Jan 3, 2024 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading

[Enhancement] Adjust calculation method for page cache mem usage metrics #37740

[Enhancement] Adjust calculation method for page cache mem usage metrics #37740

Conversation

zaorangyang commented Dec 25, 2023 • edited Loading

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

github-actions bot commented Jan 2, 2024

[FE Incremental Coverage Report]

github-actions bot commented Jan 2, 2024

[BE Incremental Coverage Report]

file detail

github-actions bot commented Jan 3, 2024

github-actions bot commented Jan 3, 2024

github-actions bot commented Jan 3, 2024

mergify bot commented Jan 3, 2024 • edited Loading

✅ Backports have been created

mergify bot commented Jan 3, 2024 • edited Loading

✅ Backports have been created

github-actions bot commented Jan 3, 2024

mergify bot commented Jan 3, 2024 • edited Loading

✅ Backports have been created

mergify bot commented Jan 3, 2024 • edited Loading

✅ Backports have been created

zaorangyang commented Dec 25, 2023 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading

mergify bot commented Jan 3, 2024 •

edited

Loading