Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(stream): remove Executor::clear_cache #1820

Merged
merged 1 commit into from
Apr 13, 2022

Conversation

TennyZhuang
Copy link
Contributor

Signed-off-by: TennyZhuang [email protected]

What's changed and what's your intention?

It's impossible to implement clear_cache in ExecutorV2, we may investigate another way.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@codecov
Copy link

codecov bot commented Apr 13, 2022

Codecov Report

Merging #1820 (815e1fe) into main (59b6898) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1820      +/-   ##
==========================================
+ Coverage   70.75%   70.77%   +0.02%     
==========================================
  Files         607      607              
  Lines       79461    79411      -50     
==========================================
- Hits        56220    56203      -17     
+ Misses      23241    23208      -33     
Flag Coverage Δ
rust 70.77% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...am/src/executor/managed_state/top_n/top_n_state.rs 89.86% <ø> (+1.41%) ⬆️
src/stream/src/executor_v2/agg.rs 97.14% <ø> (+7.66%) ⬆️
src/stream/src/executor_v2/global_simple_agg.rs 97.05% <ø> (+3.91%) ⬆️
src/stream/src/executor_v2/hash_agg.rs 95.58% <ø> (+1.21%) ⬆️
src/stream/src/executor_v2/lookup.rs 25.00% <ø> (+5.00%) ⬆️
src/stream/src/executor_v2/mod.rs 65.21% <ø> (+3.99%) ⬆️
src/stream/src/executor_v2/top_n.rs 91.79% <ø> (+1.02%) ⬆️
src/stream/src/executor_v2/top_n_appendonly.rs 92.39% <ø> (+1.21%) ⬆️
src/meta/src/hummock/mock_hummock_meta_client.rs 46.66% <0.00%> (-5.34%) ⬇️
src/storage/src/hummock/local_version_manager.rs 83.00% <0.00%> (-4.67%) ⬇️
... and 2 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

self.managed_lowest_state.clear_cache();
self.managed_middle_state.clear_cache();
self.managed_highest_state.clear_cache();
self.first_execution = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite suspicious. Why setting first_execution here before? Could that be related to @BugenZhao 's recently discovered bug?

By the way, we can remove no cache e2e test in CI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HashJoin has not been migrated to executor v2 and its clear_cache still makes sense. Let's keep it for a while.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the first_execution indicates whether we need to fetch the state from storage and was necessary when writing this. 🤣 I'll investigate into TopN in next days. cc @lmatz

@BugenZhao BugenZhao merged commit 747d2d1 into main Apr 13, 2022
@BugenZhao BugenZhao deleted the refactor/remove-clear-cache branch April 13, 2022 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants