-
Notifications
You must be signed in to change notification settings - Fork 600
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
consistent-hash: support reading by single vnode for specific states #3159
Comments
So we'll need to compute hash before every get and prefix scan in storage? |
For MV, maybe yes. For other states, we could directly use the pre-computed vnode. |
How to pre-compute the VNode of a get/prefix_scan key beforehand? We don't know the key until we need to read it. For internal state (i.e. non-MV state), I think compure hash before every get/prefix_scan is okay since we will only hit one VNode. |
I think this issue mainly focuses on the internal state of HashAgg & HashJoin. They both hold a bunch of |
Currently, all states will maintain a set of vnodes, which are stored in the state's keyspace. However, every state of hash agg and hash join will only query data that belongs to one vnode. Therefore, we could narrow the range of vnodes when reading data from storage, i.e., only pass in one vnode instead of a set of vnodes.
The vnode that belongs to a state could be computed when the state is created.
The text was updated successfully, but these errors were encountered: