fix(core): fix duplicate mutation entries for count index #9208
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
area/testing
harshil-goel
added a commit
that referenced
this pull request
Nov 5, 2024
Due to duplicate entries in the count index, sometimes we had wrong count being reported. This wrong count was causing the transcation is too old issue. This diff fixes the duplicate entries fixing the issue.
mangalaman93
pushed a commit
that referenced
this pull request
Dec 6, 2024
Currently the delta postings are stored in a map. When we use the cache, these deltas gets unnecessarily copied again and again because of the limitations of the map. We have introduced a new struct for holding the mutable layer, that would allow us to work on the list of deltas. With the introduction of this, we can now pass around posting lists from one transaction to another without copying the map. We also store results such as delete markers, uid to posting map, length to improve query performance. This PR also fixes bugs in the count index introduced in the PR #9208.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For calculating length, we want to be able to just look at the mutable layer and not check immutable layer to improve time. For this we had an assumption that we won't have duplicate mutations stored. Turns out, the assumption is only try for index keys, not for the data keys. This PR fixes it for count keys also.
Since duplicate keys can be present in the database right now for people using it from before, we advise to rebuild count index after this upgrade.