Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion for Fix the deduplication of note commitment trees #7391

Merged
merged 4 commits into from
Aug 28, 2023

Conversation

teor2345
Copy link
Contributor

@teor2345 teor2345 commented Aug 28, 2023

Motivation

This PR contains two different suggestions to fix bugs in PR #7379.

Alternative Solutions

Minimal Changes

Commit d70a233 contains a minor refactor that simplifies the code to handle the initial tip edge case, regardless of whether it is the same as the genesis tree or not. It also renames some variables to make them clearer.

Complete Rewrite

Commit b97303f rewrites and simplifies the inner loop so these edge cases never occur, by removing the range deletes entirely.

This performs almost as well as ranged deletes on a legacy state with 1719530 blocks, taking 16 seconds to do the upgrade:

2023-08-28T01:29:44.261735Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: trying to open older database format: launching upgrade task running_version=Version { major: 25, minor: 1, 
patch: 1 } disk_version=Version { major: 25, minor: 0, patch: 0 }             
...
2023-08-28T01:30:01.122526Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: marked database format as upgraded running_version=Version { major: 25, minor: 1, patch: 1 } format_upgrade_
version=Version { major: 25, minor: 1, patch: 1 } disk_version=Version { major: 25, minor: 0, patch: 0 }

Review

This PR is a suggestion for @upbqdn on PR #7379.

Feel free to choose either suggestion!

@teor2345 teor2345 self-assigned this Aug 28, 2023
@teor2345 teor2345 requested a review from upbqdn August 28, 2023 01:38
@teor2345 teor2345 changed the base branch from main to fix-tree-dedup August 28, 2023 01:58
@teor2345 teor2345 marked this pull request as ready for review August 28, 2023 04:14
@teor2345 teor2345 requested a review from a team as a code owner August 28, 2023 04:14
Copy link
Contributor

@arya2 arya2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@upbqdn
Copy link
Member

upbqdn commented Aug 28, 2023

@Mergifyio update.

@mergify
Copy link
Contributor

mergify bot commented Aug 28, 2023

update .

✅ Branch has been successfully updated

Copy link
Member

@upbqdn upbqdn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I'm happy to remove the batching.

@upbqdn upbqdn merged commit 2db0835 into fix-tree-dedup Aug 28, 2023
@upbqdn upbqdn deleted the fix-tree-dedup-suggestion branch August 28, 2023 18:58
mergify bot added a commit that referenced this pull request Aug 28, 2023
* Log errors and panic if duplicate trees are found after the de-duplicate upgrade

* Always check for duplicates, even if the state is already marked as upgraded

* Minor doc fixes

* Document ranges for `zs_delete_range`

* Revert the comment for `sapling_tree`

* Rearrange tree methods & fix their docs

* Bump DATABASE_FORMAT_PATCH_VERSION from 0 to 1

* Remove the manual tree deletion at early heights

* Add `skip_while` to `zs_range_iter`

* Refactor the tree deduplication

* Add comments to the pruning

* Turn warnings into panics

* Remove redundant checks

These checks are superseded by `check_for_duplicate_trees`

* Remove an edge case that ignored the last tree

* Suggestion for Fix the deduplication of note commitment trees (#7391)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

---------

Co-authored-by: teor <[email protected]>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants