Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose --index_address_statuses setting and improve first sync performance #35

Merged
merged 15 commits into from
May 18, 2022

Conversation

jackrobison
Copy link
Member

@jackrobison jackrobison commented May 11, 2022

  • adds --index_address_statuses flag, off by default. This is used to turn on/off the index of status hashes for addresses added in Precomputed hashx status #17 to allow scribe to run with fewer resources necessary. The index adds ~10gb to the database, it makes processing new blocks slower but makes handling the notifications for the addresses touched by transactions in the block much faster.
  • add internal caches to handle updating address statuses and reading address histories faster without needing --cache_all_tx_hashes to be on
  • sync the address status index ( --index_address_statuses ) when caught up after startup, including after first initial-sync.
  • refactors and combines updating and compacting the address history to be more performant, running in one pass, and use the rocksdb multi_get api

@jackrobison jackrobison force-pushed the optional-address-history-status branch from faf46ea to f747637 Compare May 17, 2022 15:35
@jackrobison jackrobison force-pushed the optional-address-history-status branch from 4eefd75 to 25a8c6b Compare May 17, 2022 16:04
@jackrobison jackrobison linked an issue May 17, 2022 that may be closed by this pull request
@jackrobison jackrobison merged commit 3b78508 into master May 18, 2022
@jackrobison jackrobison deleted the optional-address-history-status branch May 18, 2022 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants