Marks old storages as dirty and uncleaned in clean_accounts() #3737
Conversation
brooksprumo
force-pushed
the
excessive-storages/add-old-slots-to-uncleaned-roots-after-revert
branch
from
8ba7606
to
132b2f4
Compare
|
Backports to the stable branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. |
|
Backports to the beta branch are to be avoided unless absolutely necessary for fixing bugs, security issues, and perf regressions. Changes intended for backport should be structured such that a minimum effective diff can be committed separately from any refactoring, plumbing, cleanup, etc that are not strictly necessary to achieve the goal. Any of the latter should go only into master and ride the normal stabilization schedule. Exceptions include CI/metrics changes, CLI improvements and documentation updates on a case by case basis. |
|
Force-pushed to rebase now that #3735 has merged. No code changes were done. |
brooksprumo
deleted the
excessive-storages/add-old-slots-to-uncleaned-roots-after-revert
branch
(cherry picked from commit 31742ca) # Conflicts: # accounts-db/src/accounts_db.rs # accounts-db/src/accounts_db/tests.rs # runtime/src/bank.rs
(cherry picked from commit 31742ca) # Conflicts: # accounts-db/src/accounts_db/tests.rs
…backport of #3737) (#3748) * Marks old storages as dirty and uncleaned in clean_accounts() (#3737) (cherry picked from commit 31742ca) # Conflicts: # accounts-db/src/accounts_db/tests.rs * fixes merge conflict --------- Co-authored-by: Brooks <[email protected]>
…backport of #3737) (#3747) * Marks old storages as dirty and uncleaned in clean_accounts() (#3737) (cherry picked from commit 31742ca) # Conflicts: # accounts-db/src/accounts_db.rs # accounts-db/src/accounts_db/tests.rs # runtime/src/bank.rs * fixup merge conflict --------- Co-authored-by: Brooks <[email protected]>
Problem
Copied from #3702
We do not clean up old storages.
More context: when calculating a full accounts hash, we call
mark_old_slots_as_dirty()as a way to ensure we do not forget or miss cleaning up really old storages (i.e. ones that are older than an epoch old). But, when we enable skipping rewrites, we don't want to clean up those old storages, as they'll intentionally be treated as ancient append vecs. So insidemark_old_slots_as_dirty()we conditionally mark old slots as dirty. This is based on the value ofancient_append_vec_offset, which should be None unless ancient append vecs are enabled.Unfortunately, normal running validators, we end up never marking old slots as dirty, because the ancient append vec offset is always Some. And thus we don't clean up old storages.
Summary of Changes
Mark old storages as dirty, and add to the uncleaned roots list in clean_accounts().
We still check if ancient append vecs are enabled, but not with the
ancient_append_vec_offset. Instead we look at the skipping rewrites feature gate and the cli arg.By moving this marking into clean_accounts(), we also decouple it from accounts hash calculation, which is not necessary anymore. This also removes behavioral differences based on if snapshots are enabled or not.