This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
sc-consensus-beefy: improve beefy gossip validator #13606
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Old gossip validator was pretty dumb, being very permissive with incoming votes - only condition it had was to be newer than best finalized. New filter conditions: - voter rounds are initialized (discarding votes until voter is actually active), - only votes for current active set id are accepted, - only votes for rounds in the current voting session are accepted, - only votes for GRANDPA finalized blocks are accepted, - when BEEFY voter reaches mandatory round, only votes for said mandatory round are accepted. New validator uses the VoterOracle to easily implement above conditions and only allow through votes that are immediately useful to the voter. After every GRANDPA or BEEFY finality, the gossip validator filter is updated. Signed-off-by: acatangiu <[email protected]>
Since gossip validator will simply disallow votes for future rounds, and only allow votes that the voter can immediately process, there is no need for the voter to enqueue votes. It will see these "future" votes later in rebroadcasts, when voter will also be able to process them. Only at that point does gossip accept and consume them. Signed-off-by: acatangiu <[email protected]>
Move best-beefy and best-grandpa into VoterOracle instead of passing them around as params. VoterOracle ultimately needs to know best-beefy and/or best-grandpa for most of its functions.
Assuming mandatory done in current session: Instead of allowing votes for any round in the current session, only accept votes for rounds equal or better than best BEEFY finalized.
acatangiu
added
A0-please_review
acatangiu
requested review from
andresilva,
svyatonik,
Lederstrumpf and
serban300
14 tasks
Finalize blocks one a time in tests where we want gossip to happen in a certain round. Otherwise, some tasks may be left behind in terms of gossip round numbers because once "scheduled" a task will greedily process as much as possible. This change should be in line with the real-world scenario where voters run "in parallel" across nodes, the only points of synchronization being the finality notifications.
serban300
approved these changes
Mar 15, 2023
|
bot rebase |
|
Rebased |
Lederstrumpf
approved these changes
Mar 16, 2023
There was a problem hiding this comment.
lgtm since dropped "future" votes are just processed when they're rebroadcast, and at a later stage, we can optimise gossip load that "future" votes are only sent to nodes ready to process them, as via discussion with @andresilva on Tuesday.
breathx
pushed a commit
to gear-tech/substrate
that referenced
this pull request
Apr 22, 2023
* sc-consensus-beefy: improve beefy gossip validator Old gossip validator was pretty dumb, being very permissive with incoming votes - only condition it had was to be newer than best finalized. New filter conditions: - voter rounds are initialized (discarding votes until voter is actually active), - only votes for current active set id are accepted, - only votes for rounds in the current voting session are accepted, - only votes for GRANDPA finalized blocks are accepted, - when BEEFY voter reaches mandatory round, only votes for said mandatory round are accepted. New validator uses the VoterOracle to easily implement above conditions and only allow through votes that are immediately useful to the voter. After every GRANDPA or BEEFY finality, the gossip validator filter is updated. * sc-consensus-beefy: remove votes enqueueing Since gossip validator will simply disallow votes for future rounds, and only allow votes that the voter can immediately process, there is no need for the voter to enqueue votes. It will see these "future" votes later in rebroadcasts, when voter will also be able to process them. Only at that point does gossip accept and consume them. * sc-consensus-beefy: refactor persistent state Move best-beefy and best-grandpa into VoterOracle instead of passing them around as params. VoterOracle ultimately needs to know best-beefy and/or best-grandpa for most of its functions. * sc-consensus-beefy: further restrict gossip validator Assuming mandatory done in current session: Instead of allowing votes for any round in the current session, only accept votes for rounds equal or better than best BEEFY finalized. * sc-consensus-beefy: add a couple of comments * sc-consensus-beefy: fix tests involving multiple tasks Finalize blocks one a time in tests where we want gossip to happen in a certain round. Otherwise, some tasks may be left behind in terms of gossip round numbers because once "scheduled" a task will greedily process as much as possible. This change should be in line with the real-world scenario where voters run "in parallel" across nodes, the only points of synchronization being the finality notifications. * sc-consensus-beefy: address review comments --------- Signed-off-by: acatangiu <[email protected]>
15 tasks
nathanwhit
pushed a commit
to nathanwhit/substrate
that referenced
this pull request
Jul 19, 2023
* sc-consensus-beefy: improve beefy gossip validator Old gossip validator was pretty dumb, being very permissive with incoming votes - only condition it had was to be newer than best finalized. New filter conditions: - voter rounds are initialized (discarding votes until voter is actually active), - only votes for current active set id are accepted, - only votes for rounds in the current voting session are accepted, - only votes for GRANDPA finalized blocks are accepted, - when BEEFY voter reaches mandatory round, only votes for said mandatory round are accepted. New validator uses the VoterOracle to easily implement above conditions and only allow through votes that are immediately useful to the voter. After every GRANDPA or BEEFY finality, the gossip validator filter is updated. * sc-consensus-beefy: remove votes enqueueing Since gossip validator will simply disallow votes for future rounds, and only allow votes that the voter can immediately process, there is no need for the voter to enqueue votes. It will see these "future" votes later in rebroadcasts, when voter will also be able to process them. Only at that point does gossip accept and consume them. * sc-consensus-beefy: refactor persistent state Move best-beefy and best-grandpa into VoterOracle instead of passing them around as params. VoterOracle ultimately needs to know best-beefy and/or best-grandpa for most of its functions. * sc-consensus-beefy: further restrict gossip validator Assuming mandatory done in current session: Instead of allowing votes for any round in the current session, only accept votes for rounds equal or better than best BEEFY finalized. * sc-consensus-beefy: add a couple of comments * sc-consensus-beefy: fix tests involving multiple tasks Finalize blocks one a time in tests where we want gossip to happen in a certain round. Otherwise, some tasks may be left behind in terms of gossip round numbers because once "scheduled" a task will greedily process as much as possible. This change should be in line with the real-world scenario where voters run "in parallel" across nodes, the only points of synchronization being the finality notifications. * sc-consensus-beefy: address review comments --------- Signed-off-by: acatangiu <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes memory leak while waiting for pallet activation.
Old gossip validator was pretty dumb, being very permissive with incoming votes - only condition it had was to be newer than best finalized. It was accepting gossip messages and bloating memory even when voter wasn't yet running.
New filter conditions, accept only:
round <= best_grandpa),best_beefy <= round),New validator uses the
VoterOracleto easily implement above conditions and only allow through votes that are immediately useful to the voter.With the new gossip validator, the voter doesn't need to enqueue votes anymore, so removed that logic too. Any "future" votes are discarded to be seen later in rebroadcasts.
After every GRANDPA or BEEFY finality, the gossip validator filter is updated.
These changes also reduce the amount of BEEFY gossip messages flying around.