feat(validator): listen and get metadata only for committee validators #1787

nkryuchkov · 2024-10-10T23:42:08Z

Note: each selfSubnets call takes approx. 300-900 µs

… validators belonging to own committee

y0sher

I know this pr is still immature. quoting the issue here to clarify my original intentions -

- Consider to pull metadata from beacon node only about validators that are part of your listened subnets.
- Reject/ignore all messages that are not part of your subnet validators (this might happen automatically since you won't have their status and duties)

So about first point I see you started it, but your using the ValidatorSubnet calculation which is pre-alan way to find your subnets. We now use CommiteeSubnet and have a different way of dividing our validators to subnets (according to committees).

About second point I don't see that you did anything about message validation, though if we don't have any metadata about this operator's validators than we might just naturally reject/ignore it? please make sure what happens in this case.

nkryuchkov · 2024-10-16T01:32:20Z

So about first point I see you started it, but your using the ValidatorSubnet calculation which is pre-alan way to find your subnets. We now use CommiteeSubnet and have a different way of dividing our validators to subnets (according to committees).

Added post-fork handling, please review again

About second point I don't see that you did anything about message validation, though if we don't have any metadata about this operator's validators than we might just naturally reject/ignore it? please make sure what happens in this case.

In this case the message gets ignored:

	ErrNoShareMetadata                         = Error{text: "share has no metadata"}

operator/validator/controller.go

y0sher

Remove prefork code, as I explained its not needed anymore
to make this PR final we need to POC that it works on stage
Compare cpu mem usage
compare how this impacts the overload on beacon node
We probably need to ignore messages that we don't have duties or its validator we don't see as active (because they're not ours). instead of rejecting them.
@MatheusFranco99 can you chime in in terms of protocol and networking, other than more well spread messages, any reason why ignoring these message will impact us badly?

MatheusFranco99 · 2024-10-16T07:30:07Z

@MatheusFranco99 can you chime in in terms of protocol and networking, other than more well-spread messages, any reason why ignoring these messages will impact us badly?

@y0sher
Do you mean ignoring messages from validators that we think are liquidated / not active / exited or whatever similar? I think it's not a problem to ignore instead of reject. We also have ignoring protection against an attack so we should be fine.

But we must not do this if you meant ignoring messages about validators that we are not part of the committee.

nkryuchkov · 2024-10-17T01:33:45Z

Remove prefork code, as I explained its not needed anymore

done

to make this PR final we need to POC that it works on stage

I'm still testing it but so far it seems to be working well

Compare cpu mem usage

It was deployed between 23:46 and 00:28 and since 01:13

CPU dropped from ~2 to ~0.4-0.5:

Memory might have dropped a little bit but the difference is not that huge, also it differs between runs:

Receive bandwidth dropped from ~1.3 to ~0.4-0.5, transmit bandwidth just a little bit:

Although, for another node the transmit bandwidth difference is significant:

compare how this impacts the overload on beacon node

I don't see any difference

CPU:

Mem:

We probably need to ignore messages that we don't have duties or its validator we don't see as active (because they're not ours). instead of rejecting them.

Well, we ignore messages if there's no metadata, so if they're not ours, we should have no metadata for them, hence the message is ignored

MatheusFranco99 · 2024-10-17T07:32:48Z

Well, we ignore messages if there's no metadata, so if they're not ours, we should have no metadata for them, hence the message is ignored

@nkryuchkov
When you say "if they're not ours", you mean non SSV validators, right? Or do you mean non-committee validators? Or validators from topics we're not subscribed to? 😅
I think that's the part I got a bit confused

Great job!

nkryuchkov · 2024-10-17T12:38:33Z

@MatheusFranco99 sorry for the confusion, I meant validators from topics we're not subscribed to

operator/validator/controller.go

MatusKysel

👍

y0sher · 2024-10-20T12:30:26Z

operator/validator/controller.go

 	for _, share := range shares {
 		if c.operatorDataStore.GetOperatorID() != 0 && share.BelongsToOperator(c.operatorDataStore.GetOperatorID()) {
 			ownShares = append(ownShares, share)
 		}
-		allPubKeys = append(allPubKeys, share.ValidatorPubKey[:])
+		committeeSubnet := byte(commons.CommitteeSubnet(share.CommitteeID()))
+		if bytes.Contains(activeSubnets, []byte{committeeSubnet}) {


I believe we have SharedSubnets and some other subnets utility functions. its best to use a function across the code to make changing and bug fixing easier.

SharedSubnets checks intersections between two subnet sets but we have a single value here, so I think it's not a good fit here. However, there was a bug in my implementation, I don't need bytes.Contains, I can just check the index

However, I think we need to refactor how we work with subnets in future PRs: Create a type for it and add methods

y0sher · 2024-10-20T12:37:00Z

operator/validator/controller.go

@@ -1139,6 +1148,12 @@ func (c *controller) UpdateValidatorMetaDataLoop() {
 			if share.Liquidated {
 				return true
 			}
+
+			committeeSubnet := byte(commons.CommitteeSubnet(share.CommitteeID()))
+			if !bytes.Contains(activeSubnets, []byte{committeeSubnet}) {


operator/validator/controller.go

moshe-blox

LGTM 🔥

fixed a bug and pushed: ac49a96

moshe-blox · 2024-11-17T17:12:11Z

@MatheusFranco99 can you chime in in terms of protocol and networking, other than more well-spread messages, any reason why ignoring these messages will impact us badly?

@y0sher Do you mean ignoring messages from validators that we think are liquidated / not active / exited or whatever similar? I think it's not a problem to ignore instead of reject. We also have ignoring protection against an attack so we should be fine.

But we must not do this if you meant ignoring messages about validators that we are not part of the committee.

@nkryuchkov @y0sher

is this the case now in message validation with this PR?

if we receive a message with committee X for validator that's actually in committee Y (where both aren't our committees), would we reject it for invalid committee or ignore for missing metadata?

which check happens first?

operator/validator/controller.go

# Conflicts: # network/p2p/p2p_setup.go # operator/validator/mocks/controller.go

operator/validator/controller.go

y0sher

check latest comment. we need to make sure we always update all our validators' non liquidated shares

operator/validator/controller.go

y0sher

looks good. can you please reply my question

operator/validator/controller.go

y0sher

gj.
please lets check the resource change for calling selfSubnets for every iteration of the metadata loop before merging this.

moshe-blox

LGTM

olegshmuelov · 2024-12-22T14:11:53Z

It seems there's an issue with the current implementation of this PR. While the logic correctly focuses on listening and getting metadata only for committee validators, it does not account for the exporter node. The exporter node requires metadata for all validators, not just the committee ones.

This is a critical oversight, as it may cause incomplete data in the exporter node's operations. We need to update the logic to ensure that the exporter node retrieves metadata for all validators.

validator: listen and get metadata only for committee validators

c2a0697

nkryuchkov requested a review from y0sher October 10, 2024 23:42

nkryuchkov added 4 commits October 11, 2024 16:24

fix tests

602c350

fetch metadata from validators belonging to active subnets instead of…

709c6a4

… validators belonging to own committee

use uber's gomock

cac920e

fix TestSetupValidatorsExporter

0476e21

y0sher requested changes Oct 15, 2024

View reviewed changes

calculate subnet for committee instead of validator

62b6950

nkryuchkov requested a review from y0sher October 16, 2024 01:32

y0sher reviewed Oct 16, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

y0sher requested changes Oct 16, 2024

View reviewed changes

remove pre-fork code

0ae7821

nkryuchkov marked this pull request as ready for review October 16, 2024 23:42

nkryuchkov requested a review from y0sher October 17, 2024 01:34

nkryuchkov requested a review from MatusKysel October 17, 2024 12:38

MatusKysel reviewed Oct 17, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

MatusKysel reviewed Oct 17, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

call ActiveSubnets before the loop

4b30cc6

MatusKysel approved these changes Oct 17, 2024

View reviewed changes

y0sher reviewed Oct 20, 2024

View reviewed changes

moshe-blox reviewed Oct 21, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

nkryuchkov added 3 commits October 21, 2024 11:44

fix check for subnet

9e41edf

fix TestSetupValidatorsExporter

74c19ef

check bounds

df6e732

nkryuchkov requested review from y0sher, iurii-ssv and moshe-blox October 26, 2024 13:44

moshe-blox reviewed Nov 17, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

fix: stale subnets in loop

ac49a96

moshe-blox approved these changes Nov 17, 2024

View reviewed changes

y0sher reviewed Dec 15, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

nkryuchkov added 2 commits December 16, 2024 10:00

Merge branch 'stage' into metadata-only-committee-validators

51d2868

# Conflicts: # network/p2p/p2p_setup.go # operator/validator/mocks/controller.go

initial fetch for all self subnets

fd45beb

y0sher reviewed Dec 16, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

y0sher reviewed Dec 16, 2024

View reviewed changes

operator/validator/controller.go Show resolved Hide resolved

y0sher requested changes Dec 16, 2024

View reviewed changes

nkryuchkov added 3 commits December 16, 2024 19:53

simplify setting subnets

6448672

check active subnets

453f8f2

remove subnet check

f571d03

y0sher reviewed Dec 16, 2024

View reviewed changes

operator/validator/controller.go Show resolved Hide resolved

y0sher approved these changes Dec 16, 2024

View reviewed changes

y0sher reviewed Dec 17, 2024

View reviewed changes

operator/validator/controller.go Show resolved Hide resolved

y0sher reviewed Dec 17, 2024

View reviewed changes

operator/validator/controller.go Outdated Show resolved Hide resolved

y0sher self-requested a review December 17, 2024 12:25

filter out shares when updating metadata in a loop

13a6445

y0sher approved these changes Dec 17, 2024

View reviewed changes

moshe-blox approved these changes Dec 17, 2024

View reviewed changes

moshe-blox changed the title ~~validator: listen and get metadata only for committee validators~~ feat(validator): listen and get metadata only for committee validators Dec 17, 2024

moshe-blox merged commit 9eee883 into stage Dec 17, 2024
5 checks passed

moshe-blox deleted the metadata-only-committee-validators branch December 17, 2024 14:12

olegshmuelov mentioned this pull request Dec 22, 2024

bug(exporter): exporter node requires metadata for all validators #1961

Closed

y0sher mentioned this pull request Dec 29, 2024

refactor: optimize subnet calculations using big.Int for reduced memory usage #1969

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(validator): listen and get metadata only for committee validators #1787

feat(validator): listen and get metadata only for committee validators #1787

nkryuchkov commented Oct 10, 2024 •

edited

Loading

y0sher left a comment

nkryuchkov commented Oct 16, 2024 •

edited

Loading

y0sher left a comment •

edited

Loading

MatheusFranco99 commented Oct 16, 2024

nkryuchkov commented Oct 17, 2024

MatheusFranco99 commented Oct 17, 2024

nkryuchkov commented Oct 17, 2024

MatusKysel left a comment

y0sher Oct 20, 2024

nkryuchkov Oct 21, 2024

nkryuchkov Oct 21, 2024

y0sher Oct 20, 2024

moshe-blox left a comment •

edited

Loading

moshe-blox commented Nov 17, 2024 •

edited

Loading

y0sher left a comment

y0sher left a comment

y0sher left a comment

moshe-blox left a comment

olegshmuelov commented Dec 22, 2024

feat(validator): listen and get metadata only for committee validators #1787

feat(validator): listen and get metadata only for committee validators #1787

Conversation

nkryuchkov commented Oct 10, 2024 • edited Loading

y0sher left a comment

Choose a reason for hiding this comment

nkryuchkov commented Oct 16, 2024 • edited Loading

y0sher left a comment • edited Loading

Choose a reason for hiding this comment

MatheusFranco99 commented Oct 16, 2024

nkryuchkov commented Oct 17, 2024

MatheusFranco99 commented Oct 17, 2024

nkryuchkov commented Oct 17, 2024

MatusKysel left a comment

Choose a reason for hiding this comment

y0sher Oct 20, 2024

Choose a reason for hiding this comment

nkryuchkov Oct 21, 2024

Choose a reason for hiding this comment

nkryuchkov Oct 21, 2024

Choose a reason for hiding this comment

y0sher Oct 20, 2024

Choose a reason for hiding this comment

moshe-blox left a comment • edited Loading

Choose a reason for hiding this comment

moshe-blox commented Nov 17, 2024 • edited Loading

y0sher left a comment

Choose a reason for hiding this comment

y0sher left a comment

Choose a reason for hiding this comment

y0sher left a comment

Choose a reason for hiding this comment

moshe-blox left a comment

Choose a reason for hiding this comment

olegshmuelov commented Dec 22, 2024

nkryuchkov commented Oct 10, 2024 •

edited

Loading

nkryuchkov commented Oct 16, 2024 •

edited

Loading

y0sher left a comment •

edited

Loading

moshe-blox left a comment •

edited

Loading

moshe-blox commented Nov 17, 2024 •

edited

Loading