-
Notifications
You must be signed in to change notification settings - Fork 997
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add DataColumnSidecarsByRange
v1 Req/Resp
#3750
Conversation
Load-balancing is an interesting idea, although it's unclear to me if we need it with the current setup. When node is performing custody columns sync, they'll be downloading from peers in the same column subnet, so the When performing historic sampling, we'd likely be requesting the bulk of the samples from a different set of peers (different columns), so it probably wouldn't put too much additional load to the same peer that we're gathering custody from? Perhaps load-balancing could be use as an optimisation technique to speed up syncing? and in either case, plurals would probably be more flexible. |
👍 on the addition Want to note that this protocol is helpful to speed up forward sync and backfill sync for a custody peer to quickly fetch its columns. However, it's at odds with the server peer rate-limiting outgoing bandwidth shared between syncing and sampling peers. If by range requests will be rate-limited to the point that they become the bottleneck for syncing then there's not much benefit to just waiting to fetch the block first and then requesting by root. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
don't have the full context to say more at the moment, but the suggestion make sense
would have to go review the full peerDAS spec as-is now to say more on if this is the best way to handle this...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this makes sense and should be implemented.
I'm going to merge it for the interop milestone requirements. Thank y'all for reviewing it! |
Add
DataColumnSidecarsByRange
message for syncing as @jimmygchen suggested in #3574 (comment).The description was copied and edited from Deneb
BlobSidecarsByRange.
@djrtwo mentioned (#3574 (comment)) that considering the load-balancing, it's unclear if plural
columns: List[ColumnIndex]
or singlecolumn: ColumnIndex
makes more sense. I now usecolumns: List[ColumnIndex]
for flexibities.