Drop "blinded blob sidecar" concept after using signed header to authenticate blobs

[ralexstokes]

Following ethereum/consensus-specs#3531, we can make some strong simplifications to the builder-specs which this PR implements.

Changes to getHeader:
The relay must still provide KZGCommitments in a signed bid but they no longer need to pass along proofs or blob roots.

Changes to getPayload:
The relay simply needs the SignedBlindedBeaconBlock (as per the status quo) and can still assemble the unblinded beacon block. The relay will still reveal the execution payloads alongside the full "blob bundle".

Open questions:
The simplification from the linked consensus-specs PR requires blob gossip to include a short Merkle proof from the signed header's body root to each KZG commitment it includes. This PR assumes the relay will do this work independently although we could demand the proposer sends these along as well with the SignedBlindedBeaconBlock.

Pros: less computation for relays (and less code to debug/maintain)
Cons: demands consensus clients have done this work ahead of time (they may wait until after selecting the remote payload as the blob sidecar gossip can come after)

[avalonche]

As the relay will need to verify proposer inputs anyways it seems like relays will need to do the same amount of compute with more work on the consensus clients.

[ralexstokes]

As the relay will need to verify proposer inputs anyways it seems like relays will need to do the same amount of compute with more work on the consensus clients.

there is a small optimization here where the beacon node (who can optimistically compute the inclusion proofs) can send them to the relay, so that the relay only has to do verification (and not compute and verify themselves)

the question comes down to: are relays ok to maintain this code to compute the proofs? or should they rely on beacon nodes to compute and send along with the signed header?

[ralexstokes]

linter is failing as it depends on ethereum/beacon-APIs#369, will update once that PR is merged

[g11tech]

lgtm

[potuz]

I'd like to get a little more discussion about this design. I can see clear tradeoffs between the proposer doing it or the relay doing it. On the one hand this reduces code complexity on the relay and adds it to the proposer that presumably is running a more vetted software. On the other hand it adds latency to block production as now the proposer needs to send back the proofs. The relay still needs to assemble the sidecar with the actual blob contents. If we were to have the relay assembling the sidecars beforehand, it can start building most of the proof (out of the 17 entries in the proof, only the last four require entries from the Beacon Block Body). So most of the work can be done before the blinded beacon block comes back.

Even if we were to force the proposer to send the proofs, an intermediate step that would alleviate the extra data on the wire is to send just the last four roots in the proof and let the relay compute the remaining 13.

Given the above it seems to me that it's more beneficial for the network as a whole to have the relay building the proofs and not the proposers.

[realbigsean]

Can the proposer start computing the proofs and cache them while the validator is signing the block?

[potuz]

The proposer can, but the problem is sending them over the wire. These are 3264 bytes for 6 blobs

[ralexstokes]

@potuz this is a valid solution but it does complicate things even more than "proposer sends full proofs" or "relay and proposer both compute them independently"

im not convinced the additional latency is going to matter that much as proof creation should be quite cheap and like @realbigsean is saying either or both parties can pipeline proof creation as an optimization

I would need to see a prototype demonstrating the latency gain is enough to warrant the more complex (especially conceptually) solution you are proposing. The latest change here has the proposer compute the proofs in fact to make things as simple as possible for the relay.

Also worth noting that while block/blob gossip is latency-sensitive, the relay (if they are really that worried) can implement these optimizations and in either case they will get a head start on gossip relative to the proposer so there isn't any real latency gain from the network-global perspective

[potuz]

What I'm proposing is that the relay does the full proof so that the proposer doesn't incurr in any extra network latency. The argument about the majority of the proof being computable by the relay before getting any signature from the proposer is just to exemplify this. If instead of counting the 17 roots you count only those 13, the proposer will be sending 2496 extra bytes that could have been completely computed by the relay before even sending the bid

[realbigsean]

26,112 bits / 10Mbps = 2.6ms . That on top of the optimization @potuz is suggesting seems significant. It also does make the API a bit simpler. As far as relay complexity, all clients will have code for the proof computation implemented so the methods should be pretty widely available. It gets a bit complex if they actually implements potuz optimization. but overall it does seem worth doing. Are we able to get feedback from people actually running relays?

[g11tech]

not in favor of partial optimizations to save a few ms over http, so latency isn't a factor here imo and we should either let all proofs come from beacon or all proofs generated by the relay

i prefer beacon but again not so hard bend on this. if proof generation by relay has the benefit that it can transmit block as soon as it sees it but again with the cpus we have today, i don't think its gonna make any diff

[avalonche]

agree that the optimisations of a few ms doesn't seem significant as there are better optimisations available (like this issue with 200-300ms improvements) and proofs should either all be computed on the relay or from the beacon node.
Are there examples / libraries on how to implement the optimisation @potuz mentioned? From the perspective of relay code complexity it shouldn't be too much of an issue to include proof creation but not the more complex optimisation proposed.

[etan-status]

the maxItems should be the theoretical max: MAX_BLOB_COMMITMENTS_PER_BLOCK, for stable SSZ transfer

[ralexstokes]

going to have relay compute independently, so not a concern anymore

[potuz]

agree that the optimisations of a few ms doesn't seem significant as there are better optimisations available

It doesn't make much sense for the proposer to get the kzg commitments, then compute the proofs and then send them back when the relay has everything it needs to start computing this before it even sends the list to the proposer. The design is simply much cleaner if the relay is in charge of this.

It is also hard for the proposer to parallelize this since at the same time it will be computing proofs for its local block (together with all the other tasks when proposing). And as a plus is better forward looking for ePBS when the kzg commitments won't even be in the block and the builder will have to include these proofs in the payload.

[ralexstokes]

yeah i think this is reasonable, i'll roll back to the original state with independent computation later today

and we will be sure to call out the relevant code for the relays and make sure they can pass test vectors

[ralexstokes]

@avalonche @g11tech @potuz @realbigsean @etan-status

I've made the relevant updates and think this is ready to merge in the next day or two

I'll follow back up with some pointers on how relays should compute and verify KZG commitment inclusion proofs in another PR

[g11tech]

Considering ePBS then it makes sense for relay to do the compute 💪

[potuz]

Can't approve it cause I'm out of my depth with regards to APIs, but this looks good to me!

[g11tech]

small cleanup: i think BlobsBundleCommon would not be required, just BlobsBundle

[ralexstokes]

yeah I agree, I think this was added in an earlier rev where the factoring made more sense

I can drop it

[g11tech]

lgtm