[EIP-4844] Implement BlobsManager

[tbenr]

Introduces BlobsSidecarManager which is responsible for:

• storing unconfirmed validated BlobsSidecars coming from gossip via BlockManager
• storing unconfirmed non_validated BlobsSidecars coming from syncing flows via BlockManager
• creating a blobsSidecarAvailabilityChecker used in ForkChoice
• caching validated blobs to avoid revalidation in ForkCoiche for blocks\sidecar coming from gossip.

Under kvstore and storage:

• rename kvstore method removeUnconfirmedBlobsSidecar -> confirmBlobsSidecar
• remove two unused blobs methods in ChainStorage and related classes
• fix SafeFuture handling on ChainStorage fro void returning blobs methods

In StorageUpdate and StoreTransactionUpdates:

• migrate prunedHotBlockRoots from a Set<blockRoot> to Map<blockRoot,slot> to support blobs cleanup when hot blocks are pruned

In BlockValidator (gossip):

• implements kzg commitments validation against blobsSidecar

In BlockManager:

• added handleBlobs method in the doImportBlock pipeline to store blobsSidecar via BlobsSidecarManager. This is done after block validity check (handleInvalidBlock) and after checking it is actually a new block not seen before (handleKnownBlock). Gossip validation result is considered to allow BlobsSidecarManager to know if the availability check needs to be done or not.
• it handles the blobs "discard" conditions when the corresponding block import results in a rejection of the block.

Additional notes

• we could discard blobs sending them to the BlobsPruner which will gradually delete them from BD without queue the deletions in the storageUpdateChannel which can cause slowdown if many blobs are discarded in sequence (various pending blocks on a fork that turns out to be invalid). We could implement a BlobsDeletionChannel and block BlockPruner will consume it

• having unconfirmed affects how we handle BlobsSidecarByRange rpc method:

  • for non finalized blobs, we should have a fast way to check if corresponding blocks are in recentChainData (via containsBlock)
  • for finalized we should always lookup on the unconfirmed column to filter them out (even in the case we have only one blob per slot, it might be still unconfirmed (a proposed invalid block))

• for the unconfirmed column I found an interesting approach that could make the confirmed-unconfirmed flag more efficient to store and update: https://groups.google.com/g/leveldb/c/qOMWvp7gyxg but would require some changes in our kvstore layer

  • the idea is to have a column key stored like <id, key, attributeId> where id is our current prefix and attributeIds is a new postfix byte from 0 to number of attributes defined + the actual value we store. The column will have multiple value serializers one for each attributeId. In our case we can have:
    • <blobsSidecarId, <SlotAndBlockRoot>, 0> -> KvStoreSerializer<Boolean> (confirmed flag)
    • <blobsSidecarId, <SlotAndBlockRoot>, 1> -> KvStoreSerializer<Bytes> (BlobsSidecar ssz data)
      so when streaming\lookingup blobs we can efficiently filter by unconfirmed flag without even try to deserialize the object if doesn't match the filter.
      This need further investigation for rocksDB.

fixes #6629
fixes #6473

Documentation

• I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

• I thought about adding a changelog entry, and added one if I deemed necessary.

[zilm13]

Looks definitely like an improvement. Just a few things:

[zilm13]

Why? We couldn't unvalidate blobs, but we could get, say, IGNORE result on the second import try and remove validation in this case. I don't understand why it's needed. From my point of view, it could only damage us in some case

[tbenr]

yeah it isn't required.. it is a strange case, that should never occur, so the idea was to be conservative and force a revalidation of the sidecar against commitments during subsequent block import. But since it is strictly coupled with the block via blockroot I can remove it.