Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BlobSidecars pruning #7085

Merged
merged 7 commits into from
Apr 28, 2023
Merged

Conversation

zilm13
Copy link
Contributor

@zilm13 zilm13 commented Apr 27, 2023

PR Description

BlobSidecar pruning + tests + some extra

  • several BlobsSidecar traces removal
  • fixes for deletedIds in db columns
  • fixes for no blobs slots handling

Store.retrieveBlobsSidecars... was removed. We will add Store.retrieveBlobSidecars... when moving store flow from manager to transactionStore. I propose to add in the same PR replacements for tests:

  • retrieveSignedBlockAndBlobsSidecar_shouldReturnEmptyIfBlockNotPresent
  • retrieveSignedBlockAndBlobsSidecar_withInconsistentSidecar
  • retrieveSignedBlockAndBlobsSidecar_withEmptyBlobs
  • retrieveSignedBlockAndBlobsSidecar_withBlobs

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

--remaining;
final boolean finished = remaining < 0;
final SlotAndBlockRootAndBlobIndex key = it.next();
if (finished && (key.isNoBlobsKey() || key.getBlobIndex().equals(ZERO))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, IIUC, we want to continue pruning blobs for the current slot so we don't partially delete blobs for a given slot.
Correct?

if yes it means that pruned can go over pruneLimit and the return statement should be

return pruned >= pruneLimit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch

}

@Command(
name = "validate-blobs-sidecars",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why you completely removed this command? It was useful to me to check that we have all blobs we expect to have

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd better to start it from the blank, there are soo many changes in categories

Copy link
Contributor

@tbenr tbenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just a nit


if (blobsSidecarStorageCountersEnabled) {
if (blobSidecarStorageCountersEnabled) {
LabelledGauge labelledGauge =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need a labelled gague anymore, we could go with metricsSystem.createGauge now.
OR we can add a label to track also the earliest slot so we can see the availability window moving.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because now we have the total blob counter, which doesn't track directly the number of slots anymore

Copy link
Contributor

@tbenr tbenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zilm13 zilm13 merged commit 921438e into Consensys:master Apr 28, 2023
@zilm13 zilm13 deleted the storage/blobsidecar-pruning branch April 28, 2023 13:53
@zilm13 zilm13 mentioned this pull request Apr 28, 2023
44 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants