Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cleanup/Prune old data from IPFS S3 bucket (if enabled) #25

Open
xadahiya opened this issue Dec 16, 2024 · 2 comments
Open

Cleanup/Prune old data from IPFS S3 bucket (if enabled) #25

xadahiya opened this issue Dec 16, 2024 · 2 comments
Assignees
@anomit @xadahiya @Seth-Schmidt
Labels
enhancement New feature or request

Comments

@xadahiya
Copy link
Member

Current full nodes for a use case like Uniswap V2 generate about 1-2 GB of snapshot data daily; we need a cleanup mechanism to prune/remove old data from the s3 bucket.
@xadahiya xadahiya added the enhancement New feature or request label Dec 16, 2024
@anomit anomit changed the title Cleanup/Prune old data from IPFS S3 bucked (if enabled) Cleanup/Prune old data from IPFS S3 bucket (if enabled) Jan 20, 2025
@anomit
Copy link
Member

anomit commented Jan 20, 2025

We need a meta of uploaded files to S3. Then it is easier to prune.

@anomit
Copy link
Member

anomit commented Jan 20, 2025

We are generating the UUIDs of the file names associated with the CID uploads. Should not be much work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@anomit anomit

@xadahiya xadahiya

@Seth-Schmidt Seth-Schmidt

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@anomit @xadahiya @Seth-Schmidt