feat(share/availability/full): Introduce Q4 trimming for archival nodes

[renaynay]

This PR introduces trimming of .q4 files outside the sampling window for archival nodes.

It also updates the pruner checkpoint such that it's possible to convert from an archival node --> full pruning node but not the other way around (the node runner would need to resync the node in this case).

All deletion logic now lives in either share/availability/light or share/availability/full but the pruner service is still contained in a separate package and now runs by default as all node types will have some form of pruning happening.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 37.50000% with 75 lines in your changes missing coverage. Please review.

Project coverage is 45.09%. Comparing base (2469e7a) to head (3e1bb92).
Report is 417 commits behind head on main.

Files with missing lines	Patch %	Lines
nodebuilder/pruner/module.go	0.00%	47 Missing ⚠️
nodebuilder/pruner/migration_utils.go	69.23%	6 Missing and 2 partials ⚠️
share/availability/full/availability.go	0.00%	7 Missing ⚠️
share/availability/full/utils.go	63.15%	5 Missing and 2 partials ⚠️
pruner/service.go	33.33%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4028      +/-   ##
==========================================
+ Coverage   44.83%   45.09%   +0.25%     
==========================================
  Files         265      309      +44     
  Lines       14620    22326    +7706     
==========================================
+ Hits         6555    10067    +3512     
- Misses       7313    11185    +3872     
- Partials      752     1074     +322     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[renaynay]

There's currently an annoying edge-case to the solution presented in 6087e64 that wouldn't be able to detect a node that has been previously run with pruner enabled (on older version where archival trimming was not present), and once upgraded to new version, runs without pruning enabled successfully even though it should not be allowed to revert to an archival run.

I need to figure out how to prevent this without making a super ugly solution.

[renaynay]

The solution to the above comment ^ is f370346

[walldiss]

Looks good. It took me a while to understand migration. I know it is temporary code, so not insisting to change. Perhaps it might be simpler if migration was detected inside nodebuilder? This way It will be easier to remove later, wihtout the need for refactor. Smth like this:

	// nodebuilder/share
	migration(isarhival bool){
		// Should check internaly prunned <-> archival migration of new endpoints
		created, err := avail.CreateCheckpoint(isarhival)
		
		// temporary method, can be removed later with migration() func
		wasPrunned := pruner.HasCheckpoint()
		
		if isarhival && wasPrunned && !created{
			// unwanted migration detected
		}
	}

Another thing to consider is lack of archival -> pruned conversion support in the PR. We should reset prunner checkpoint to allow it to Remove Q1 files.

[cristaloleg]

SGTM

[renaynay]

@walldiss

re #4028 (comment)

I can move the logic to nodebuilder, yes.

[walldiss]

Thank you for dedication for making it better! Happy that we have temporary logic extracted to migration_utils.go. It has its flaws of reimplementing logic for storing previous run, but I don't think it is a big deal, since it will be remove relatively soon.

Added Few optional nits, otherwise Looks good to me!