Light forward sync mechanism #6515
Conversation
cheatfate
force-pushed
the
hybrid-sync
branch
2 times, most recently
from
5be1b71 to
575d711
Compare
| buffer = Chunk.init(kind, uint64(slot), uint32(plainSize), data) | ||
| wrote = writeFile(chandle.handle, buffer).valueOr: | ||
| discard truncate(chandle.handle, origOffset) | ||
| discard fsync(chandle.handle) |
There was a problem hiding this comment.
Why are some fsync calls fine to discard/ignore the return value of but others not?
There was a problem hiding this comment.
So this is because of error in writeFile (initial error we going to report to user), so this is attempt to cleanup. In case where number of bytes written was less than we expect, it means that file become inconsistent, so we trying to truncate file to known good size and we trying to fsync it, but we still want to report original error message to the user.
cheatfate
force-pushed
the
hybrid-sync
branch
from
bc640a0 to
361bc0b
Compare
There was a problem hiding this comment.
This is a great improvement for the security of default sync compared to the existing genesis sync.
As this is timely to properly test, would be great to have at least a happy case test with a few epochs worth of minimal blocks to avoid accidental regressions.
If data size is a concern, one may consider syncing 1 (or N) sync committee period (~27 hrs / 8192 slots) at a time by treating each intermediate light client sync step as its own separate sync target, applying each of them separately before syncing to the next period. The hook to obtain intermediate results is LightClientUpdateObserver. It would add quite a bit of complexity, though, so maybe waiting for demand is better for now.
| @@ -92,6 +89,8 @@ proc initLightClient*( | |||
| .shouldSyncOptimistically(node.currentSlot): | |||
| return | |||
There was a problem hiding this comment.
The new sync mechanism works for all lcDataFork > LightClientDataFork.None, but here we have an additional >= Capella restriction, because that's required for the EL blockHash and setOptimisticHead.
All networks that we actively support have advanced past Capella, so letting this remain in the >= Capella section is also fine.
cheatfate
force-pushed
the
hybrid-sync
branch
4 times, most recently
from
d9ea11c to
7584886
Compare
…t after LC block received.
Remove some compilation warnings. Fix `CatchableError` leaks on Windows.
Recover binary test data.
Add some tests to ensure that checkResponse() also checks for correct order.
cheatfate
force-pushed
the
hybrid-sync
branch
from
81dccb8 to
7b6d6ff
Compare
Light forward sync uses the light client protocol to accelerate forward syncing done after checkpoint sync, when restarting the client or indeed when syncing up after a longer period of being offline.
It works by establishing a "safe" head to sync to using the light client protocol, tracing this history back to the latest known state in the database and then using a less computationally intensive method of verifying blocks based on the knowledge that the blocks lead to a valid head.
It can currently be enabled by specifying
--debug-long-range-sync=light, though it should be understood that--debug-options are not compatible across client versions and should not be used in production setups.