Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

investigate and fix snappy stream encoding of a big bellatrix block which was rejected by the lighthouse bn #3903

Closed
4 tasks done
g11tech opened this issue Apr 9, 2022 · 1 comment · Fixed by #3912
Closed
4 tasks done
Assignees

Comments

@g11tech
Copy link
Contributor

g11tech commented Apr 9, 2022

Pawan from lighthouse raised an issue where there snappy decoder rejected a lodestar encoded block. Apparently it was a big bellatrix block from goerli shadowfork 2.

Investigation Updates:

  • After a bit of joint setup and investigation, pawan came up with theory that it was because of the snappy stream specification where a decoded/uncompressed chunk can't be > 65k bytes, and this particular block was around 130k bytes.
  • Was able to stream decode the provided data, as suspected it was just 1 chunk, was able to successfully ssz decode bellatrix block
    image
  • Chunkified the uncompressed data into three chunks and reencoded it as a stream and dumped it into a file. Passed that re-encoded chunkified stream against the test code to confirm that it was now being decoded as 3 chunks (60k,60k,13k) something bytes. Provided the file to pawan over discord for him to run through his decoder and confirm
  • Waiting on Pawan/Lighthouse team for confirmation on the chunk post, details here in comment

Expectation: Upon successful confirmation from Pawan/lighthouse regarding the successful decoding of the chunkified stream, do a PR in lodestar to chunkify the snappy stream payloads as well as add tests for the same using the same block data

@g11tech g11tech self-assigned this Apr 9, 2022
@g11tech g11tech changed the title investigate and fix snappy stream encoding of a big bellatrix block being rejected by the lighthouse investigate and fix snappy stream encoding of a big bellatrix block which was rejected by the lighthouse bn Apr 9, 2022
@g11tech
Copy link
Contributor Author

g11tech commented Apr 9, 2022

lighthouse <> lodestar chunked encoding/decoding status

  • Aparently, lighthouse is getting checksum error on the chunked stream encoded data by lodestar.
    However the my decoder able to decode lighthouse chunk encoded data correctly. So apparently, there are two differentt encodings for the same uncompressed data.
    Verified that the decoder was correctly decoding two different encodings/compressions into the same data.
  • Check using other libraries/with other CLs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant