using lzma to compress/decompress state_parts #1771
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1771 +/- ##
=========================================
Coverage ? 81.25%
=========================================
Files ? 168
Lines ? 36922
Branches ? 0
=========================================
Hits ? 30002
Misses ? 6920
Partials ? 0
Continue to review full report at Codecov.
|
| _ => { | ||
| byzantine_assert!(false); | ||
| return Err(ErrorKind::Other( | ||
| "set_state_part failed: lzma::decompress failed".into(), |
There was a problem hiding this comment.
Why not show the decompression error here? Might help with debugging
There was a problem hiding this comment.
It should not fail at all.
There was a problem hiding this comment.
So every byte sequence can be decoded?
There was a problem hiding this comment.
Not failing means that failing caused by clearly malicious data.
|
is it possible to create a decompression bomb for lzma? |
|
https://bomb.codes/bombs |
|
We need a library that allows setting a resource limit for decompression + somehow know an upper bound for the size of one part |
Rust-lzma allows to limit memory usage, as usually others do. I'm trying to find a library which allows to set limited output buffer. We have already limited upper bound of one part with 1 Mb unpacked data. |
|
Can we guarantee 1 MB limit? The logic for splitting is approximate |
Trie relies on |
|
we might be able to guarantee that for some X after the coming storage changes + setting a limit on value sizes |
Yep, it's important. Should we create a task for it? |
|
created #1786 , for now we should set max size to something big - probably not infinite because of bombs, but potentially no smaller than the whole state size |
|
What number you suggest to set for now? |
|
|
I expect it should be a constant. Anyway, we can wait till #1792 will be accepted. |
|
After #1792 we still need limit on keys and values |
|
Is this PR still alive? |
It's blocked by #1805. |
|
doesn't seem actual for now |
Fixes #1048.