Skip to content

Commit

Permalink
Basic on-the-fly zstd compression
Browse files Browse the repository at this point in the history
Prefix every bin-key value with a single uint64 varint containing:
  (3bit store-type)            // 0 for as-is, 1 for basic zstd compression, remainder reserved
    +
  (one reserverd bit)    << 3  // always 0 for now
    +
  (3bit compressability) << 4  // 0 if store-type is 0, otherwise `c := (origLen-compLen) * 8 / origLen`
                               // (using integer math, `origLen > compLen > 0` holds for any non-0 store-type)
    +
  (IPLD block data size) << 7  // 0 if store-type is 0

Include a rudimentary, dictionary-less zstd compressor as the first
non-verbatim storage type 1
  • Loading branch information
ribasushi committed Apr 21, 2024
1 parent 9718c09 commit fb23c7e
Show file tree
Hide file tree
Showing 4 changed files with 265 additions and 38 deletions.
Loading

0 comments on commit fb23c7e

Please sign in to comment.