Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compression of repdef buffers in mini-block encoding #3355

Open
westonpace opened this issue Jan 7, 2025 · 0 comments
Open

Compression of repdef buffers in mini-block encoding #3355

westonpace opened this issue Jan 7, 2025 · 0 comments
Assignees
Milestone

Comments

@westonpace
Copy link
Contributor

westonpace commented Jan 7, 2025

We currently don't compress rep-def buffers at all when mini-block encoding. This is pretty wasteful as it requires 4-bytes per value. We should, at the very least, make sure we are byte packing and, for mini-block, it would probably be suitable to use bitpacking.

I think the main challenge for bit-packing is that we only have the "bitpack in 1024-element chunks" variant of bit-packing and not a simple "bitpack this buffer" interface.

This is not strictly required for the beta and could be done as part of the stabilization.

@westonpace westonpace self-assigned this Jan 7, 2025
@westonpace westonpace added this to the 2.1 Beta milestone Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant