BitNet.cpp for Rust

Almost safe wrapper for BitNet-cpp based on llama-cpp

Development

See bitnet-cpp-sys

Roadmap

• minimize crate size
• rewrite sampler PR 9294
• look into Metal implementation of MatMul kernerls for BitNet
• add more examples
• replace all remaining python from BitNet repo
• add better division between arm64 and x86_64
• move python code generation into patch files

Platform support

arm64	x64	OS	comments
✅	❌	MacOS	tested fully on Apple Silicon macs
⚠️	❌	Linux	should test on ARM based linux
❓	❌	Windows	should test on ARM based windows

Thanks

Heavily inspired by llama-cpp-rs

License

MIT

Copyright

© 2024, Eugene Hauptmann