daac-tools

All

13 repositories

vaporetto
Public
🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
nlp rust analyzer segmentation morphological-analysis tokenization japanese tokenizer
Rust
•
Apache License 2.0
•9•230•0•1•Updated Dec 24, 2024Dec 24, 2024
daachorse
Public
🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure in Rust.
search finite-state-machine text-processing aho-corasick no-std double-array substring-matching rust
Rust
•
Apache License 2.0
•14•209•2•2•Updated Dec 15, 2024Dec 15, 2024
vibrato
Public
🎤 vibrato: Viterbi-based accelerated tokenizer
nlp rust japanese tokenizer segmentation morphological-analysis tokenization
Rust
•
Apache License 2.0
•14•340•6•0•Updated Sep 23, 2024Sep 23, 2024
python-vaporetto
Public
🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.
python nlp rust japanese tokenizer analyzer segmentation morphological-analysis tokenization
Rust
•
Apache License 2.0
•1•21•0•0•Updated Sep 4, 2024Sep 4, 2024
python-vibrato
Public
Viterbi-based accelerated tokenizer (Python wrapper)
python nlp tokenizer segmentation morphological-analysis
Rust
•
Apache License 2.0
•1•41•0•0•Updated Sep 4, 2024Sep 4, 2024
trie-match
Public
Fast match expression optimized for string comparison
rust performance no-std double-array
Rust
•
Apache License 2.0
•0•38•0•0•Updated Jan 29, 2024Jan 29, 2024
vaporetto-models
Public
Tokenization models and training scripts for Vaporetto fast tokenizer
Rust
•
Apache License 2.0
•0•1•0•0•Updated May 30, 2023May 30, 2023
crawdad
Public
🦞 Rust library of natural language dictionaries using character-wise double-array tries.
trie data-structures no-std cjk-characters double-array search rust
Rust
•
Apache License 2.0
•2•29•0•0•Updated Feb 20, 2023Feb 20, 2023
include-bytes-zstd
Public
Includes a file with zstd compression in Rust
rust zstd zstandard
Rust
•
Apache License 2.0
•0•12•0•0•Updated Feb 17, 2023Feb 17, 2023
guidelines
Public
Guidelines for daac-tools community
0•0•0•0•Updated Feb 16, 2023Feb 16, 2023
python-daachorse
Public
🐎 A fast implementation of the Aho-Corasick algorithm using the compact double-array data structure. (Python wrapper for daachorse)
python search finite-state-machine text-processing aho-corasick double-array substring-matching
Rust
•
Apache License 2.0
•1•16•0•0•Updated Feb 2, 2023Feb 2, 2023
rucrf
Public
Conditional Random Fields implemented in pure Rust
rust machine-learning crf
Rust
•
Apache License 2.0
•2•8•0•0•Updated Dec 14, 2022Dec 14, 2022
find-simdoc
Public
Finding all pairs of similar documents time- and memory-efficiently
rust similarity-search document-search all-pairs
Rust
•
Apache License 2.0
•3•58•1•0•Updated Sep 27, 2022Sep 27, 2022