fix: Switch from tiktoken to js-tiktoken for worker compatibility#703

Merged

shakkernerd merged 1 commit intoelizaOS:mainfrom antpb:switch-to-js-tiktoken

Nov 30, 2024

+401-101

Commits on Nov 30, 2024

migrates tiktoken implementation to js-tiktoken
antpb
committed