LLMFarm_core swift library to work with large language models (LLM). It allows you to load different LLMs with certain parameters.
Based on ggml and llama.cpp by Georgi Gerganov.
- MacOS (13+)
- iOS (16+)
- Various inferences
- Various sampling methods
- Metal (dont work on intel Mac)
- Model setting templates
- LoRA adapters support (read more)
- LoRA train support
- LoRA export as model support
- Restore context state (now only chat history)
- LLaMA
- GPTNeoX
- Replit
- GPT2 + Cerebras
- Starcoder(Santacoder)
- RWKV (20B tokenizer)
- Falcon
- MPT
- Bloom
- StableLM-3b-4e1t
- Qwen
- Yi models
- Deepseek models
- Mixtral MoE
- PLaMo-13B
Note: For Falcon, Alpaca, GPT4All, Chinese LLaMA / Alpaca and Chinese LLaMA-2 / Alpaca-2, Vigogne (French), Vicuna, Koala, OpenBuddy (Multilingual), Pygmalion/Metharme, WizardLM, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Mistral AI v0.1, Refact, Persimmon 8B, MPT, Bloom select llama inferece in model settings.
- Temperature (temp, tok-k, top-p)
- Tail Free Sampling (TFS)
- Locally Typical Sampling
- Mirostat
- Greedy
- Grammar (dont work for GPTNeoX, GPT-2, RWKV)
- Classifier-Free Guidance
git clone https://github.com/guinmoon/llmfarm_core.swift
Add llmfarm_core to your project using Xcode (File > Add Packages...) or by adding it to your project's Package.swift file:
dependencies: [
.package(url: "https://github.com/guinmoon/llmfarm_core.swift")
]To Debug llmfarm_core package, do not forget to comment .unsafeFlags(["-Ofast"]) in Package.swift.
Don't forget that the debug version is slower than the release version.
To build with QKK_64 support uncomment .unsafeFlags(["-DGGML_QKK_64"]) in Package.swift.
- rwkv.cpp by saharNooby.
- Mia by byroneverson.
- swift-markdown-ui by gonzalezreal
App to run LLaMA and other large language models locally on iOS and MacOS.