Replies: 4 comments 3 replies
-
|
rusted.ai, rusty.ai [your own local trusty ai companion/app!], (since it uses rust llm) |
Beta Was this translation helpful? Give feedback.
-
|
Yes Rusted.ai will be better...when read Rust we think speed,secure..and so and so... |
Beta Was this translation helpful? Give feedback.
-
|
Here is a thought, why not join forces with those in the https://localai.io community and make https://www.localai.app the App based overlay to the joint project to maximize community engagement? I think this App variation is a brilliant idea and my bet is many within the LocalAI community would think the same as it would make using LLM models more reachable for development community. |
Beta Was this translation helpful? Give feedback.
-
|
That'd be fun IMO, but we need someone leading this initiative on the advocate side and also... the integration side. The tricky part with the integration is that localai.io uses go as their backends, whereas localai.app implements it own custom rust backend. Our rust backend is not as feature rich as the go backend, but it has custom logic that integrates with desktop app serving much better than the go one, with model concurrency and file-system handling calls for the app. I can see a future where the app uses the go backend, but it will be a long road. The go backend would either need to be compiled into a reusable binary per platform (more dependency up the chain which might be flaky architecture), or be packed into a docker that the app would spawn down the line. Both will affect performance and also affect the compactness of the app IMO :d The streaming API on the app is also optimized for customization from the app as well, whereas the go backend might not head toward that direction and more toward docker serving. |
Beta Was this translation helpful? Give feedback.
-
|
This is a nice idea, but if converting this project to "go" backend, please do compare performance of both backends, this one may be a little bit buggy( or maybe some other program on my system is causing problems), the rust backend is faster than go on CPU (since I don't have a GPU), my system is a dell latitude 7480, 8GB RAM , 7th gen core i7 7600U. only ran smaller models |
Beta Was this translation helpful? Give feedback.
-
|
literally just ran into this in premAI-io/state-of-open-source-ai#73 😅 |
Beta Was this translation helpful? Give feedback.
-
|
What about my.AI, which would also let you expand with other prefixes like getMY.ai |
Beta Was this translation helpful? Give feedback.
-
Our project name might be too close to LocalAI, could cause confusion down the line...
Maybe we should rename the project?
42 votes ·
Beta Was this translation helpful? Give feedback.
All reactions