Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the FastEmbed GPU implementation #905

Open
aymbot opened this issue Jul 16, 2024 · 0 comments
Open

Support the FastEmbed GPU implementation #905

aymbot opened this issue Jul 16, 2024 · 0 comments
Labels
feature request Ideas to improve an integration integration:fastembed P3

Comments

@aymbot
Copy link

aymbot commented Jul 16, 2024

Is your feature request related to a problem? Please describe.
Currently the Fastembed.... embedders are not utilizing the GPUs which makes it so that i.e. SPLADE takes a substantial amount of time for embeddings vs. its counterparts.

Describe the solution you'd like
QDrant supports GPUs with another library, see here. Utilizing that library would allow us to leverage our GPUs. GPU-mode could be enabled with a flag or another method.

Describe alternatives you've considered
Besides the Fastembed.. embedders, there are no out-of-the-box, nor integration, alternatives for sparse embeddings, meaning the only alternative would be to not use GPUs.

Additional context
For now, none.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Ideas to improve an integration integration:fastembed P3
Projects
Development

No branches or pull requests

3 participants