Added Groq LLM model support #1447
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 task
|
Should I add support for Gemini APIs to model providers in the same pull request, or should I create a new pull request for it ? |
Gemini‘s already supported. |
Why closed? |
|
Apologies from my side.
I thought there were conflicts on my end. I am new to open-source contributions.
…________________________________
From: Kevin Hu ***@***.***>
Sent: Thursday, July 11, 2024 3:26 PM
To: infiniflow/ragflow ***@***.***>
Cc: PARESH MAKWANA ***@***.***>; State change ***@***.***>
Subject: Re: [infiniflow/ragflow] Added Groq LLM model support (PR #1447)
@JinHai-CN<https://github.com/JinHai-CN> & @KevinHuSh<https://github.com/KevinHuSh>
Should I add support for Gemini APIs to model providers in the same pull request, or should I create a new pull request for it ?
Why closed?
I was in the middle of merging conflicts.
—
Reply to this email directly, view it on GitHub<#1447 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATOTSC2DJJXMPJ3MESWVMLDZLZJFLAVCNFSM6AAAAABKSNRBSWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRSGUYTMMRTGA>.
You are receiving this because you modified the open/close state.Message ID: ***@***.***>
|
KevinHuSh
added a commit
that referenced
this pull request
Jul 12, 2024
#1432 #1447 This PR adds support for the GROQ LLM (Large Language Model). Groq is an AI solutions company delivering ultra-low latency inference with the first-ever LPU™ Inference Engine. The Groq API enables developers to integrate state-of-the-art LLMs, such as Llama-2 and llama3-70b-8192, into low latency applications with the request limits specified below. Learn more at [groq.com](https://groq.com/). Supported Models | ID | Requests per Minute | Requests per Day | Tokens per Minute | |----------------------|---------------------|------------------|-------------------| | gemma-7b-it | 30 | 14,400 | 15,000 | | gemma2-9b-it | 30 | 14,400 | 15,000 | | llama3-70b-8192 | 30 | 14,400 | 6,000 | | llama3-8b-8192 | 30 | 14,400 | 30,000 | | mixtral-8x7b-32768 | 30 | 14,400 | 5,000 | --------- Co-authored-by: paresh0628 <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
Halfknow
pushed a commit
to Halfknow/ragflow
that referenced
this pull request
Nov 11, 2024
infiniflow#1432 infiniflow#1447 This PR adds support for the GROQ LLM (Large Language Model). Groq is an AI solutions company delivering ultra-low latency inference with the first-ever LPU™ Inference Engine. The Groq API enables developers to integrate state-of-the-art LLMs, such as Llama-2 and llama3-70b-8192, into low latency applications with the request limits specified below. Learn more at [groq.com](https://groq.com/). Supported Models | ID | Requests per Minute | Requests per Day | Tokens per Minute | |----------------------|---------------------|------------------|-------------------| | gemma-7b-it | 30 | 14,400 | 15,000 | | gemma2-9b-it | 30 | 14,400 | 15,000 | | llama3-70b-8192 | 30 | 14,400 | 6,000 | | llama3-8b-8192 | 30 | 14,400 | 30,000 | | mixtral-8x7b-32768 | 30 | 14,400 | 5,000 | --------- Co-authored-by: paresh0628 <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
This PR adds support for the Groq LLM (Large Language Model).
Groq is an AI solutions company delivering ultra-low latency inference with the first-ever LPU™ Inference Engine. The Groq API enables developers to integrate state-of-the-art LLMs, such as Llama-2 and llama3-70b-8192, into low latency applications with the request limits specified below. Learn more at groq.com.
Type of change