✨ feat: Add Fireworks AI Model Provider

[hezhijie0327]

💻 变更类型 | Change Type

• ✨ feat
• 🐛 fix
• ♻️ refactor
• 💄 style
• 👷 build
• ⚡️ perf
• 📝 docs
• 🔨 chore

🔀 变更说明 | Description of Change

1. 基于 📑 [DEMO] This is a demo PR of how to add an OpenAI Compatible Provider #2804 实现支持 Fireworks AI
2. 添加 firefunction-v2 及 firellava-13b 模型（FireworksAI model）
3. firefunction-v2 模型 function call 调用 [TODO] 禁用 Stream as Work Around 已支持流式调用

📝 补充信息 | Additional Information

设置页面



对话页面


工具调用

[vercel]

@hezhijie0327 is attempting to deploy a commit to the LobeHub Pro Team on Vercel.

A member of the Team first needs to authorize it.

[lobehubbot]

👍 @hezhijie0327

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

[codecov]

Codecov Report

Attention: Patch coverage is 95.56650% with 9 lines in your changes missing coverage. Please review.

Project coverage is 91.85%. Comparing base (dd7c8df) to head (34506d8).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/app/api/chat/agentRuntime.ts	14.28%	6 Missing ⚠️
src/libs/agent-runtime/AgentRuntime.ts	57.14%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3392      +/-   ##
==========================================
+ Coverage   91.83%   91.85%   +0.02%     
==========================================
  Files         455      457       +2     
  Lines       30452    30654     +202     
  Branches     2116     2960     +844     
==========================================
+ Hits        27965    28158     +193     
- Misses       2487     2496       +9     

Flag	Coverage Δ
app	91.85% <95.56%> (+0.02%)	⬆️
server	97.36% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hezhijie0327]

1. Stream 关闭后能调用 Plug-in，但是调用后停止输出了
   

{"id":"414bef39-ab67-4058-983b-c8d473f4a200","object":"chat.completion","created":1722735109,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"message":{"role":"assistant","content":"I don't have real-time information. I need to fetch the current weather information for Shanghai.","tool_calls":[{"index":0,"id":"call_mwuk5cLkFm8vChkBrwm8vRn4","type":"function","function":{"name":"realtime-weather____fetchCurrentWeather","arguments":"{\"city\": \"Shanghai\"}"}}]},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":361,"total_tokens":407,"completion_tokens":46}}

2. Stream 启用后输出为空
   

[stream start] 2024-8-4 2:10:49.86
[chunk 0] 2024-8-4 2:10:49.206
{"id":"98d1cdb4-efb0-4a62-b4ec-6abaf128c596","object":"chat.completion.chunk","created":1722737448,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}],"usage":null}
[chunk 1] 2024-8-4 2:10:49.246
{"id":"98d1cdb4-efb0-4a62-b4ec-6abaf128c596","object":"chat.completion.chunk","created":1722737448,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_jsvhZfOOhfQFNr6wstCPAZ7j","type":"function","function":{"name":"realtime-weather____fetchCurrentWeather"}}]},"finish_reason":null}],"usage":null}
[chunk 2] 2024-8-4 2:10:49.373
{"id":"98d1cdb4-efb0-4a62-b4ec-6abaf128c596","object":"chat.completion.chunk","created":1722737448,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":null,"type":"function","function":{"arguments":" {\"city\": \"上海"}}]},"finish_reason":null}],"usage":null}
[chunk 3] 2024-8-4 2:10:49.374
{"id":"98d1cdb4-efb0-4a62-b4ec-6abaf128c596","object":"chat.completion.chunk","created":1722737448,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":null,"type":"function","function":{"arguments":"\"}"}}]},"finish_reason":null}],"usage":null}
[chunk 4] 2024-8-4 2:10:49.375
{"id":"98d1cdb4-efb0-4a62-b4ec-6abaf128c596","object":"chat.completion.chunk","created":1722737448,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":425,"total_tokens":451,"completion_tokens":26}}
[stream finished] total chunks: 5

Stream 中存在一段 ECONNRESET

Error: aborted
    at abortIncoming (node:_http_server:797:17)
    at socketOnClose (node:_http_server:791:3)
    at Socket.emit (node:events:531:35)
    at TCP.<anonymous> (node:net:339:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'ECONNRESET'
}

[hezhijie0327]

神奇，现在 Stream 关闭后又可用了...

那现在就是 Stream 下有问题了...

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Magic, now the Stream is available again after closing...

Then there is a problem under Stream...

[hezhijie0327]

@arvinxx
遇到个很怪的问题，stream 中间会插一个 ECONNRESET，非 Stream 模式下就没事

Error: aborted
    at abortIncoming (node:_http_server:797:17)
    at socketOnClose (node:_http_server:791:3)
    at Socket.emit (node:events:531:35)
    at TCP.<anonymous> (node:net:339:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'ECONNRESET'
}

[hezhijie0327]

对比了下其他 Provider，看上去是 Stream 第一个流中缺少 content?
{"id":"a3bf7f4f-bbcb-458b-8be4-5ff5e4c411bf","object":"chat.completion.chunk","created":1722751530,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}],"usage":null}

Anyway, 暂时他们家也只有一个模型支持 function call，暂时禁用 stream 当 tools 存在时

[arvinxx]

@hezhijie0327 abort 大概率是 stream 的解析过程抛错导致。可能比较好的做法应该是等我搞完这个： #3361

然后针对 Firework 写一个自定义的 stream chunk 解析函数来规避 content 为空的情况

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@hezhijie0327 abort is most likely caused by an error in the stream parsing process. Maybe a better approach would be to wait until I finish this: #3361

Then write a custom stream chunk parsing function for Firework to avoid the situation where content is empty.

[arvinxx]

OpenAI 的流默认解析规则在这里：https://github.com/lobehub/lobe-chat/blob/main/src/libs/agent-runtime/utils/streams/openai.ts#L15-L65

当然另外一种策略就是直接把 firework 的解析问题在 OpenAI 的流解析里也默认集成了，这样的好处就是能提升 OpenAI 流解析函数的鲁棒性，坏处就是存在耦合性

[hezhijie0327]

那先等改造把，Spark 的 function call 我昨天也是试了下，他的 Stream 流会报一个 "e.message.tool_calls?.map is not a function 我过会儿补在 #3361

[hezhijie0327]

@arvinxx 从昨天折腾 Bedrock Stream 时有了改造的思路

对以下两种情况进行优化：

1. tool_calls 存在，但 content 为空，引起 UI 对话界面显示空，但无报错
   

{"choices":[{"index":0,"message":{"role":"assistant","content":"","tool_calls":[{"id":"3NcHNntdRyaHu8zisKJAhQ","function":{"name":"realtime-weather____fetchCurrentWeather","arguments":"{\"city\": \"Singapore\"}"}}]},"stop_reason":"tool_calls"}]}

2. tool_calls.function 中，name 或 arguments 仅一项存在，引起 UI 界面报错，提示 "message": "[{"code": "invalid_type","expected": "string","received": "undefined","path": ["function","arguments"],"message": "Required"}]"
   

{"id":"c1c08c27-475e-4d8f-921b-557b9cb6ed68","object":"chat.completion.chunk","created":1725713803,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_VFFb6PEAADjxT0wk39AXJ5MQ","type":"function","function":{"name":"clock-time____getCurrentTime____standalone"}}]},"finish_reason":null}],"usage":null}`
{"id":"c1c08c27-475e-4d8f-921b-557b9cb6ed68","object":"chat.completion.chunk","created":1725713803,"model":"accounts/fireworks/models/firefunction-v2","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":null,"type":"function","function":{"arguments":" {"}}]},"finish_reason":null}],"usage":null}

[arvinxx]

@hezhijie0327 第一种状态看下能不能单独提出来拆一个 PR 看看？我刚有优化一个PR： #3872 ，有调整这部分，但是有人反馈说对一些模型有影响，看看你的方案是不是会更好一些

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@hezhijie0327 In the first state, can you propose a separate PR and split it? I just optimized a PR: #3872. I have adjusted this part, but some people reported that it has an impact on some models. Let’s see if your solution is better.

[arvinxx]

@hezhijie0327 abort 大概率是 stream 的解析过程抛错导致。可能比较好的做法应该是等我搞完这个： #3361

另外我现在优化了一个实现，如果 stream 解析时候抛错了的话会直接传递 error chunk，所以这个应该可以先合了，后续有人反馈抛错可以再优化

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@hezhijie0327 abort is most likely caused by an error in the stream parsing process. Maybe a better approach would be to wait until I finish this: #3361

In addition, I have now optimized an implementation. If an error is thrown during stream parsing, the error chunk will be passed directly, so this should be able to be combined first. If someone reports an error later, it can be optimized again.

[hezhijie0327]

好的 我过会儿拆一下

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Okay, I'll take it apart later.

[hezhijie0327]

@arvinxx 刚把 CI 修了下，没啥问题了

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

@arvinxx just fixed the CI and there is no problem anymore

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
lobe-chat-preview	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 10, 2024 4:08pm

[lobehubbot]

❤️ Great PR @hezhijie0327 ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
项目的成长离不开用户反馈和贡献，感谢您的贡献! 如果您对 LobeHub 开发者社区感兴趣，请加入我们的 discord，然后私信 @arvinxx 或 @canisminor1990。他们会邀请您加入我们的私密开发者频道。我们将会讨论关于 Lobe Chat 的开发，分享和讨论全球范围内的 AI 消息。

[lobehubbot]

🎉 This PR is included in version 1.16.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀