Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ feat: spport qwen-vl and tool call for qwen #3114

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

Mingholy
Copy link
Contributor

@Mingholy Mingholy commented Jul 2, 2024

💻 变更类型 | Change Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 🔨 chore
  • ⚡️ perf
  • 📝 docs

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

如果使用默认的top_p参数(即0.999),将会有较大概率生成不令人满意的回答。因此除了模型文档注明的不支持的参数外,top_p也将对vl模型禁用;

qwen-vl现在已经支持图像传入base64,OpenAI Compatible模式更加贴近OpenAI行为;

如需使用S3或者阿里云OSS保存上传的图片文件,需要正确配置下列环境变量,以阿里云OSS为例:

  1. 需要配置AK、SK:
    • S3_ACCESS_KEY_ID
    • S3_SECRET_ACCESS_KEY
  2. 需要指定OSS信息:
    • S3_BUCKET
    • S3_ENDPOINT(如https://oss-cn-hangzhou.aliyuncs.com
    • S3_REGION(如oss-cn-hangzhou)
    • NEXT_PUBLIC_S3_DOMAIN(一般为https://${bucket}.${endpoint}

Copy link

vercel bot commented Jul 2, 2024

@Mingholy is attempting to deploy a commit to the LobeHub Pro Team on Vercel.

A member of the Team first needs to authorize it.

@lobehubbot
Copy link
Member

👍 @Mingholy

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献,请确保您已经遵循了我们的贡献指南,我们会尽快审查它。
如果您遇到任何问题,请随时与我们联系。

Copy link

codecov bot commented Jul 2, 2024

Codecov Report

Attention: Patch coverage is 96.38158% with 11 lines in your changes missing coverage. Please review.

Project coverage is 93.99%. Comparing base (3ce59ca) to head (e9bffab).
Report is 1 commits behind head on main.

Files Patch % Lines
src/libs/agent-runtime/utils/streams/qwen.ts 93.93% 6 Missing ⚠️
...ent-runtime/utils/openaiCompatibleFactory/index.ts 90.74% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3114      +/-   ##
==========================================
+ Coverage   93.96%   93.99%   +0.02%     
==========================================
  Files         388      389       +1     
  Lines       24149    24368     +219     
  Branches     1896     2791     +895     
==========================================
+ Hits        22692    22905     +213     
- Misses       1457     1463       +6     
Flag Coverage Δ
app 93.99% <96.38%> (+0.02%) ⬆️
server 66.94% <25.98%> (-1.27%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lobehubbot
Copy link
Member

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


Codecov Report

Attention: Patch coverage is 24.17219% with 229 lines in your changes missing coverage. Please review.

Project coverage is 66.35%. Comparing base (a148c3b) to head (30a26fd).
Report is 15 commits behind head on main.

Files Patch % Lines
src/libs/agent-runtime/qwen/index.ts 14.20% 151 Missing ⚠️
src/libs/agent-runtime/utils/streams/qwen.ts 21.21% 78 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (a148c3b) and HEAD (30a26fd). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (a148c3b) HEAD (30a26fd)
app 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3114       +/-   ##
===========================================
- Coverage   93.43%   66.35%   -27.08%     
===========================================
  Files         382       98      -284     
  Lines       23629     7452    -16177     
  Branches     1717      111     -1606     
===========================================
- Hits        22077     4945    -17132     
- Misses       1552     2507      +955     
Flag Coverage Δ
app ?
server 66.35% <24.17%> (-1.70%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Mingholy Mingholy mentioned this pull request Jul 2, 2024
7 tasks
@Mingholy Mingholy marked this pull request as draft July 2, 2024 04:33
@Mingholy
Copy link
Contributor Author

Mingholy commented Jul 2, 2024

Woking on unit test...

@arvinxx
Copy link
Contributor

arvinxx commented Jul 2, 2024

所以目前 qwen tool calling 是不支持流式是么?

@lobehubbot
Copy link
Member

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


So currently qwen tool calling does not support streaming, right?

@Mingholy
Copy link
Contributor Author

Mingholy commented Jul 2, 2024

所以目前 qwen tool calling 是不支持流式是么?

是的,目前暂不支持stream=True。同时,后续如果支持stream=True也不支持incremental_output=True,所以还要看支持流式之后的效果再做修改。

@Mingholy Mingholy marked this pull request as ready for review July 2, 2024 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants