fix(core): preserve image_url content parts for DeepSeek vision model - #10026
fix(core): preserve image_url content parts for DeepSeek vision model#10026nachoeverywhere wants to merge 1 commit into
Conversation
DeepSeek's text models require message content to be a plain string, but the dedicated vision model (deepseek-v4-flash-vision-exp) accepts OpenAI content parts with image_url. The DeepSeek provider always flattened every non-text part into a `[Unsupported content type: image_url]` placeholder, so images read via read_file never reached the vision model and the request silently degraded to text. Skip flattening when the config declares image support (`modalities.image`), so image_url parts are sent as-is to the vision model while text-only models keep the existing placeholder behavior. Adds a regression test for the vision case.
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Thanks for the PR, @nachoeverywhere! Before we can review the change itself, the PR description needs to follow the repository's pull request template.
The current body uses free-form headings (Summary, Changes, Test plan, Screenshots / Demo), but none of the template's required sections are present:
## What this PR does## Why it's needed## Reviewer Test Plan(with### How to verify,### Evidence (Before & After), and the### Tested onOS table)## Risk & Scope## Linked Issues- the bilingual
<details>Chinese summary the template asks for
Please edit the PR body to follow the template: https://github.com/QwenLM/qwen-code/blob/main/.github/pull_request_template.md
The substance you've already written maps cleanly onto those sections — this is a reformat, not new work. Once the body follows the template, re-trigger triage (@qwen-code /triage) and we'll continue with the review.
中文说明
感谢提交 PR,@nachoeverywhere!在进入代码审查之前,PR 描述需要按照仓库的 pull request 模板来填写。
当前正文使用的是自由格式的小节(Summary、Changes、Test plan、Screenshots / Demo),模板要求的小节一个都没有:
## What this PR does## Why it's needed## Reviewer Test Plan(包含### How to verify、### Evidence (Before & After)以及### Tested on操作系统表格)## Risk & Scope## Linked Issues- 模板要求的双语
<details>中文说明
请按照模板编辑 PR 正文:https://github.com/QwenLM/qwen-code/blob/main/.github/pull_request_template.md
你已经写好的内容可以很自然地对应到这些小节——这只是重新排版,不需要新增内容。正文符合模板后,重新触发 triage(@qwen-code /triage),我们会继续审查。
— Qwen Code · qwen3.8-max
|
@qwen-code /triage |
|
Sandboxed verification: The verification job was cancelled before producing a report. 中文 — 判定:
|
|
tmux real-user testing: cancelled - workflow run The testing job was cancelled before producing a verdict. See the workflow run for details. — Qwen Code · tmux real-user testing |
|
Thanks for the review and for the related work in #9854 — it targets the same root cause with the same Closing this in favor of #9854, which is the right home for the change — earlier, maintained by the team, and with broader regression coverage. Happy to help with anything else if a piece of this is useful there; feel free to tag me. |
What this PR does
The DeepSeek provider was flattening every message into a plain string for all models, which turned image content into a
[Unsupported content type: image_url]placeholder. When a user configures the vision modeldeepseek-v4-flash-vision-exp(withmodalities.image), an image it reads viaread_filenever reaches the model — the request silently degrades to text. This PR stops flattening content when the config declares image support, soimage_urlparts are forwarded to the vision model as-is, while text-only models keep the existing placeholder behavior.Why it's needed
DeepSeek's text models require
message.contentto be a plain string, which is why the provider flattens arrays and substitutes non-text parts. But the dedicated vision model accepts OpenAI content parts withimage_url. Because flattening was unconditional, images read viaread_filewere replaced with[Unsupported content type: image_url]and the vision model never saw them, making vision-capable configs silently text-only. Users who setdeepseek-v4-flash-vision-expas their active model (a documented vision model) get a broken experience with no error.Reviewer Test Plan
How to verify
Configure
deepseek-v4-flash-vision-expas the active model withmodalities.image: true, then ask the agent to look at an image (orread_filea.png). Before this PR the request body contains[Unsupported content type: image_url]in place of the image; after this PR theimage_urlcontent part is preserved and sent to the vision model. The regression test asserts thatimage_urlparts survive whenmodalities.imageis set.Evidence (Before & After)
N/A — non-UI change (internal request shaping, no user-facing difference). See the regression test for the before (placeholder) vs after (preserved
image_url) of the request body.Tested on
Environment (optional)
Unit tests via
npm run test --workspaces(core package,vitest run); 28 tests including the new regression case.Risk & Scope
modalities.image === true, so text-only configs are unaffected.Linked Issues
Fixes #10027
中文说明
这个 PR 做了什么
DeepSeek 提供商此前将所有消息扁平化为纯字符串,这会把图片内容转换成
[Unsupported content type: image_url]占位符。当用户配置视觉模型deepseek-v4-flash-vision-exp(并设置modalities.image)时,它通过read_file读取的图片永远不会到达模型——请求会静默退化为纯文本。此 PR 在配置声明支持图片时停止扁平化内容,从而将image_url部分原样转发给视觉模型,而纯文本模型则保留原有的占位符行为。为什么需要
DeepSeek 的文本模型要求
message.content是纯字符串,这也是提供商扁平化数组并用占位符替代非文本部分的原因。但专用的视觉模型接受带有image_url的 OpenAI 内容部分。由于扁平化是无条件的,通过read_file读取的图片会被替换为[Unsupported content type: image_url],视觉模型从未真正看到它们,使得配置了视觉能力的模型静默地变成纯文本。用户将deepseek-v4-flash-vision-exp设为主模型(一个文档化的视觉模型)时会得到一个没有报错的破损体验。Reviewer 测试计划
如何验证
将
deepseek-v4-flash-vision-exp配置为主模型并设置modalities.image: true,然后让智能体查看一张图片(或read_file一个.png)。在此 PR 之前,请求体中图片位置是[Unsupported content type: image_url];在此 PR 之后,image_url内容部分被保留并发送给视觉模型。回归测试断言当modalities.image被设置时image_url部分能够存活。前后证据(Before & After)
不适用——非 UI 变更(内部请求整形,无用户可见差异)。请参阅回归测试中的请求体前后对比(占位符 vs 保留的
image_url)。测试环境
环境(可选)
通过
npm run test --workspaces(core 包,vitest run)运行单元测试;28 个测试,包括新增的回归用例。风险与范围
modalities.image === true时触发,因此纯文本配置不受影响。关联 Issue
Fixes #10027