fix(core): preserve images for multimodal DeepSeek - #9854
Conversation
E2E test report
Supporting checks after rebasing onto current upstream
The isolated runtime configuration was removed after the test. |
|
Thanks for the PR! Template: complete ✓ — all required sections present, with the bilingual summary. Problem: observed bug, not theory. #9832 reports that an image sent to a DeepSeek-named vision model ( Direction: aligned. qwen-code already carries a full input-modalities contract (explicit declaration, auto-detection fallback, converter placeholders for unsupported types). This PR makes the DeepSeek route respect that contract instead of overriding it — it fixes an inconsistency rather than adding a new surface. No direct CHANGELOG reference, but the area is clearly established in the codebase. Size: core path ( Approach: the scope feels right. One condition gating the flatten on the declared capability reuses the existing contract instead of inventing a new knob; the doc-comment updates track the new behavior; nothing unrelated in the diff. The declared risk (a text-only endpoint misconfigured with Risk: Moving on to code review. 🔍 中文说明感谢贡献！ 模板：完整 ✓ —— 所有必填段落齐全，含中文对照。 问题：已观测到的 bug，不是理论性问题。#9832 报告：向 DeepSeek 命名的视觉模型（ 方向：对齐。qwen-code 已有完整的输入模态契约（显式声明、自动检测回退、对不支持类型的占位符处理）。本 PR 让 DeepSeek 路由遵守该契约而不是覆盖它——修复的是不一致，而非新增能力面。CHANGELOG 无直接引用，但该领域在代码库中已明确确立。 规模：核心路径（ 方案：范围合理。用一个条件按声明的能力门控扁平化，复用现有契约而非新增开关；文档注释与新行为同步更新；diff 无夹带改动。已声明的风险（纯文本端点被误配 风险： 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
|
Code review. Before reading the diff, my own proposal for this bug was exactly what landed here: gate the DeepSeek flatten on the declared input modalities, because the converter already replaces media parts for undeclared modalities — the provider-level flatten was a second, unconditional layer clobbering that decision. The implementation is one condition in No correctness blockers found:
One non-blocking nit: the gate is The two-layer interaction that caused the bug, and where the gate sits: sequenceDiagram
participant P1 as User image message
participant P2 as Pipeline request context
participant P3 as Converter
participant P4 as DeepSeek buildRequest
participant P5 as Backend wire
P1->>P2: model config declares image input
P2->>P3: context carries declared modalities
P3->>P4: keeps image_url part, placeholders for undeclared types
P4->>P4: gate on declared image capability
P4->>P5: multipart preserved, or flattened to text as before
Test evidence — this PR's own CI on the reviewed commit (unattended run: the gate never executes PR code; this is read from the PR's own CI via the API): CI on
The three skips are repo policy — those jobs carry Sandboxed verification would settle the one remaining gap: 中文说明代码审查：读 diff 之前我自己对这个 bug 的方案与 PR 一致——按声明的输入模态门控 DeepSeek 的扁平化，因为 converter 已经会为未声明的模态生成占位符，provider 层的扁平化是第二个无条件层，覆盖了前者的决定。实现就是 未发现正确性阻塞项：
一个非阻塞小点：门控只看 上方时序图展示了导致 bug 的两层交互以及门控的位置。 测试证据（无人值守运行：门禁从不执行 PR 代码，以下通过 API 读取本 PR 自身的 CI）：所审提交 沙箱验证可补齐剩余缺口： — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 4/5 — clean, minimal fix that makes the DeepSeek route respect the existing modalities contract; the only non-blocking nit is the image-only gate, and end-to-end wire acceptance rests on the author's E2E report (the sponsored Stepping back over the whole thing:
The one thing to hold in mind after merge: the explicit declaration is now authoritative on DeepSeek routes, same as everywhere else. If a user declares image support on a text-only endpoint, the server's rejection is the feedback loop — that's the declared tradeoff, and it's consistent with every other OpenAI-compatible route. Approving, pinned to the reviewed commit. 中文说明信心：4/5 —— 干净、最小化的修复，让 DeepSeek 路由遵守既有的 modalities 契约；唯一的非阻塞小点是门控只看 image，端到端线上接受度目前依据作者的 E2E 报告（上文指名的受助 整体回顾：
合并后需要记住一点：显式声明在 DeepSeek 路由上现在是权威的，与其他路由一致。若用户在纯文本端点上声明图片支持，服务端拒绝就是反馈机制——这是已声明的取舍，与其他所有 OpenAI 兼容路由一致。 已批准，锚定到所审提交。 — Qwen Code · qwen3.8-max Reviewed at |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 1a": could not execute deepseek.test.ts — the review worktree has no node_modules and no built dist/ (core tests resolve the package entry through dist), and i….
中文说明
已审查。 建议见行内评论。
未探索到全部深度（达到工具调用预算）："agent 1a"：could not execute deepseek.test.ts — the review worktree has no node_modules and no built dist/ (core tests resolve the package entry through dist), and i…。
— qwen3.8-max via Qwen Code /review (v0.22.0)
| const messages = reshaped.messages.map((message) => | ||
| ensureReasoningContentOnAssistantMessage( | ||
| this.contentGeneratorConfig.modalities?.image |
There was a problem hiding this comment.
[Suggestion] The new image-preservation branch is not pinned by any test for the reasoning_content injection (the issue #3695 guard). Both branches currently go through ensureReasoningContentOnAssistantMessage, which is correct — but a one-line refactor that applies the helper only to the flattening branch would pass the entire suite: the existing reasoning_content tests all run on the text-only config (flatten branch), and the new vision test sends only user messages, which the helper returns unchanged. I confirmed this by running the mutant: image ? message : ensureReasoningContentOnAssistantMessage(flattenContentParts(message)) passes all 29 tests, and a probe with a vision-enabled config plus an assistant turn missing reasoning_content fails on the mutant (expected undefined to be '') while it passes on this code. If such a mutant ever shipped, DeepSeek thinking mode would reject every subsequent request mid-conversation on exactly the routes this PR enables. Suggested fix — extend the vision test with an assistant turn lacking reasoning_content and assert it is injected as '' while multipart user content is still preserved:
const result = visionProvider.buildRequest(
{
model: 'deepseek-v4-flash-vision-exp',
messages: [
structuredClone(expectedMessage),
{ role: 'assistant', content: 'A cat.' },
],
},
userPromptId,
);
expect(result.messages?.[1]).toEqual({
role: 'assistant',
content: 'A cat.',
reasoning_content: '',
});中文说明
新的图片保留分支没有任何测试来固定 reasoning_content 注入（issue #3695 的防护）。当前两个分支都会经过 ensureReasoningContentOnAssistantMessage，这是正确的——但如果未来有一行重构把该辅助函数只应用到扁平化分支上，整个测试套件仍会全部通过：现有的 reasoning_content 测试都运行在纯文本配置（扁平化分支）下，而新增的视觉测试只发送 user 消息（辅助函数对其原样返回）。实际运行变体后确认：image ? message : ensureReasoningContentOnAssistantMessage(flattenContentParts(message)) 通过全部 29 个测试；而一个使用视觉配置、且 assistant 回合缺少 reasoning_content 的探针测试在该变体上失败（expected undefined to be ''），在本 PR 代码上通过。若该变体将来被引入，DeepSeek 思考模式会在本 PR 启用的路由上于对话中途拒绝后续所有请求。建议修复——在视觉测试中补充一个缺少 reasoning_content 的 assistant 回合，断言其被注入为 ''，同时多段 user 内容仍被保留（见上方代码示例）。
— qwen3.8-max via Qwen Code /review (v0.22.0)
| this.contentGeneratorConfig.modalities?.image | ||
| ? message | ||
| : flattenContentParts(message), |
There was a problem hiding this comment.
[Suggestion] The gate keys on modalities.image alone but skips flattenContentParts message-wide, so any additionally declared modality on a DeepSeek route bypasses the placeholder safety net that previously guaranteed the request went through. I confirmed this with an A/B run: with modalities: { image: true, pdf: true } (a declaration the model registry preserves verbatim), the converter emits an OpenAI type: 'file' part for the PDF; on this branch the raw file part ships as-is, while the pre-change code replaced it with [Unsupported content type: file] and the request always went through. If the endpoint's wire format rejects that part, a configuration that previously degraded gracefully now fails mid-conversation. The updated doc comment ("Models that explicitly declare image input retain multipart content") also states a narrower guarantee than the code implements. Suggested fix: make the skip part-type-aware — when image input is declared, keep image_url parts and still flatten/placeholder every other non-text part type.
中文说明
该判断只检查 modalities.image，却对整个消息跳过 flattenContentParts，因此 DeepSeek 路由上额外声明的其他模态会绕过此前保证请求始终可用的占位符安全网。A/B 运行验证：配置 modalities: { image: true, pdf: true }（模型注册表会原样保留该声明）时，转换器会为 PDF 生成 OpenAI type: 'file' 部分；本分支会把原始 file 部分原样发出，而改动前的代码会用 [Unsupported content type: file] 占位符替换、保证请求始终可用。如果端点的 wire 格式拒绝该部分，原本可以优雅降级的配置现在会在对话中途直接失败。此外，更新后的文档注释（"显式声明图片输入的模型保留多段内容"）所承诺的范围也小于代码的实际实现。建议修复：让跳过逻辑感知内容部分类型——声明图片输入时保留 image_url 部分，对其余非文本类型继续扁平化/占位。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
Third-party validation — this also breaks image passthrough through custom gateways, not just DeepSeek's own vision models I'm building a local gateway that relays qwen-code traffic to OpenAI-compatible endpoints and transcribes images into text for text-only models (so the upstream doesn't even need vision support). Reproduced this on v0.22.0 with a deepseek-named model served from a non-DeepSeek host:
Gating on declared While reading this class we noticed two more behaviors that follow the model-name fallback onto custom gateways, in the same "collateral damage" family as the flattening:
Meanwhile |
Maintainer verification — real local environment, A/B against the PR baseVerdict: works as advertised, recommend merge. The bug reproduces on the base commit, the fix resolves it end-to-end through the real CLI, and the text-only DeepSeek route is byte-for-byte unchanged. Three non-blocking notes below — one of them (N1) is worth a one-line follow-up either here or as a separate PR. How this was verified
1. The bug reproduces on base, and the fix resolves itIdentical config, identical prompt, identical file — only the code under test differs. In both arms
The image used (fixture) is a 320×320 PNG containing a purple-violet interlocking angular mark and a blue circle. The colours the gateway reports back — 2. What actually left the process
Both arms were run under equal-length filesystem paths so the byte counts are directly comparable:
3. No regression on the text-only DeepSeek routeSame scenario with Worth recording why: on a text-only config the converter already replaces the image with its own text placeholder before the provider runs, so the DeepSeek flatten never sees an image part on that route at all. The flatten's real remaining job there is joining text parts into a string, and that still happens. 4. Test evidence
5. Capability matrix
Notes (none blocking)N1 — the gate turns flattening off for every message, not just image-bearing onesThis is the one I'd like a follow-up on. On a permissive gateway that is harmless. On a route that genuinely requires string content — the constraint the flatten was written for, and which the PR explicitly allows to be paired with
The PR body does disclose this risk ("a text-only endpoint incorrectly configured with image support … may reject it"), but the disclosure reads as "image turns may be rejected", whereas what actually happens is "the session dies on hello". A misconfigured user gets an error that points nowhere near the image they were trying to send. Gating on the message instead of the config keeps the fix and shrinks the radius to the turn that actually carries an image: const keepMultipart = this.contentGeneratorConfig.modalities?.image === true;
const messages = reshaped.messages.map((message) =>
ensureReasoningContentOnAssistantMessage(
keepMultipart && hasImagePart(message)
? message
: flattenContentParts(message),
),
);
function hasImagePart(message: OpenAI.Chat.ChatCompletionMessageParam): boolean {
const content = (message as { content?: unknown }).content;
return (
Array.isArray(content) &&
content.some(
(part) =>
typeof part === 'object' &&
part !== null &&
(part as { type?: string }).type === 'image_url',
)
);
}I built and ran this variant: the image fix still works end-to-end (identical gateway description), text turns keep the legacy string shape, and on the strict endpoint only the image turn is rejected (lower pane of the screenshot above). Note that this variant also passes all 29 tests unchanged — so the current suite cannot tell the two gates apart. If you keep the broad gate, a test pinning N2 — "explicitly declares" is slightly stronger than what the code checksThe PR body says the multipart content is preserved "when the selected model explicitly declares image support", but the gate reads For the case this PR targets it makes no difference: N3 — informational: audio and PDF parts also stop being flattenedWith 中文版维护者验证 —— 真实本地环境，与 PR 基线做 A/B 对照结论：功能符合描述，建议合并。 缺陷在基线提交上可复现，修复在真实 CLI 上端到端生效，纯文本 DeepSeek 路由逐字节未变。下面三条备注均不阻断合并，其中 N1 建议在本 PR 或后续 PR 中补一行改动。 验证方式
1. 基线可复现缺陷，PR 修复生效配置、提示词、文件完全一致，仅被测代码不同。两臂中
所用图片（fixture）是 320×320 PNG，包含紫罗兰色交织棱角图形和蓝色圆形。网关回报的颜色 2. 实际发出的请求内容
两臂运行在等长文件系统路径下，字节数可直接比较：
3. 纯文本 DeepSeek 路由无回归同一场景改用 值得记录原因：纯文本配置下，converter 已经在 provider 运行之前把图片替换成了自己的文本占位符，因此该路由上 DeepSeek 的扁平化逻辑根本见不到图片部件。扁平化在此处真正剩下的职责是把文本部件拼成字符串，这一行为依然保留。 4. 测试证据
5. 能力矩阵
备注（均不阻断合并）N1 —— 该门控关闭的是所有消息的扁平化，而不仅是携带图片的消息这一条希望能有后续跟进。 在宽松网关上这无害。但在真正要求字符串内容的路由上——也正是扁平化逻辑当初针对的约束，而本 PR 明确允许它与
PR 描述确实披露了该风险（"纯文本端点被错误配置为支持图片……可能拒绝请求"），但这段表述读起来像是"图片回合可能被拒绝"，而实际情况是"会话在打招呼时就挂了"。配置有误的用户拿到的报错，与他试图发送的那张图片毫无指向关系。 改为按消息而非按配置门控，可以保留修复效果，同时把影响面收窄到真正携带图片的那一回合： const keepMultipart = this.contentGeneratorConfig.modalities?.image === true;
const messages = reshaped.messages.map((message) =>
ensureReasoningContentOnAssistantMessage(
keepMultipart && hasImagePart(message)
? message
: flattenContentParts(message),
),
);
function hasImagePart(message: OpenAI.Chat.ChatCompletionMessageParam): boolean {
const content = (message as { content?: unknown }).content;
return (
Array.isArray(content) &&
content.some(
(part) =>
typeof part === 'object' &&
part !== null &&
(part as { type?: string }).type === 'image_url',
)
);
}我构建并运行了该变体：图片修复端到端依然生效（网关给出完全相同的描述），文本回合保持旧有的字符串形态，在严格端点上只有图片回合被拒绝（见上图下半部分）。 需要注意的是，该变体同样能原封不动地通过全部 29 个测试——也就是说现有测试集无法区分这两种门控。如果决定保留宽门控，建议至少补一个用例，把 N2 —— "显式声明"的说法比代码实际检查的条件更强PR 描述称"当选中的模型明确声明支持图片输入时"保留多段内容，但门控读取的是 对本 PR 针对的场景没有影响： N3 —— 信息性：audio 与 PDF 部件同样不再被扁平化在 |
|
Released in v0.22.2. |
4 participants
What this PR does
This change preserves multipart image input for DeepSeek-compatible routes when the selected model explicitly declares image support. Text-only DeepSeek routes retain their existing request normalization, and regression coverage exercises both the official endpoint and a custom OpenAI-compatible gateway.
Why it's needed
DeepSeek provider selection also matches model names on custom gateways. That path previously flattened every multipart message after capability resolution, replacing an image with an unsupported-content placeholder even when the user explicitly configured image input. Vision-capable DeepSeek models therefore could not receive images through Qwen Code.
Reviewer Test Plan
How to verify
Configure a DeepSeek-named OpenAI-compatible model with image input enabled, send an image, and confirm the backend receives multipart content and describes the image. Then use a text-only DeepSeek configuration and confirm its multipart content still follows the existing string-flattening behavior.
Evidence (Before & After)
Before: the outgoing image part became
[Unsupported content type: image_url].After: the same local CLI path sent a real PNG to
deepseek-v4-flash-vision-exp; the model correctly identified a purple-violet interlocking angular mark and a blue circle. The focused provider test passes all 29 cases.Tested on
Environment (optional)
Local source CLI without sandbox; Node.js 24.18.0. Focused provider tests and the core package typecheck passed.
Risk & Scope
Linked Issues
Fixes #9832
中文说明
本 PR 的改动
当选中的模型明确声明支持图片输入时，此改动为 DeepSeek 兼容路由保留多段图片内容。纯文本 DeepSeek 路由继续使用现有请求整形，并通过官方端点和自定义 OpenAI 兼容网关的回归用例覆盖该行为。
为什么需要此改动
DeepSeek 提供商选择也会在自定义网关上按模型名匹配。此前该路径会在能力解析后无条件扁平化所有多段消息，即使用户明确配置了图片输入，图片仍会被替换成不支持内容类型的占位符。因此，具备视觉能力的 DeepSeek 模型无法通过 Qwen Code 接收图片。
审查者测试计划
验证方式
配置一个启用图片输入的 DeepSeek 命名 OpenAI 兼容模型，发送图片，并确认后端收到多段内容且能够描述图片。随后使用纯文本 DeepSeek 配置，确认其多段内容仍采用现有字符串扁平化行为。
证据（修复前后）
修复前：发出的图片部分会变成
[Unsupported content type: image_url]。修复后：同一本地 CLI 路径向
deepseek-v4-flash-vision-exp发送了真实 PNG；模型正确识别出紫罗兰色交织棱角图形和蓝色圆形。提供商专项测试的 29 个用例全部通过。测试平台
环境（可选）
本地源码 CLI，未启用沙箱；Node.js 24.18.0。提供商专项测试和核心包类型检查均已通过。
风险与范围
关联问题
Fixes #9832