fix(core): disable qwen thinking via chat_template_kwargs on non-DashScope servers - #6271
Conversation
…Scope servers For hybrid-thinking qwen models, enable_thinking:false was only emitted for DashScope providers, always as a top-level request field. Self-hosted OpenAI-compatible servers (vLLM, SGLang) render the chat template server-side and read the switch from chat_template_kwargs; they silently ignore a top-level enable_thinking. The visible failure is that auto-approval mode is unusable against a self-hosted qwen endpoint: the permission classifier issues short structured-output calls with a small token budget, and because thinking is never disabled the model spends that budget emitting <think>, returns truncated JSON, and every tool call fails closed. When the model is a qwen/coder-model and the provider is not DashScope, set chat_template_kwargs.enable_thinking = false (merged with any existing chat_template_kwargs) instead of the top-level field. DashScope behavior is unchanged. Adds a unit test for the non-DashScope path.
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Hi @beantownbytes, thanks for the PR — the fix itself looks well-targeted, but the PR body doesn't follow our pull request template.
The body uses custom headings (Problem / Fix / Notes) instead of the template's required sections. The missing ones are:
- What this PR does / Why it's needed — your Problem/Fix sections cover this content, but we need the standard headings for consistency
- Reviewer Test Plan — this is the big one. The template requires "How to verify", "Evidence (Before & After)", and a "Tested on" table. Your mention of vLLM verification is helpful context but doesn't substitute for a structured test plan that reviewers can follow
- Risk & Scope — main risk/tradeoff, what's out of scope, breaking changes
- Linked Issues — reference related issues (you mention #4676 in Notes, which should go here)
- 中文说明 — the Chinese translation in the
<details>block
Could you restructure the body to match the template? The content is there — it just needs the right format so maintainers can review efficiently.
中文说明
你好 @beantownbytes,感谢你的 PR!修复方向很精准,但 PR 正文没有使用我们的 PR 模板。
目前使用的是自定义标题(Problem / Fix / Notes),缺少模板要求的以下部分:
- What this PR does / Why it's needed — 你的 Problem/Fix 部分已覆盖这些内容,但需要使用标准标题以保持一致性
- Reviewer Test Plan — 这是最重要的部分。模板要求包含"How to verify"、"Evidence (Before & After)"以及"Tested on"表格。你提到了 vLLM 验证,这是有帮助的背景信息,但不能替代一个结构化的、审查者可以跟随的测试计划
- Risk & Scope — 主要风险/权衡、不在范围内的内容、破坏性变更
- Linked Issues — 引用相关 issue(你在 Notes 中提到了 #4676,应该放在这里)
- 中文说明 —
<details>块中的中文翻译
能否按照模板重新组织正文?内容都在——只是需要正确的格式,以便维护者高效审查。
— Qwen Code · qwen3.7-max
| ) | ||
| ) { | ||
| typed['enable_thinking'] = false; | ||
| } else { |
There was a problem hiding this comment.
[Critical] The non-DashScope branch sets chat_template_kwargs: { enable_thinking: false } but does not strip any top-level enable_thinking that extra_body may have already injected into the request.
When a provider preset (e.g. ModelScope's Qwen/Qwen3.5-397B-A17B) has enableThinking: true, provider-config.ts compiles it into extra_body: { enable_thinking: true }, and DefaultOpenAICompatibleProvider.buildRequest spreads that into the top-level request. The resulting wire request then carries both "enable_thinking": true (top-level) and "chat_template_kwargs": { "enable_thinking": false } — contradictory signals.
While vLLM/SGLang may ignore the top-level field (as the comment notes), other OpenAI-compatible servers (ModelScope, LiteLLM proxies) that read both could keep thinking enabled despite the explicit opt-out. The new test doesn't catch this because its mock doesn't inject enable_thinking: true via extra_body, so expect(apiCall.enable_thinking).toBeUndefined() passes trivially.
| } else { | |
| } else { | |
| // Non-DashScope OpenAI-compatible servers (vLLM, SGLang, ...) render | |
| // the model's chat template server-side and read the thinking switch | |
| // from `chat_template_kwargs`, not a top-level `enable_thinking` | |
| // (which they silently ignore). Send it there so hybrid qwen models | |
| // actually stop emitting <think> when reasoning is disabled — e.g. | |
| // the auto-mode permission classifier's short structured-output | |
| // calls, which otherwise spend their small token budget on thinking | |
| // and fail closed. | |
| // Strip any top-level enable_thinking injected by extra_body | |
| // (provider-config.ts sets it for models with enableThinking: true). | |
| delete typed['enable_thinking']; | |
| const existing = (typed['chat_template_kwargs'] ?? {}) as Record< | |
| string, | |
| unknown | |
| >; | |
| typed['chat_template_kwargs'] = { | |
| ...existing, | |
| enable_thinking: false, | |
| }; | |
| } |
— qwen3.7-max via Qwen Code /review
| expect(apiCall.enable_thinking).toBeUndefined(); | ||
| }); | ||
|
|
||
| it('disables qwen thinking via chat_template_kwargs on a non-DashScope endpoint (vLLM/SGLang)', async () => { |
There was a problem hiding this comment.
[Suggestion] The only coder-model test (line ~986) uses AuthType.QWEN_OAUTH, which causes isDashScopeProvider to return true. The || model === 'coder-model' branch inside the non-DashScope else arm is therefore never exercised in tests.
Consider adding a test with model: 'coder-model', baseUrl: 'https://llm.example.com/v1' (non-DashScope), includeThoughts: false, asserting chat_template_kwargs equals { enable_thinking: false }.
— qwen3.7-max via Qwen Code /review
| ) { | ||
| typed['enable_thinking'] = false; | ||
| } else { | ||
| // Non-DashScope OpenAI-compatible servers (vLLM, SGLang, ...) render |
There was a problem hiding this comment.
[Suggestion] chat_template_kwargs is a vLLM/SGLang convention, but this else-branch fires for every non-DashScope endpoint — including LiteLLM proxies, Ollama, and other OpenAI-compatible servers. Most will silently ignore unknown fields (harmless), but servers that validate request bodies strictly could reject the request.
Consider documenting this assumption in the comment, or gating the field on a known server list.
— qwen3.7-max via Qwen Code /review
…cope path Address review: when a qwen model runs against a non-DashScope endpoint and a provider preset injected enable_thinking:true via extra_body, emitting only chat_template_kwargs.enable_thinking=false left the contradictory top-level field in place. Servers that honour both signals could keep thinking enabled despite the opt-out. Delete the top-level field on this path so the nested switch is authoritative, matching the codebase's rule of not leaking the qwen-specific enable_thinking field to non-DashScope servers. Also covers the coder-model non-DashScope arm with a dedicated test, and strengthens the vLLM test to inject a top-level enable_thinking:true and assert it is stripped.
|
Thanks for the review. Addressed in 652ab68, and the PR body is restructured to the template. [Critical] top-level [Suggestion] coder-model non-DashScope arm untested - added a dedicated test: [Suggestion] gate |
| ); | ||
| (mockClient.chat.completions.create as Mock).mockResolvedValue({ | ||
| id: 'r', | ||
| choices: [{ message: { content: 'ok' }, finish_reason: 'stop' }], |
There was a problem hiding this comment.
[Suggestion] The test now verifies that a top-level enable_thinking: true injected via extra_body gets stripped — nice improvement. However, the chat_template_kwargs merge with pre-existing keys (other than enable_thinking) is still untested.
The production code at pipeline.ts spreads ...existing before appending enable_thinking: false. Consider adding a test case where mockProvider.buildRequest also returns chat_template_kwargs: { apply_chat_template: true } and asserting:
expect(apiCall.chat_template_kwargs).toEqual({ apply_chat_template: true, enable_thinking: false });This guards against a future refactor that drops the ...existing spread and silently loses user-configured kwargs.
— qwen3.7-max via Qwen Code /review
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No review findings. The fix correctly restructures the thinking-disable logic for non-DashScope servers and is well-covered by tests (83 passing). Downgraded from Approve to Comment: CI still running.
— qwen3.7-max via Qwen Code /review
Address review: assert the else-branch spreads pre-existing
chat_template_kwargs before appending enable_thinking:false, so a refactor
that drops the spread and loses user-configured kwargs is caught. Injects
chat_template_kwargs:{apply_chat_template:true} via buildRequest and expects
the merged {apply_chat_template:true, enable_thinking:false}.
|
Good call - added in 874553a. New test injects |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No review findings. The fix correctly restructures the thinking-disable logic for non-DashScope servers: chat_template_kwargs: { enable_thinking: false } replaces the top-level field that vLLM/SGLang ignore, the delete typed['enable_thinking'] properly strips any contradictory provider-preset injection, and the merge with pre-existing chat_template_kwargs preserves user-configured keys. Test coverage is solid (84 passing) — qwen model, coder-model alias, and kwargs merge are all exercised on the non-DashScope path. Downgraded from Approve to Comment: CI still running.
— qwen3.7-max via Qwen Code /review
|
@qwen-code /triage |
|
Thanks for the PR, @beantownbytes! Template looks good ✓ Problem: This is a real, observed bug — not theoretical hardening. Self-hosted OpenAI-compatible servers (vLLM, SGLang) read the thinking-disable switch from Direction: Clearly aligned with qwen-code's core mission — users pointing the CLI at self-hosted qwen endpoints expect auto-mode to work. The DashScope path already had this; extending it to non-DashScope servers closes a real gap. No CHANGELOG reference (this is qwen-specific plumbing), but the area is clearly in-scope. Approach: Scope is tight — 2 files, one focused change. The restructuring is the minimal fix: split the existing One note: the inline comments in the non-DashScope branch are thorough but verbose (~15 lines of prose for ~8 lines of code). Not a blocker — just flagging for the author's awareness in case they want to tighten it up. Moving on to code review. 🔍 中文说明感谢 PR! 模板完整 ✓ 问题: 这是一个真实存在的 bug,不是理论性加固。自托管的 OpenAI 兼容服务器(vLLM、SGLang)从 方向: 与 qwen-code 的核心使命明确对齐——用户把 CLI 指向自托管 qwen 端点时,期望自动模式能正常工作。DashScope 路径已经有了这个功能;扩展到非 DashScope 服务器填补了一个真实的缺口。CHANGELOG 无直接参考(这是 qwen 特有的底层逻辑),但该领域明确在范围内。 方案: 范围紧凑——2 个文件,一个聚焦的改动。重构是最小修复:将现有的 一个观察:非 DashScope 分支中的内联注释详尽但冗长(约 15 行文字对应约 8 行代码)。不阻塞——仅提醒作者考虑是否可以精简。 进入代码审查 🔍 — Qwen Code · qwen3.7-max |
Code ReviewThe diff restructures the thinking-disable logic in
The Reuse check: the non-DashScope branch builds No blockers found. The logic is correct, well-scoped, and consistent with the surrounding code patterns. Test ResultsBefore (main code + PR's new tests)Tests run against After (this PR)All 84 tests pass with the PR's production code change: Note: This is a request-shaping fix, not a TUI change. Live verification requires a self-hosted vLLM/SGLang endpoint, which is not available in this CI environment. The unit tests pin the exact request body shape and are sufficient to verify the fix. 中文说明代码审查diff 清晰地重构了
复用检查: 非 DashScope 分支内联构建 未发现阻塞问题。 逻辑正确,范围合理,与周围代码模式一致。 测试结果改动前(main 代码 + PR 新测试)在 main 分支的生产代码上运行 PR 的 3 个新测试用例。全部 3 个新测试失败—— 改动后(本 PR)所有 84 个测试通过: 注: 这是请求格式修复,不是 TUI 改动。实时验证需要自托管 vLLM/SGLang 端点,CI 环境不具备此条件。单元测试固定了精确的请求体格式,足以验证修复。 — Qwen Code · qwen3.7-max |
|
This PR does exactly what it says: fixes a real bug where auto-mode is broken against self-hosted qwen endpoints because the thinking-disable switch was only being sent in the shape DashScope understands, not the shape vLLM/SGLang reads. My independent proposal for this fix would have been identical — restructure the existing The before/after is clean: 3 new tests fail on main (the non-DashScope path never set The author has been responsive to prior review feedback — addressed the critical item (top-level stripping), added the missing coder-model test, and documented the no-allowlist design decision with clear reasoning. Verdict: Approve. The fix is correct, focused, and well-tested. Ships a real user-facing improvement with no risk to the existing DashScope path. ✅ 中文说明这个 PR 完成了它声称的功能:修复了一个真实的 bug——自动模式在自托管 qwen 端点上不可用,因为禁用思考的开关只以 DashScope 理解的格式发送,而不是 vLLM/SGLang 读取的格式。 我对这个修复的独立方案会完全相同——将现有的 Before/after 很清晰:3 个新测试在 main 上失败(非 DashScope 路径从未设置 作者对先前的审查反馈响应及时——解决了关键问题(顶层字段移除)、添加了缺失的 coder-model 测试、并用清晰的推理记录了不做白名单的设计决策。 结论: 批准。修复正确、聚焦、测试充分。带来了真实的用户侧改进,对现有 DashScope 路径无风险。✅ — Qwen Code · qwen3.7-max |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
What this PR does
For hybrid-thinking qwen models, the "disable thinking" switch is now delivered in a way that self-hosted OpenAI-compatible servers actually honour. When the wire model is a
qwen*model orcoder-modeland the provider is not DashScope, the pipeline setschat_template_kwargs.enable_thinking = false(merged with any existingchat_template_kwargs) instead of the top-levelenable_thinkingfield, and strips any top-levelenable_thinkinga provider preset injected viaextra_bodyso the two signals cannot contradict. The DashScope path is unchanged.Why it's needed
Previously
enable_thinking: falsewas only emitted for DashScope providers, and always as a top-level request field. Self-hosted OpenAI-compatible servers (vLLM, SGLang) render the chat template server-side and read the thinking switch fromchat_template_kwargs; they silently ignore a top-levelenable_thinking. So on a self-hosted qwen endpoint, thinking is never actually disabled.The user-facing failure is that auto-approval mode is unusable against such an endpoint. The permission classifier issues short structured-output calls with a small token budget; because thinking stays on, the model spends that budget emitting
<think>, returns empty or truncated JSON, and every tool call fails closed.Reviewer Test Plan
How to verify
Point qwen-code at a self-hosted vLLM (or SGLang) server serving a hybrid-thinking qwen model such as
Qwen3.6-27B, configured as an OpenAI-compatible provider (non-DashScope base URL), and enable auto-approval mode.Before this change: the auto-mode permission classifier calls come back with
content: null(the model consumes the token budget on<think>), and auto mode fails closed on every tool call. After this change: the classifier returns valid JSON and auto mode works, because the server seeschat_template_kwargs.enable_thinking = falseand stops emitting reasoning.Unit coverage:
npx vitest run packages/core/src/core/openaiContentGenerator/pipeline.test.ts(83 passing). New/updated cases assert that aqwen*model and acoder-modelon a non-DashScope endpoint emitchat_template_kwargs: { enable_thinking: false }with no top-levelenable_thinking, and that a top-levelenable_thinking: trueinjected viaextra_bodyis stripped on this path.Evidence (Before & After)
Not a TUI change. Observable request/response difference on a self-hosted vLLM endpoint: before, the classifier response is
content: null; after, it is valid JSON (e.g.{"decision":"allow"}). Behavior is pinned by the unit tests above.Tested on
macOS: unit tests + live verification against a vLLM v0.22.1 server. Windows/Linux: not run locally; the change is pure request-shaping logic covered by cross-platform unit tests in CI.
Environment (optional)
Local unit tests via
npx vitest; live check against vLLMv0.22.1servingQwen3.6-27BNVFP4.Risk & Scope
chat_template_kwargson every non-DashScope qwen request. Servers that do not recognise the field ignore it (harmless no-op); this is the same convention vLLM and SGLang already use. It also deletes a top-levelenable_thinkingon this path, which is intentional (the field is qwen-specific and should not leak top-level to non-DashScope servers).Linked Issues
Related to #4676 (auto-mode classifier timeouts / stage-2 thinking), which routes through the same DashScope-only path and does not reach self-hosted servers. No closing keyword; this is a distinct fix.
中文说明
这个 PR 做了什么
对于混合思考(hybrid-thinking)的 qwen 模型,现在以自托管的 OpenAI 兼容服务器能真正识别的方式来传递"禁用思考"开关。当线上(wire)模型是
qwen*模型或coder-model且提供方不是 DashScope 时,pipeline 会设置chat_template_kwargs.enable_thinking = false(与已有的chat_template_kwargs合并),而不是使用顶层的enable_thinking字段;同时会移除提供方预设通过extra_body注入的任何顶层enable_thinking,以免两个信号相互矛盾。DashScope 路径保持不变。为什么需要它
此前
enable_thinking: false只对 DashScope 提供方发送,并且总是作为顶层请求字段。自托管的 OpenAI 兼容服务器(vLLM、SGLang)在服务端渲染聊天模板,并从chat_template_kwargs读取思考开关;它们会静默忽略顶层的enable_thinking。因此在自托管的 qwen 端点上,思考实际上从未被禁用。面向用户的故障是:自动批准(auto-approval)模式在这类端点上无法使用。权限分类器(permission classifier)会发出预算很小的短结构化输出请求;由于思考仍然开启,模型把预算花在输出
<think>上,返回空的或被截断的 JSON,导致每次工具调用都失败关闭(fail closed)。审查者测试计划
如何验证
将 qwen-code 指向一个自托管的 vLLM(或 SGLang)服务器,服务一个混合思考的 qwen 模型(如
Qwen3.6-27B),配置为 OpenAI 兼容提供方(非 DashScope 的 base URL),并启用自动批准模式。改动之前:自动模式的权限分类器请求返回
content: null(模型把 token 预算消耗在<think>上),每次工具调用都失败关闭。改动之后:分类器返回有效的 JSON,自动模式正常工作,因为服务器看到chat_template_kwargs.enable_thinking = false并停止输出推理内容。单元测试覆盖:
npx vitest run packages/core/src/core/openaiContentGenerator/pipeline.test.ts(83 个通过)。新增/更新的用例断言:非 DashScope 端点上的qwen*模型和coder-model会发送chat_template_kwargs: { enable_thinking: false }且没有顶层enable_thinking;并且通过extra_body注入的顶层enable_thinking: true会在该路径上被移除。证据(前后对比)
不是 TUI 改动。在自托管 vLLM 端点上可观察到的请求/响应差异:之前分类器响应为
content: null;之后为有效 JSON(例如{"decision":"allow"})。该行为由上述单元测试固定。测试平台
macOS:单元测试 + 针对 vLLM v0.22.1 服务器的实机验证。Windows/Linux:本地未运行;该改动是纯请求构造逻辑,由 CI 中的跨平台单元测试覆盖。
环境(可选)
通过
npx vitest运行本地单元测试;针对服务Qwen3.6-27BNVFP4 的 vLLMv0.22.1做了实机检查。风险与范围
chat_template_kwargs。不识别该字段的服务器会忽略它(无害的空操作);这正是 vLLM 和 SGLang 已经采用的约定。它还会在该路径上删除顶层的enable_thinking,这是有意为之(该字段是 qwen 专有的,不应以顶层形式泄漏到非 DashScope 服务器)。关联 Issue
与 #4676(自动模式分类器超时 / 第二阶段思考)相关,该 issue 走的是同一条仅限 DashScope 的路径,无法到达自托管服务器。不使用关闭关键字;这是一个独立的修复。