Skip to content

docs(providers): clarify vllm qwen reasoning output - #47850

Closed
HwangJohn wants to merge 1 commit into
NousResearch:mainfrom
HwangJohn:docs/38360-vllm-qwen-reasoning-parser
Closed

HwangJohn wants to merge 1 commit into
NousResearch:mainfrom
HwangJohn:docs/38360-vllm-qwen-reasoning-parser

Conversation

@HwangJohn

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR documents the expected behavior for Qwen reasoning models served through OpenAI-compatible vLLM endpoints.

Hermes can preserve structured reasoning metadata such as reasoning, reasoning_content, and streamed reasoning deltas, but that metadata is treated as reasoning/thinking trace data. The assistant's user-visible answer should still be emitted in content.

For vLLM/Qwen deployments where --reasoning-parser qwen3 leaves content empty, the docs now point users to two stable options:

  • disable the reasoning parser for that deployment, or
  • pass a server-supported request option such as chat_template_kwargs.enable_thinking: false through extra_body.

Related Issue

Fixes #38360

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • website/docs/integrations/providers.md
    • Clarifies how Hermes treats Qwen/vLLM reasoning-parser output.
    • Adds the extra_body.chat_template_kwargs.enable_thinking: false workaround for deployments that separate all generated text into reasoning fields.
  • cli-config.yaml.example
    • Adds the nested vLLM/Qwen extra_body shape to the auxiliary-model comment example.

How to Test

  1. Read the vLLM provider section in website/docs/integrations/providers.md.
  2. Confirm it explains that separated reasoning metadata is not a replacement for visible content.
  3. Confirm the custom-provider extra_body section shows the nested chat_template_kwargs.enable_thinking: false shape.

Validation run:

  • Local Windows:
    • git diff --check
  • DGX Spark Linux:
    • git diff --check
    • cd website && npm ci && npm run build

Results:

  • Diff check: passed
  • Docusaurus build: passed
  • Note: the docs build still reports existing broken link/anchor warnings elsewhere in the site; the build exits successfully and this PR does not add new links.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11 local diff check and DGX Spark Linux docs build

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — yes
  • I've updated cli-config.yaml.example if I added/changed config keys — comment-only example update
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — N/A, docs-only
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

For New Skills

N/A

Screenshots / Logs

DGX Spark docs build:

[SUCCESS] Generated static files in "build".
[SUCCESS] Generated static files in "build/zh-Hans".

Signed-off-by: HwangJohn <angelic805@gmail.com>

Co-authored-by: OpenAI Codex <codex@openai.com>
@alt-glitch alt-glitch added type/docs Documentation improvements provider/qwen Qwen / Alibaba Cloud (OAuth) P3 Low — cosmetic, nice to have labels Jun 17, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Merged via PR #49944 (docs salvage cluster 2). Your commit was cherry-picked onto current main with your authorship preserved in git log — rebase-merged so it lands under your name. Thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have provider/qwen Qwen / Alibaba Cloud (OAuth) type/docs Documentation improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clarify support for Qwen/vLLM reasoning-parser output in custom OpenAI-compatible providers

3 participants