[Bugfix] Fix Harmony preamble visibility in Responses API

[thepushkarp]

Summary

Per the Harmony spec, preambles (commentary channel messages with no recipient) are "intended to be shown to end-users". vLLM was incorrectly treating them as hidden reasoning across multiple code paths.

This PR fixes preamble visibility in 6 code paths across 3 files:

Parser (harmony_utils.py):

• parse_output_message(): route preambles to ResponseOutputMessage instead of ResponseReasoningItem
• parse_remaining_state(): return visible ResponseOutputMessage (streaming) instead of hidden reasoning
• parse_chat_output(): include preambles in final_texts (Chat Completions API), exclude tool call JSON that was leaking into visible content

Streaming events (streaming_events.py):

• emit_content_delta_events(): emit response.output_text.delta for preamble tokens (were silently dropped)
• emit_previous_item_done_events(): emit text done events for completed preambles

Token counting (context.py):

• _update_num_reasoning_tokens(): exclude preamble tokens from reasoning_tokens count

Channel config (harmony_utils.py):

• get_system_message(): stop stripping commentary from valid channels when with_custom_tools=False. The spec always declares all 3 channels; without commentary the model cannot generate preambles in built-in-tool-only sessions (web_search, code_interpreter).

Before/After

The model generates a preamble to preview an upcoming tool call:

<|channel|>commentary
<|message|>I'll search for the weather in SF.<|end|>
<|start|>assistant to=functions.web_search
<|channel|>commentary
<|message|>{"query": "weather in SF"}<|end|>

Responses API — before:

{"type": "reasoning", "content": [{"type": "reasoning_text", "text": "I'll search for the weather in SF."}]}

Preamble hidden as reasoning — user never sees it.

Responses API — after:

{"type": "message", "content": [{"type": "output_text", "text": "I'll search for the weather in SF."}]}

Preamble visible as a message.

Streaming — before: Preamble tokens emitted no SSE events (silently dropped).
Streaming — after: Preamble tokens emit response.output_text.delta events, same as final channel text.

Chat Completions — before: content included tool call JSON ({"query": "weather in SF"}) because the filter was channel != "analysis".
Chat Completions — after: content only includes preambles and final text; tool call payloads excluded.

Token counting — before: Preamble tokens counted as reasoning_tokens.
Token counting — after: Preamble tokens counted as regular output tokens.

Channel config — before: commentary stripped from valid channels when no custom tools — model can't generate preambles at all.
Channel config — after: All 3 channels always present per spec.

---

Note

Makes Harmony preambles user-visible and adjusts parser behavior accordingly.

• Parse commentary with no recipient as ResponseOutputMessage (visible content) rather than ResponseReasoningItem (hidden reasoning_content)
• Update parse_remaining_state to emit message (status incomplete) for commentary preambles; keep analysis as reasoning
• Clarify built-in tools (python, browser, container) return reasoning; non-builtin recipients become MCP calls; functions remain function_call
• Tests updated to expect ResponseOutputMessage for preambles, single message with multiple contents, and streaming status; added coverage for parser edge cases

^{Written by Cursor Bugbot for commit 50cc9b1. This will update automatically on new commits. Configure here.}

---

Note

Makes Harmony preambles visible to users and aligns commentary parsing with the Harmony spec.

• Commentary with no recipient now emits a ResponseOutputMessage (visible content) rather than ResponseReasoningItem; analysis remains reasoning
• Streaming parser updated: commentary preambles return a message with status="incomplete"; built-in tools (python, browser, container) explicitly return reasoning; MCP parsing unchanged
• Tests updated/added to cover preamble visibility, multi-content aggregation into a single message, and streaming states

^{Written by Cursor Bugbot for commit 8711d3d. This will update automatically on new commits. Configure here.}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[gemini-code-assist]

Code Review

This pull request correctly addresses a bug in the handling of Harmony format preambles in the Responses API. Previously, preambles (commentary messages with no recipient) were incorrectly treated as hidden reasoning content. The changes ensure they are now parsed as visible ResponseOutputMessage items, aligning with the intended behavior of showing them to end-users.

The fix is implemented consistently for both complete messages in parse_output_message and for streaming scenarios in parse_remaining_state. The logic is sound, and the accompanying test updates in test_harmony_utils.py are thorough, including new assertions for the corrected output type and a new test case for streaming preambles. The changes appear correct and well-tested.

[chaunceyjiang]

/cc @qandrew PTAL.

[qandrew]

@thepushkarp can you add in the PR description what an example request / response would look like, before / after this change? Would make it easier to review and understand

[thepushkarp]

Updated the description and added a few more fixes @qandrew!

sorry for the delay (๑•́ㅿ•̀๑)ᔆᵒʳʳᵞ

[qandrew]

lgtm, thanks for the updates! It would be nice to merge chatCompletion / responsesAPI code paths to simplify our logic too

cc @DarkLight1337 @chaunceyjiang to merge

[chaunceyjiang]

Thanks~ @thepushkarp I’ll take a look at this today.

[chaunceyjiang]

@thepushkarp The CI failed — could you take a look?

[thepushkarp]

Hey @chaunceyjiang, the previous error was fixed. There are still some build failures from tests unrelated to the changes in this PR. Can you please check and let me know what to do for them?

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @thepushkarp.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[thepushkarp]

Hi @chaunceyjiang

The remaining tests are passing, except for two in the v1/e2e/test_spec_decode.py module. Please advise on what to do here.

[DarkLight1337]

Force-merging

[sfeng33]

lgtm, thanks for the updates! It would be nice to merge chatCompletion / responsesAPI code paths to simplify our logic too

cc @DarkLight1337 @chaunceyjiang to merge

I can work on merging the harmony parsing overlaps on chatCompletion & responsesAPI, there are many duplicate routing logic.