[thegoldenflow]

Purpose

Fixes #50768. On /v1/responses with a Kimi model and --tool-call-parser kimi_k2, multi-turn agentic sessions intermittently lose the model's tool call and the turn comes back empty — the reporter's "performs work but never answers", at a rate that climbs sharply with the number of turns (9/15 on short tasks, 0-3/15 on long ones).

Scope note on the issue: the reporter also captured a payload with raw <|tool_call_begin|> call_... markup inside output_text. This PR does not explain that half. On their exact build (e2fa28594, which is 5 commits behind
this base and byte-identical across vllm/parser/, vllm/entrypoints/openai/responses/, vllm/entrypoints/chat_utils.py,
vllm/tool_parsers/) a call_<uuid> header produces a dropped tool call with the markers consumed, not leaked. The leak is likely a second, separate problem — #48940 covers special-token leakage when tools/tool_choice are omitted on a
later agent-loop turn, which is a plausible candidate but unconfirmed here. I've asked the reporter for the detail that would separate the two.

Root cause (verified on main @ 0055b8bfa)

Kimi's chat template renders a tool-call ID verbatim, in two places, and renders the function name nowhere else:

{%- set formatted_id = tool_call['id'] -%}
<|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>...<|tool_call_end|>
...
{%- set tool_call_id = message.tool_call_id -%}
## Return of {{ tool_call_id }}

Verified identical on these two lines across Kimi-K2-Instruct, Kimi-K2-Thinking, Kimi-K2.5 and Kimi-K2.7-Code (the last two are what the reporter runs). All of K2.5 / K2.6 / K2.7-Code report model_type: kimi_k25 with text_config.model_type: kimi_k2, both of which are already in _KIMI_MODEL_TYPES, so get_tool_call_id_type gates them correctly. (Moonshot's own tool_call_guidance.md documents the ID format as functions.{func_name}:{idx} "from which we can parse the function name".)

The chain:

1. Output. On the streaming Responses path, emit_simple_tool_call_open (vllm/entrypoints/openai/responses/streaming_events.py) sets state.tool_call_id = f"call_{random_uuid()}", discarding the native ID the parser put on the DeltaToolCall (ParserEngine._emit_name_delta → _ensure_tool_id). The client sees call_<uuid>.
2. Input. Next turn the client echoes that history back. construct_chat_messages_with_tool_call
   (vllm/entrypoints/openai/responses/utils.py) passed call_id straight through onto both tool_calls[].id and the tool message's tool_call_id.
3. Prompt. The template therefore renders <|tool_call_begin|>call_<uuid> and ## Return of call_<uuid>. Because the function name lives only inside the ID, prior turns become unreadable to the model — it can see that a tool ran and what it returned, but not which tool. It also learns to imitate the call_<uuid> header shape.
4. Parse. KimiK2Parser._extract_tool_id_and_name matches _TOOL_ID_RE = r"(?P<id>.+:\d+)". call_4144fac2c5e1f3e0 has no :N, so it returns (None, None) and ParserEngine._build_extracted_result skips the slot — the tool call is dropped silently. That silent drop is the "no answer" symptom, and it compounds with turn count because every extra turn adds more contaminated history.

This is the Responses-path twin of #32768, which fixed chat completions via make_tool_call_id(id_type="kimi_k2") / _KIMI_MODEL_TYPES in vllm/entrypoints/chat_utils.py. Chat completions never needed a rewrite because they hand the native ID to the client and get it back unchanged; the Responses path substitutes its own ID, so the round trip has to be undone on the way in.

Fix

Input side only. When the served model is Kimi-family, construct_chat_messages_with_tool_call renumbers any history tool-call ID that is not already native to functions.{tool_name}:{idx}, and applies the same old -> new mapping to the
matching tool message's tool_call_id so the pair stays consistent (the template needs both).

idx is a single counter over the whole input array, matching count_response_history_tool_calls in vllm/parser/utils.py — the same counter the parser uses to number the next tool call. Per-turn numbering would produce duplicate ## Return of functions.foo:0 headers and collide with the parser's next index; there is a test pinning the no-collision invariant.

Behaviour is idempotent (already-native IDs pass through untouched) and byte-for-byte unchanged for non-Kimi models — get_tool_call_id_type returns "random" for everything else and the rewriter is an identity function in that case.

Second commit: KimiK2Parser now logs a warning_once naming an unparseable <|tool_call_begin|> header before dropping the call, so this class of failure is diagnosable from server logs instead of silent. No change to the shared
ParserEngine.

Why input-side rewrite, not preserving the native ID on the wire

The alternative is to stop minting call_<uuid> in streaming_events.py and emit functions.exec_command:0 as the Responses call_id. Rejected here:

• It changes the public wire format for every Kimi Responses user, and OpenAI clients have varying expectations about call_id shape. Input-side rewriting is invisible to clients.
• It only fixes histories that vLLM itself generated. A client that mints its own call_ids, or a proxy that rewrites them, would still poison the prompt. The input-side rewrite covers those too.
• The non-streaming Responses path already preserves the native ID (build_response_output_items uses tool_call.id or ...), so the wire format is currently inconsistent between streaming and non-streaming. The input-side fix makes both correct without having to pick a wire format first.

Worth doing separately: making the streaming and non-streaming wire IDs agree. Deliberately out of scope here.

Scope note

This describes the mechanism as verified on current main. No bisect was run, so this makes no claim about which commit changed the behaviour relative to v0.26.0; the reporter's observation that v0.26.0 behaved differently is recorded as their observation only.

Known limitation, not addressed here

When a tool call is dropped (malformed header, first in a section), the surviving calls keep their raw slot indices, so the stream emits e.g. index=1 with no index=0 before it. Renumbering slots is a ParserEngine-wide change; leaving it for a separate PR. The new parser test places the malformed call last and documents why.

Test Plan

.venv/bin/python -m pytest tests/entrypoints/openai/responses/test_responses_utils.py -v
.venv/bin/python -m pytest tests/tool_parsers/test_kimi_k2_tool_parser.py -v
.venv/bin/python -m pytest tests/entrypoints/openai/responses/ tests/tool_parsers/ \
    tests/reasoning/test_kimi_k2_reasoning_parser.py \
    tests/entrypoints/unit_tests/test_chat_utils.py -q
pre-commit run --files vllm/entrypoints/chat_utils.py \
    vllm/entrypoints/openai/responses/utils.py \
    vllm/entrypoints/openai/responses/serving.py vllm/parser/kimi_k2.py \
    tests/entrypoints/openai/responses/test_responses_utils.py \
    tests/tool_parsers/test_kimi_k2_tool_parser.py

New coverage in tests/entrypoints/openai/responses/test_responses_utils.py (TestNativeToolCallIds):

• call_<uuid> history on a Kimi model → functions.{name}:{idx} on both the assistant tool_calls[].id and the paired tool message tool_call_id; two calls in one turn get :0 / :1.
• Indices run across assistant turns, and no history ID collides with the ID the parser will mint next (asserted against count_response_history_tool_calls).
• Idempotence: already-native IDs unchanged.
• Non-Kimi models: call_<uuid> preserved, output identical to the default.
• Dict-shaped function_call_output items rewritten (these are not coerced to typed items by ResponsesRequest.input_item_parsing, unlike function_call).
• Namespaced tools use the flattened functions.{namespace}__{name}:{idx} name.
• construct_input_messages threads the ID type through.

In tests/tool_parsers/test_kimi_k2_tool_parser.py: a call_<uuid> header drops only that call, leaks no <|tool_call_*|> markers into content, leaves a sibling well-formed call parseable (streaming and non-streaming), and is logged.

Test Result

Focused suites:

tests/entrypoints/openai/responses/test_responses_utils.py   47 passed
tests/tool_parsers/test_kimi_k2_tool_parser.py               53 passed

Regression sweep over everything that touches the changed code (tests/entrypoints/openai/responses/, tests/tool_parsers/, tests/reasoning/test_kimi_k2_reasoning_parser.py, tests/entrypoints/unit_tests/test_chat_utils.py):

	passed	errors
main (baseline)	1101	120
this branch	1111	120

+10 = the new tests; the error count is unchanged. All 120 errors are pre-existing setup failures for tests that need a live server or a GPU (test_simple.py, test_stateful.py, test_structured_output.py, the multimodal test_chat_utils.py cases) — the run was on a CPU-only box.

pre-commit run --files <the 6 changed files> → all hooks pass, exit 0 (ruff check, ruff format, typos, mypy-3.10, SPDX, forbidden imports, …).

Model evaluation

This changes what is rendered into the prompt for Kimi models, so it is output-affecting. I could not run a Kimi K2/K2.5 eval — the model needs an 8×B200-class host that I do not have. What is verified instead is deterministic and needs no GPU: the rendered prompt itself. Feeding the same two-turn function_call / function_call_output history through the real
moonshotai/Kimi-K2.7-Code/chat_template.jinja (the reporter's model):

--- BEFORE (main)
    <|tool_call_begin|>call_4144fac2c5e1f3e0
    ## Return of call_4144fac2c5e1f3e0
    <|tool_call_begin|>call_9bd0e1a7c2
    ## Return of call_9bd0e1a7c2
--- AFTER  (this PR)
    <|tool_call_begin|>functions.exec_command:0
    ## Return of functions.exec_command:0
    <|tool_call_begin|>functions.read_file:1
    ## Return of functions.read_file:1

The "after" form is exactly what the chat-completions path (fixed in #32768) already produces and what KimiK2Parser._TOOL_ID_RE accepts; the "before" form returns (None, None) from _extract_tool_id_and_name.

A reviewer with Kimi hardware should confirm the end-to-end fix by running the reporter's scenario — Codex CLI against /v1/responses, streaming, tool_choice: auto, a multi-turn task — and checking that the "no answer" rate drops. @GeorgiaM-honestly, if you can retest with this patch, that would settle it.

Duplicate check

Not a duplicate. As of 2026-08-02 there is no PR referencing #50768, no open PR touches vllm/entrypoints/openai/responses/utils.py or vllm/parser/kimi_k2.py, and none of the 30 open PRs matching "kimi tool call" addresses tool-call ID preservation on the Responses path. #32768 (merged) fixed the chat-completions half of this; the Responses path was missed.

AI assistance

AI assistance was used to investigate and draft this change. Every claim above was checked against the code and against the upstream chat template, and the test commands were run locally.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wsyjh8.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[RadomirMinkov]

There is an issue with the proposed implementaiton. _NATIVE_TOOL_CALL_ID_RE = re.compile(r".+:\d+") matches any string ending with :digit, not just the Kimi-native functions.{name}:{idx} format. An OpenAI-style ID like call_abc123:0 (unlikely but possible if a client reuses an ID from a prior response) would be incorrectly treated as native, skipping the rewrite to functions.{name}:{idx}. The chat template would then render call_abc123:0 verbatim as the <|tool_call_begin|> header, which the KimiK2Parser cannot parse, causing the tool call to be dropped. Consider tightening the regex to r"functions..+:\d+" to match only the actual native format.

[RadomirMinkov]

During my testing I found out that the proposed implementation won't fix the real issue. The problem in the current implementation is that in chat_utils.py in get_tool_call_id_type the check is only for the nested text-model type. This is an issue, because when using Kimi models, their text-model is based on the DeepSeek architecture, hence why the check returns that tool call id is not Kimi native. Due to these reasons the parsing of the tool calling is not based on the Kimi standard, but on DeepSeek's. To fix this the code should check not only the nested text-type model. The implementation can be something like this.
def get_tool_call_id_type(model_config: ModelConfig) -> str: """Return the tool-call ID type for a given model configuration.""" hf_overrides = getattr(model_config, "hf_overrides", None) model_types = { getattr(getattr(model_config, "hf_config", None), "model_type", None), getattr(getattr(model_config, "hf_text_config", None), "model_type", None), } if model_types.intersection(_KIMI_MODEL_TYPES) or ( isinstance(hf_overrides, dict) and hf_overrides.get("model_type") in _KIMI_MODEL_TYPES ): return "kimi_k2" return "random"

Check and test this before merging.