[Parser] Flush Kimi K3 partial marker text at stream EOF

[whx-sjtu]

Overview

Kimi K3 streaming can permanently hold text that looks like the start of an XTML marker when generation ends. Flush the literal unfinished suffix to its reasoning/content channel at EOF, so value < is not truncated to value .

Rebased onto main bfa7e5b7afe40ebe48288801a2baf5c1135cd881. This remains an independent child of closed #58968.

Claims

stream ends with an unfinished marker candidate
- buffered suffix is lost
+ emit the generated suffix in its current channel

Keep complete framing/tool boundaries and reasoning suppression intact. This parser fix also applies to single-instance serving.

Validation

Validated on 0de4078b842913dafa869d221dfeb2414c8c5e67 (2026-10-12): 142 passed. Changed-file pre-commit passed, including mypy for Python 3.11 and test/CI tethering checks.

VLLM_TARGET_DEVICE=cpu .venv/bin/python -m pytest -q -ra \
  tests/reasoning/test_kimi_k3_reasoning_parser.py tests/tool_use/test_kimi_k3_tool_parser.py tests/tool_parsers/test_kimi_k3_named_tool_choice.py

Real parser/protocol tests cover empty final deltas, split markers, reasoning/tool combinations, complete control markers, include_reasoning=False and mixed DeepSeek reasoning/K3 tool parsing. Original October 8 production-file negative controls (194da61de1) gave 56 failing EOF cases / 86 passing controls; all 142 pass on this head.

The defect was reproduced while investigating AIPerf empty-content errors, but the original generated-token evidence is unavailable, so those errors are not attributed wholesale to it. Exact-source model/HTTP-streaming validation remains pending.

The eight independent PRs compose on this same main. The combined CPU suite passes 344 tests, with 33 skips (32 ROCm/aiter-gated connector cases and one GPU LLM case) and one model-loading deselection. Counts overlap with the independent suites.

Details

Use the existing finalization callback and generated text. Reasoning tails stay in reasoning; tool-content flush runs only after reasoning ends. No generic parser interface or generation behavior changes.

Duplicate checks: #50229 broadly migrates K3 to Parser Engine and overlaps these files; this is the smaller fix for the existing implementation, and its regressions should survive that migration. #51400 handles complete message-close markers; #57366/#50291 handle non-streaming truncation; #60565 handles response-only transitions. They do not implement this legacy EOF flush. #58968 was superseded by the split PRs.

Merge danger: two-way door; blast radius: K3 streaming text extraction. Normal serving finalizes a choice once; this change does not extend the contract to repeated finalization.

#60655 addresses quoted think markers in non-streaming output. The broader Parser Engine migration in #50229 still overlaps these files; preserve the EOF behavior when composing it.

AI assistance was used for implementation, rebase, regression checks and this description. Human line-by-line review and the runtime acceptance items above remain pending. No performance improvement is claimed.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).
• AI assistance was used during the creation of this PR.
• Design Fit: Source/compatibility review completed; human line-by-line review remains pending.
• Testing and Validation: Rebased CPU regressions passed; the runtime acceptance gaps above remain open.
• Code Quality and Style: Changed-file pre-commit, mypy and test/CI tethering checks passed.
• Pull Request Contents: Current source, evidence, related PRs and remaining limitations documented.