[Bugfix][Tool Parser] Fix Kimi-K2 streaming regex to handle leading newline before tool call ID

[saifmb0]

Summary

Fixes #38441.

The model (Kimi K2 / K2.5) occasionally emits a stray \n between <|tool_call_begin|> and the function name during streaming (observed on long-context inference with tool_choice: auto, without constrained decoding):

# What the model sometimes produces:
<|tool_call_begin|>
functions.edit:15<|tool_call_argument_begin|>{"path": "..."}

# Instead of:
<|tool_call_begin|>functions.edit:15<|tool_call_argument_begin|>{"path": "..."}

Because Python regex's . does not match \n by default, both stream_tool_call_portion_regex and stream_tool_call_name_regex silently failed to match, causing the tool call to be entirely dropped during streaming.

Root cause

# Before (broken on leading \n)
self.stream_tool_call_portion_regex = re.compile(
    r"(?P<tool_call_id>.+:\d+)\s*<\|tool_call_argument_begin\|>\s*(?P<function_arguments>.*)"
)
self.stream_tool_call_name_regex = re.compile(r"(?P<tool_call_id>.+:\d+)\s*")

Fix

# After
self.stream_tool_call_portion_regex = re.compile(
    r"\s*(?P<tool_call_id>.+:\d+)\s*"
    r"<\|tool_call_argument_begin\|>\s*(?P<function_arguments>.*)",
    re.DOTALL,
)
self.stream_tool_call_name_regex = re.compile(
    r"\s*(?P<tool_call_id>.+:\d+)\s*", re.DOTALL
)

Two changes per regex:

1. Leading \s* — consumes any leading whitespace/newlines before the function name.
2. re.DOTALL — makes . match \n so the tool_call_id capture group spans newlines.

Why this is not a duplicate

Checked open PRs: #37384, #37445, #32504, #24847, #26918, #36891.

• PR [Bugfix][Tool Parser] Fix Kimi-K2.5 parser accuracy, buffer limits, and token leaks #37384 adds re.DOTALL only to stream_tool_call_portion_regex (for multi-line arguments), but does not add the leading \s* that handles a newline before the tool_call_id, and does not fix stream_tool_call_name_regex at all. The \s* prefix is the critical fix for this issue.
• No other open PR addresses stream_tool_call_name_regex.

Test plan

Two new tests added to tests/tool_parsers/test_kimi_k2_tool_parser.py:

• test_stream_tool_call_portion_regex_handles_leading_newline — unit test: both regexes must match with/without a leading \n, and correctly extract tool_call_id and function_arguments.
• test_streaming_tool_call_with_newline_after_begin_token — end-to-end streaming simulation of the exact failure scenario from the issue; asserts at least one tool-call delta is emitted (i.e., the tool call is not silently dropped).

Existing tests: pre-commit run ruff-format and pre-commit run ruff-check pass clean.

AI assistance disclosure

This fix was investigated, reproduced, and implemented with AI assistance (GitHub Copilot / Claude Sonnet 4.6). Every changed line has been reviewed. The reviewer (submitter) understands the change end-to-end.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request fixes an issue where the Kimi K2 tool parser fails to process tool calls when a leading newline is emitted by the model. The fix involves updating the regex patterns for tool call portions and names to handle leading whitespace and using the re.DOTALL flag. Comprehensive regression tests have been added to verify these changes. Review feedback suggests further refining the regex patterns to use \S.*:\d+ instead of .+:\d+ to avoid capturing surrounding whitespace within the tool_call_id group, which would simplify the code by removing the need for manual .strip() calls.

[saifmb0]

Applied all four suggestions:

• Both streaming regexes now use \S.*:\d+ instead of .+:\d+ — the capture starts at the first non-whitespace character, so trailing whitespace before the :\d+ is excluded from the group.
• Removed both .strip() calls in the parser logic (lines 456 and 466).
• Removed both .strip() calls in the new tests.

Amended and force-pushed to the same branch.

[gemini-code-assist]

Thanks for the update, @saifmb0. The changes look correct and address the identified issues. I've verified that the regex updates and the removal of .strip() calls are consistent with the requirements. The added tests also correctly cover the edge cases for leading newlines.