Skip to content

[Bugfix] glm47 parser: tolerate missing opening <arg_value> tag - #49249

Open
drakosha wants to merge 1 commit into
vllm-project:mainfrom
drakosha:fix-glm47-missing-arg-value-tag
Open

drakosha wants to merge 1 commit into
vllm-project:mainfrom
drakosha:fix-glm47-missing-arg-value-tag

Conversation

@drakosha

Copy link
Copy Markdown
Contributor

Purpose

FIX #49248

Under concurrent load, GLM-4.7-family models sporadically skip the opening <arg_value> tag when emitting tool calls, producing

<tool_call>search<arg_key>query</arg_key>how many vacation days left</arg_value></tool_call>

with the argument value fully intact (raw sampled tokens captured via logprobs: true; reproduces at temperature: 0 too, ~5–17% of calls at concurrency 6 on our production trace family, 0 sequentially).

_ARG_RE/_PARTIAL_ARG_RE in vllm/parser/glm47_moe.py require the opening tag, so such calls silently parse to arguments: "{}", which then fails the client's tool schema validation and degrades agent loops (the broken {} call gets rendered back into history, so retries tend to fail the same way).

This PR makes the opening <arg_value> optional in both regexes. The closing </arg_value> still delimits the value, so:

  • well-formed output parses exactly as before (the optional group matches the tag when present);
  • streaming is unaffected: the incremental lexer buffers ambiguous tag prefixes, so a partially streamed <arg_value> never leaks into a value snapshot.

Test Plan

  • tests/tool_parsers/test_glm47_moe_tool_parser.py: new non-streaming cases (single arg with missing opening tag; mixed malformed + well-formed args in one call) and a streaming case with the malformed sequence split across chunks; all pre-existing cases unchanged.
  • Live validation on a 4×GPU GLM-5.2 deployment (--tool-call-parser glm47), replaying the production request that triggers the malformation at concurrency 6 with logprobs: true:
    • before: 12/120 responses parsed to arguments: "{}";
    • after: 12/72 responses still show the raw malformation in sampled tokens (model-side quirk, expected), all 12 parse to full arguments, 0 empty; streaming replay 0/72 empty.

Test Result

pytest tests/tool_parsers/test_glm47_moe_tool_parser.py — 16 passed.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify

mergify Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @drakosha.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Sep 17, 2026
@drakosha
drakosha force-pushed the fix-glm47-missing-arg-value-tag branch from 389e86c to a0a57e2 Compare September 17, 2026 12:39
@mergify mergify Bot removed the needs-rebase label Sep 18, 2026
Under concurrent load, GLM-4.7-family models occasionally skip the
opening <arg_value> tag when emitting tool calls, producing
<arg_key>key</arg_key>value</arg_value> with the value fully intact.
The argument regexes required the opening tag, so such calls were
silently parsed as empty arguments ({}), failing downstream tool
validation. Make the opening tag optional in both the full and
partial argument regexes; the closing </arg_value> still delimits
the value. Streaming is unaffected for well-formed output: the
incremental lexer buffers ambiguous tag prefixes, so a partially
streamed <arg_value> tag never leaks into the value.

Signed-off-by: Mikhail Kostryukov <mike@triptrack.net>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working glm tool-calling

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[Bug]: glm47 tool parser silently returns empty arguments when the model omits the opening <arg_value> tag

1 participant