Skip to content

Recover GLM-4.7 tool calls missing arg_key start - #47190

Draft
Tib-Gridello wants to merge 1 commit into
vllm-project:mainfrom
Tib-Gridello:codex/glm47-recover-malformed-arg-tags
Draft

Tib-Gridello wants to merge 1 commit into
vllm-project:mainfrom
Tib-Gridello:codex/glm47-recover-malformed-arg-tags

Conversation

@Tib-Gridello

@Tib-Gridello Tib-Gridello commented Jun 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • recover GLM-4.7 XML tool calls where the model emits tool_namearg</arg_key> instead of tool_name<arg_key>arg</arg_key>
  • keep recovery limited to prefixes that match a requested tool name
  • recover before streaming name emission, so malformed names are not sent as function names mid-stream
  • add regression coverage for adjacent, spaced, multi-argument, and streaming malformed calls

Context

This is separate from #47146. That PR enables required/named tool-choice when strict tool calling is disabled; it does not repair malformed structural tag output where the opening <arg_key> token is missing.

Observed malformed examples from a GLM-4.7-compatible serving setup include:

  • <tool_call>send_command_to_shellcommand</arg_key><arg_value>echo test</arg_value></tool_call>
  • <tool_call>grep_toolpattern</arg_key><arg_value>...</arg_value></tool_call>
  • <tool_call>browser_action task</arg_key><arg_value>...</arg_value></tool_call>

Verification

  • python3 -m py_compile vllm/parser/glm47_moe.py tests/tool_parsers/test_glm47_moe_tool_parser.py

Could not run the targeted pytest suite locally because the repository uv environment does not resolve on this macOS environment due the pinned Torch split, and pytest is not installed in the no-sync environment.

@mergify mergify Bot added the tool-calling label Jun 30, 2026
@Tib-Gridello
Tib-Gridello force-pushed the codex/glm47-recover-malformed-arg-tags branch from 76a6347 to aa428e6 Compare June 30, 2026 17:56
Signed-off-by: tibG <tib.gridello@gmail.com>
@Tib-Gridello
Tib-Gridello force-pushed the codex/glm47-recover-malformed-arg-tags branch from aa428e6 to 135c874 Compare July 1, 2026 02:12
@Tib-Gridello

Copy link
Copy Markdown
Contributor Author

Box3 validation note: the deployed vLLM image still uses the older vllm.tool_parsers.glm47_moe_tool_parser implementation, and live traffic still logs parser warnings for malformed calls shaped like grep_tool multiline</arg_key><arg_value>true</arg_value><arg_key>output_mode...</arg_key>.... The agent-side middleware can repair these before execution, but the root parser issue remains in that old path.

This PR's parser-engine implementation avoids the old early-return behavior and recovers by splitting the longest valid tool name from the accumulated name+args text. That is the behavior we need upstream so the API does not emit invalid tool names before application-layer repair.

@mergify

mergify Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Tib-Gridello.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Sep 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant