Skip to content

[Bugfix] Fix GLM zero-arg streaming tool names#41654

Open
joninco wants to merge 2 commits into
vllm-project:mainfrom
joninco:fix/glm47-zero-arg-streaming-tools
Open

[Bugfix] Fix GLM zero-arg streaming tool names#41654
joninco wants to merge 2 commits into
vllm-project:mainfrom
joninco:fix/glm47-zero-arg-streaming-tools

Conversation

@joninco
Copy link
Copy Markdown
Contributor

@joninco joninco commented May 4, 2026

Summary

  • Treat a completed GLM tool-call region with no newline or <arg_key> as a zero-argument tool call name.
  • Strengthen GLM-4.7 streaming tests to assert the emitted function name and {} argument fragments.
  • Adds coverage for an MCP-style tool name: mcp__gmail-cleanup__list_filters.

Why this is not duplicating an existing PR

I checked open PRs for the related areas before submitting. PR #39615 fixes the generic serving.py finish-flush path that can blank arguments when all args arrive in one delta, but it does not change the GLM parser. This PR is narrower: it fixes GLM parser name extraction for completed zero-argument streaming tool calls.

Related but different open PRs found:

Human review / AI assistance

I reviewed the 5 logical code lines modified in the parser. AI assistance was used to prepare the branch and tests.

Tests

  • pre-commit run --files vllm/tool_parsers/glm4_moe_tool_parser.py tests/tool_parsers/test_glm47_moe_tool_parser.py
  • .venv/bin/ruff check vllm/tool_parsers/glm4_moe_tool_parser.py tests/tool_parsers/test_glm47_moe_tool_parser.py
  • .venv/bin/ruff format --check vllm/tool_parsers/glm4_moe_tool_parser.py tests/tool_parsers/test_glm47_moe_tool_parser.py
  • .venv/bin/python -m py_compile vllm/tool_parsers/glm4_moe_tool_parser.py tests/tool_parsers/test_glm47_moe_tool_parser.py
  • Targeted behavior script verified that get_current_date and mcp__gmail-cleanup__list_filters stream their names and {} args.

Full pytest collection did not run in my local checkout because the existing local vllm/_C.abi3.so extension fails to import with a CUDA symbol mismatch before this test file loads.

Treat a completed GLM tool-call region with no newline or arg tags as a zero-argument tool call name. This lets streaming emit the function name and {} arguments for calls such as mcp__gmail-cleanup__list_filters.

Strengthen the GLM-4.7 streaming regression tests to assert the emitted name and argument fragments instead of only checking parser state allocation.

Assisted-by: OpenAI Codex
Signed-off-by: jon <joninco@bullpoint.org>
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added tool-calling bug Something isn't working labels May 4, 2026
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented May 4, 2026

Hi @joninco, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the GLM4 tool parser to support zero-argument tool calls by allowing the tool name to be extracted from a complete tool-call region even when delimiters are absent. It introduces an 'is_complete' parameter to the name extraction logic and refactors the associated tests to use helper functions for streaming chunks, including a new test case for MCP-style tool names. I have no feedback to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working tool-calling

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant