fix(parser): keep GLM tool delimiters inside argument values

[logprobz]

Problem

GLM-4.7 and GLM-5.3 tokenize a literal </tool_call>, </arg_value>, or <tool_call> inside an argument value to the same token ID as the structural tag. When the model writes those strings as data, such as while editing parser or template code, the parser can close the tool call at the first </tool_call> and leak the rest of the value into content.

Fix

Position is the only remaining signal, so the parser applies two rules:

• </tool_call> is structural only outside an argument value: before arguments, between arguments, or immediately after a candidate </arg_value>. The (TOOL_ARGS, TOOL_END) transition is removed from the GLM transition table.
• </arg_value> is structural only when the next non-whitespace text is <arg_key> or </tool_call>. A new TOOL_ARG_END_PENDING state holds that decision. Any other non-whitespace text returns the engine to TOOL_ARGS.

The second rule uses a small parser-engine hook, ParserEngineConfig.non_whitespace_transitions. Its default is empty, so existing parser configurations do not change. _emit_for_state applies the configured transition when plain content contains a non-whitespace character.

The argument converter uses _ARG_RE with a lookahead for the next <arg_key> or the end of the argument buffer, plus _TAIL_ARG_RE for the open final argument. In partial mode, a final </arg_value> is held until later text proves whether it is structural or data. This keeps streamed argument prefixes monotonic.

Compatibility and failure behavior

• The shared glm47_moe_config is also used by the GLM-4.5 aliases and Ling3. Their existing parser tests pass.
• non_whitespace_transitions is opt-in and default-inert for every other parser configuration.
• The exact sequences </arg_value><arg_key> and </arg_value></tool_call> are necessarily structural. The token stream contains no information that could distinguish those boundaries from identical literal data.
• If a malformed call omits its closing </arg_value>, a later </tool_call> is treated as argument data and following text remains in the argument until finish(). This is the deliberate tradeoff for preserving literal tool delimiters inside values.
• At final conversion, if no confirmed boundary follows, the last </arg_value> candidate is used as the malformed fallback and later bytes are discarded.

Related work and duplicate check

This is not a duplicate of vllm-project/vllm#54971. That PR prevents stray arg_key tags from corrupting argument keys; this PR disambiguates delimiters inside argument values. Both edit the GLM argument regexes, so a future upstream rebase must compose its _ARG_KEY prefix with this PR's lookahead patterns.

This PR replaces the GLM delimiter portion of #540. The companion scanner correction is #640. Other parser-wide experiments from #540 are intentionally not included.

Open-PR searches found no other upstream or fork PR implementing this argument-value boundary behavior.

Tests

Pytest validation used Python 3.12 in /opt/venv inside the existing rtx-station/glm53-jovian:lp15 image. Source and temporary test dependencies were mounted read-only. Lint validation used the repository-pinned Ruff version through uvx on the host.

/opt/venv/bin/python -m pytest tests/parser/engine/test_glm47_moe.py -q
/opt/venv/bin/python -m pytest tests/parser/engine -q
/opt/venv/bin/python -m pytest tests/tool_parsers/test_glm47_moe_tool_parser.py -q
/opt/venv/bin/python -m pytest tests/reasoning/test_glm4_moe_reasoning_parser.py -q
/opt/venv/bin/python -m pytest \
  tests/tool_parsers/test_glm4_moe_tool_parser.py \
  tests/parser/engine/test_ling3.py -q
uvx ruff@0.14.0 check \
  tests/parser/engine/test_glm47_moe.py \
  vllm/parser/engine/parser_engine_config.py \
  vllm/parser/engine/streaming_parser_engine.py \
  vllm/parser/glm47_moe.py
uvx ruff@0.14.0 format --check \
  tests/parser/engine/test_glm47_moe.py \
  vllm/parser/engine/parser_engine_config.py \
  vllm/parser/engine/streaming_parser_engine.py \
  vllm/parser/glm47_moe.py

Results:

• The original regression file is 6 failed and 1 passed on the test-only commit, then 7 passed with the fix. Two review-added malformed-input regressions also pass, for 9 tests total.
• tests/parser/engine: 3,800 passed.
• Existing GLM-4.7 tool-parser tests: 13 passed.
• Existing GLM reasoning tests: 16 passed.
• GLM-4.5 and Ling3 compatibility tests: 12 passed.
• Ruff lint, Ruff format, and git diff --check: passed.
• Both this PR and fix(parser): keep anchors when a stop token's text is stripped #640 combined: 3,806 parser-engine tests passed.

Model and serving validation

The cached GLM-5.3 tokenizer maps the relevant delimiters to IDs 154842, 154843, 154844, 154847, 154848, 154849, and 154850, with observation stop ID 154829. An independent token-by-token replay using the production incremental detokenizer passed with literal delimiters, Unicode data, and a stripped final observation token text.

No live model-generation or serving re-probe was run for this revision. The tests validate parser behavior for recorded and constructed token streams, not the model's likelihood of generating them.

Out of scope

• A model escaping its own reasoning by writing a literal </think>. That token is byte-identical to the structural one, so mitigation belongs in the prompt or a logits processor.
• The generic TokenIDScanner case where a trailing stop token ID has no text in the delta. That is handled by fix(parser): keep anchors when a stop token's text is stripped #640.

AI assistance and human review

Claude Code assisted with implementation and tests. OpenAI Codex performed an independent line-by-line diff review and validation. The human submitter reviewed and understands every changed line.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 127231c8-333d-4cee-9eae-536c79f8e100

📥 Commits

Reviewing files that changed from the base of the PR and between f668dd5 and 3839fb4.

📒 Files selected for processing (1)

• tests/parser/engine/test_glm47_moe.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The GLM-4.7 MoE parser preserves literal tool delimiters in argument values. The parser engine adds pending argument-value handling and non-whitespace transitions. Tests cover streaming, non-streaming, chunked, partial, whitespace, and speculative-decoding inputs.

Changes

GLM-4.7 MoE parser updates

Layer / File(s)	Summary
Parser engine pending-state support vllm/parser/engine/parser_engine_config.py, vllm/parser/engine/streaming_parser_engine.py	Adds TOOL_ARG_END_PENDING and configurable non-whitespace transitions. The streaming engine handles this state during emission, tool-state detection, and completion.
GLM-4.7 delimiter disambiguation vllm/parser/glm47_moe.py	Updates argument matching and conversion to preserve literal </arg_value> text. Adds TOOL_BETWEEN and TOOL_ARG_END_PENDING transitions and emits argument chunks across these states.
Delimiter parsing test coverage tests/parser/engine/test_glm47_moe.py	Adds fixtures and tests for converter, non-streaming, streaming, tag-split, character-split, partial, whitespace, and speculative-decoding inputs.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 3839f

GLM tool parsing now preserves literal delimiter text inside argument values while continuing to recognize structural delimiters. The added coverage includes the missing argument-terminator case, with no remaining merge-blocking risk identified.

Sequence Diagram(s)

sequenceDiagram
  participant Tokenizer
  participant StreamingParserEngine
  participant Glm47MoeParser
  participant ToolCallResult
  Tokenizer->>StreamingParserEngine: Deliver tag and text chunks
  StreamingParserEngine->>Glm47MoeParser: Apply parser state transitions
  Glm47MoeParser->>StreamingParserEngine: Preserve literal delimiter as argument content
  StreamingParserEngine->>ToolCallResult: Emit parsed tool call

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 11.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 4 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: preserving literal GLM tool delimiters inside argument values during parsing.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}