[Frontend] glm47 tool parser: required-first schema order in prompt rendering and strict structural tags

[JaredforReal]

Purpose

Fix a tool-calling failure mode of the glm47 parser family (GLM-4.5/4.6/4.7/5.x, incl. GLM-5.3-Flash): the model omits a required property of a nested object when a bulky optional property precedes it in the schema.

A request (a collect_preferences tool whose items[] objects require prompt + tag, with an optional choices array declared between them) fails 50-66% of the time on vLLM with GLM-5.3-Flash: the emitted items[] objects are missing tag. The same failure reproduces on sglang at the same rate (46-60%), so this is a model-behavior boundary issue, not a stack bug.

Mechanism (measured with prefix logprob probes on /v1/completions):

• GLM renders tool schemas verbatim into the prompt and tends to emit argument keys following the schema's property order.
• The first key of each item object is a near-tie between the content fields (p(choices)=0.62 vs p(prompt)=0.38, p(tag)~=0 — the model never opens an object with the label field). Once it starts with the optional choices array, the object "feels complete" after prompt and tag is skipped (p(tag) after choices ≈ 0.0002; after prompt ≈ 0.9998). At temp=0 the outcome is still non-deterministic — kernel noise flips the near-tie.
• The coupling is stack-independent (vLLM: prompt-first → 65/65 keep tag, choices-first → 78% lose it; sglang: 11/11 vs 77%).

Fix (two commits on this branch):

1. reorder_tool_schema_required_first: render each object schema's properties with required fields first (declaration order), recursing into nested object/array schemas. JSON Schema property order is not semantically meaningful, so validation and clients are unaffected. Opt-in per tool parser; enabled for the glm47 parser family. The reorder runs in OnlineRenderer right after the per-request tool dicts are built — the parser's adjust_request hook runs after chat-template rendering, which is too late to affect the prompt.
2. Strict-mode alignment: the structural tag for guided tool calling is now built from the same reordered schema. xgrammar's builtin glm_4_7 tag pins object keys to the schema's declared order (any_order=False), so previously the prompt and the grammar disagreed: after the model committed to a required key that the prompt placed early, every optional key declared before it became unreachable and was silently dropped (and we observed a ~1-2% runaway-to-max_tokens pathology under concurrency, consistent with the grammar fighting the model's key-order prior). any_order=True is not an alternative: per GrammarCompiler.compile_json_schema it only bounds the number of entries and does not check key presence or uniqueness.

Not a duplicate: the only in-flight GLM-5.3-Flash PRs we found (#55219 KV layout, #55358 indexer move) are orthogonal; no open PR touches tool-schema rendering order or the glm47 structural tag.

Test Plan

Unit:

pytest tests/tool_parsers/test_utils.py -k ReorderPropertiesRequiredFirst   # 5 passed
pytest tests/tool_parsers/test_glm47_moe_tool_parser.py                     # incl. new -k StructuralTag (9 passed)

E2E on 4x GB300, zai-org/GLM-5.3-Flash (native FP8, TP4, --tool-call-parser glm47 --reasoning-parser glm45), the request verbatim, serial non-streaming runs:

Test Result

Schema-compliance failure rate (items[] missing the required tag), before → after:

configuration	baseline	with this PR
GLM-5.3-Flash, original client payload	64.5% (129/200; earlier arms 50%/66% at n=30/100)	14.0% (28/200)
GLM-5.3-Flash, same reorder applied client-side	—	3% (1/30)
GLM-5.3-Flash, this PR + strict: true	—	0% (0/144)
GLM-5.3-NVFP4 (GlmMoeDsa, same parser/template)	12% (6/50)	0% (0/50)
sglang (xinyuan/glm-5.3-flash-support), same payload, for reference	46.5% (93/200)	—

Residual failures all start the item object with the optional choices key first (the J1 coin flip can still land wrong; only guided decoding or a model-side fix eliminates it).

Strict mode eliminates the residual failures entirely: "strict": true on the tool definition with tool_choice: "auto" (free text and reasoning stay unconstrained; once the model starts a <tool_call>, its arguments are grammar-enforced to match the schema). No server-side flag needed beyond the parser flags above.

Performance

Prompt-side reorder cost (reorder_properties_required_first + the per-request model_dump copy it operates on): ~48 µs per request for the original schema (9 µs for the reorder walk alone; the deepcopy dominates and model_dump was already happening). Negligible against ~1s request latencies.

Strict-mode guided decoding overhead, measured on the same 4x GB300 GLM-5.3-Flash setup with the request (~180 completion tokens, almost all inside the grammar-constrained <arg_value> body — a near-worst-case constrained fraction):

concurrency (n)	baseline	strict (this PR)
1 (32)	1.21s p50, 143.9 tok/s/req	1.09s p50, 143.9 tok/s/req
16 (48)	2.89s p50, 6.45s p99	2.46s p50, 7.66s p99
32 (64)	3.74s p50, 7.78s p99	3.51s p50, 7.75s p99
64 (128)	4.90s p50, 1677 tok/s agg, 9.81 req/s	4.84s p50, 1768 tok/s agg, 9.94 req/s

Strict is never slower (slightly faster in aggregate: constrained outputs are shorter and more regular). TTFT is unchanged once the grammar is compiled; the first request with a new schema pays the ~0.5s compile (cached per schema afterwards).

Per-token grammar cost (isolated microbenchmark of the xgrammar matcher on this schema): fill_next_token_bitmask 82.5 µs/tok + accept_token 1.0 µs/tok ≈ 83.5 µs per constrained token, i.e. ~8.4% of one CPU core per 1000 constrained tok/s of aggregate traffic. The glm_4_7 structural tag is trigger-gated: only tokens inside <tool_call> blocks pay this; reasoning and free text are unconstrained.

Behavioral scope note: the reorder changes the rendered prompt for every model served with --tool-call-parser glm47/glm45 (GLM-4.5/4.6/4.7/5.x). On the GLM-5.3 family it strictly helps (table above); other glm47-family models share the same emission-order prior.

docs/features/tool_calling.md updated for the GLM-5.x entry.

This change was implemented with AI assistance and reviewed by JaredforReal.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mergify]

Documentation preview: https://vllm--55558.org.readthedocs.build/en/55558/

[gaby]

@JaredforReal Is this bug the reason for:

• https://huggingface.co/zai-org/GLM-5.3-Flash/discussions/23
• https://huggingface.co/zai-org/GLM-5.3-Flash/discussions/47

There is a similar report for sglang here: sgl-project/sglang#36669

[JaredforReal]

thanks gaby, and no, this is for tool call related argument tag loss @gaby