[Parser] Migrate Hermes tool parser to the new streaming Parser Engine

[muhammadfawaz1]

[Parser] Migrate Hermes tool parser to the streaming Parser Engine

Summary

Migrates the Hermes tool parser from the legacy regex/diff implementation (~275 lines) to the declarative streaming Parser Engine, following the pattern already used by Qwen3, MiniMax M2, Gemma4, Kimi K2 and Inkling.

• New vllm/parser/hermes.py: HermesParser(ParserEngine) + a declarative ParserEngineConfig.
• vllm/tool_parsers/hermes_tool_parser.py becomes a thin bridge. The registry entry and the Hermes2ProToolParser class name are unchanged, so --tool-call-parser hermes is unaffected.
• structural_tag_model = "hermes" is preserved, keeping guided decoding (and the supports_required_and_named=False it implies) identical.

This follows the suggestion on #49535 to migrate tool parsers to the Parser Engine rather than patch the legacy implementations further.

Why Hermes first, and not hunyuan_a13b / xlam

Those two parsers emit a single undelimited JSON array ([{...},{...}]), where the boundary between one tool call and the next is a bare comma at the array's top nesting level — there is no terminal token. The engine's state machine only transitions on explicit terminal tokens; brace/bracket depth tracking in the engine exists only to decide which characters are safe to stream, not to drive a transition. Migrating those two needs new engine-level capability, not just a per-model config, so it's a separate design question rather than something to fold into this PR.

Hermes uses the same per-call delimiter shape as the already-migrated models (<tool_call>…</tool_call> repeated per call), which is proven engine capability, so it's independently actionable now.

Root cause / design notes

Why an arg_converter is mandatory. Hermes's tool-call body is a {"name":..., "arguments":{...}} wrapper, so the whole wrapper lands in TOOL_ARGS. ParserEngine._compute_arg_delta returns the raw delta and never calls _fix_arg_types when arg_converter is None, while non-streaming extraction always coerces via _build_extracted_result. Without a converter, streaming and non-streaming silently disagree on argument types.

Why a verbatim scanner rather than re-serialising. The engine diffs successive converter outputs and requires each to extend the previous one (startswith); a violation silently drops argument deltas. Re-serialising through json.dumps changes whitespace and closes unterminated structures differently across ticks, so the only prefix-stable output is a verbatim substring of the input. _hermes_arg_converter therefore carves the arguments value out with a string/escape-aware scanner, mirroring _inkling_arg_converter, which has the same wrapper shape. The scanner is also what makes an argument literally named "name" safe — the bug class that started this work (#49535).

Why token_id_terminals is empty. StreamingParserEngine._process_lex_tokens enters a strict token-id mode once any real token IDs have been seen, and refuses to accept a text-matched terminal that is listed in token_id_terminals unless it was also pre-lexed from an actual matching token ID. Hermes delimiters reliably arrive as literal text (ParserEngine.adjust_request forces skip_special_tokens=False), so token-ID pre-lexing buys nothing and would starve text matching for tokenizers where the delimiter is not a single true token. Inkling opts out the same way.

Non-streaming strictness is preserved explicitly. The legacy parser required the whole tool-call body to parse as valid JSON and reported no tool calls at all otherwise. The engine's generic EOS path is best-effort and would happily emit a tool call from a truncated wrapper, so HermesParser.extract_tool_calls_from_content keeps the single-shot json.loads check. Streaming keeps the engine's normal best-effort behavior.

Longcat. LongcatFlashToolParser subclassed Hermes2ProToolParser and customised the delimiters by assigning tool_call_start_token / tool_call_end_token / tool_call_regex, none of which the engine consults. Left as-is, the migration would have silently broken it (13 of its 17 tests). It now has its own HermesParser subclass overriding the delimiters as class attributes, which is why TOOL_CALL_START/TOOL_CALL_END are class attributes on HermesParser at all.

Test-suite change, and why

tests/parser/test_streaming.py used Hermes as its reference non-engine tool parser (a section titled "Engine-based reasoning + non-engine tool parser", asserting engine_based_streaming is False). This migration invalidates that premise by design.

Those tests now use Granite4ToolParser, which is still non-engine, uses the identical <tool_call>/</tool_call> delimiters so MODEL_OUTPUT is unchanged, and has supports_required_and_named=True natively — which also removes the need for the autouse fixture that was force-setting that flag on Hermes. test_parse_delta_finished_appends_remaining_args still uses Hermes explicitly, since its subject is that parser's unstreamed-args behavior. No test was weakened or deleted; only the reference model changed.

Out of scope (pre-existing, not introduced here)

While validating the above I found that tool_choice="required" crashes for any engine-based parser whose supports_required_and_named is True: StreamState.advance() passes only the latest delta for engine-based parsers, but extract_required_tool_call_streaming() expects the full accumulated text, so the final chunk reaches partial_json_parser as a bare ] and raises IndexError: pop from empty list.

This reproduces on the already-merged KimiK2ToolParser with no Hermes involvement, so it is not caused by this PR. It is normally unreachable because every engine-based parser sets supports_required_and_named=False via structural_tag_model + VLLM_ENFORCE_STRICT_TOOL_CALLING (default on), which Hermes continues to do here. Happy to file it separately.

Not a duplicate

• gh pr list --repo vllm-project/vllm --search "hermes parser engine" --state all and --search "migrate parser engine in:title is:open" — no PR migrates Hermes. In-flight migrations cover other models (Kimi K3 [Parser] Migrate Kimi K3 to Parser Engine #50229, llama3_json/llama4_json [Frontend] [Parser] Migrate llama3_json/llama4_json to the streaming parser engine #51577, Granite [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine #49648, Olmo3 [Bugfix] Migrate Olmo3 reasoning parser to the streaming parser engine #48160, poolside_v1 [Tool Parser] poolside_v1: migrate to the declarative parser engine #47167).
• vllm/parser/hermes.py does not exist on main; the registry still points at the legacy hermes_tool_parser / Hermes2ProToolParser.
• Three open PRs touch vllm/tool_parsers/hermes_tool_parser.py (fix: reject duplicate JSON keys in Hermes tool parser for streaming consistency #48864, [Bugfix][ToolParser] Make Hermes streaming tool call boundaries JSON string aware #45310, [Bugfix] Hermes tool parser: parse tool call JSON by object boundary, not literal </tool_call> #48353). This PR replaces that file wholesale, so it will conflict with them textually. Flagging explicitly so maintainers can sequence them — [Bugfix] Hermes tool parser: parse tool call JSON by object boundary, not literal </tool_call> #48353 in particular changes tool-call boundary semantics, and if it is preferred first I am happy to rebase on top of it.
• gh pr list --search "longcat" — no open PR touches vllm/tool_parsers/longcat_tool_parser.py ([Rust Frontend] Add LongCat tool parser #45773 is the Rust frontend).

Test plan

Environment note: run offline against a torch-stubbed environment, so the CUDA/engine-dependent top-level tests/conftest.py is skipped via --confcutdir=tests/tool_parsers.

pytest tests/tool_parsers/test_hermes_tool_parser.py \
       tests/tool_parsers/test_longcat_tool_parser.py \
       tests/parser/test_parse.py \
       tests/parser/test_streaming.py \
       tests/parser/test_include_reasoning.py \
       -q --confcutdir=tests/tool_parsers

Result: 105 passed, 1 skipped, 1 xfailed (the skip and xfail are pre-existing Granite4-specific cases, unchanged by this PR).

Per-suite, before vs after this change:

Suite	Before	After
tests/tool_parsers/test_hermes_tool_parser.py	30 passed, 1 skipped	34 passed, 1 skipped
tests/tool_parsers/test_longcat_tool_parser.py	16 passed, 1 xfailed	16 passed, 1 xfailed
tests/parser/test_parse.py, test_streaming.py, test_include_reasoning.py	55 passed	55 passed

The 4 added Hermes tests are one new case (test_hermes_streaming_tool_call_with_name_key_argument) across the existing stream_interval parametrisation (2/3/5/8), covering an argument literally named "name" — the bug class from #49535 — at multiple chunk granularities.

Lint: pre-commit hooks ran on commit — ruff check, ruff format, and mypy for Python 3.10 all passed.

End-to-end, against a real server and model

Run separately on a T4 GPU, since the suite above is offline-only:

pytest tests/entrypoints/tool_parsers/test_hermes_tool_parser.py -k llama -v -s

Result: 4 passed, 4 deselected (the deselected cases are the granite4 config). This boots a real vllm serve with --tool-call-parser hermes against meta-llama/Llama-3.2-1B-Instruct plus the minpeter/LoRA-Llama-3.2-1B-tool-vllm-ci adapter, and drives it through the openai client — covering non-streaming, streaming, and parallel tool calls.

The streaming case confirms argument type coercion survives the migration end to end, reconstructing {'inserted': True, 'product_id': 7355608} — a real boolean and a real integer rather than strings, which is the divergence the arg_converter exists to prevent.

Longcat has no e2e suite upstream, so its coverage here is the 17 unit tests only — worth noting since it was changed as a consequence of the Hermes migration rather than being its target.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[chaunceyjiang]

/ci run

[github-actions]

✅ @muhammadfawaz1, CI is now available for this PR.

• /ci run starts a CI build.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /ci cancel cancels scheduled or running CI builds for this PR branch.

[github-actions]

✅ Triggered Buildkite CI #83699 for commit 52400838de34.

[github-actions]

✅ Triggered Buildkite CI #83721 for commit 665ed0d85f05.

[muhammadfawaz1]

@chaunceyjiang all the CI tests have passed, would appreciate a review on this when you get a chance so we can get it merged and keep moving.

One thing we ran into while verifying this: tool_choice="required" crashes for engine-based parsers (documented in the PR's "Out of scope" section) — reproduces on the already-merged Kimi K2 parser, so it's unrelated to Hermes. Should we file a PR for that now, or hold off and keep going on migrations first?

[chaunceyjiang]

/cc @bbrowning @sfeng33 PTAL.

[muhammadfawaz1]

@bbrowning @sfeng33 following up here, CI's been green since the last push and this is tagged ready. Would appreciate a review when you get a chance so I can keep moving on the rest of the parser-engine migration series. Happy to address anything that comes up.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @professorsab.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork