Skip to content

feat: add Gemma 4 tool call parser#269

Merged
Thump604 merged 8 commits intowaybarrios:mainfrom
jackneil:pr/gemma4-tool-parser
Apr 10, 2026
Merged

feat: add Gemma 4 tool call parser#269
Thump604 merged 8 commits intowaybarrios:mainfrom
jackneil:pr/gemma4-tool-parser

Conversation

@jackneil
Copy link
Copy Markdown
Contributor

@jackneil jackneil commented Apr 9, 2026

Summary

  • Adds gemma4_tool_parser.py — parses Gemma 4's <|tool_call>call:name{<|"|>key<|"|>:val}<tool_call|> format into OpenAI-compatible tool_calls
  • Handles nested objects, arrays, unicode, braces in strings, quoted keys, multiple calls per block, think tags
  • Registers as --tool-call-parser gemma4
  • Adds streaming filter tags for <|tool_call>/<tool_call|>
  • 30 tests including OpenAI format integration tests verifying exact Claude Code compatibility

Usage

vllm-mlx serve google/gemma-4-27b-it \
  --enable-auto-tool-choice --tool-call-parser gemma4

Test plan

  • 30 unit/integration tests passing
  • OpenAI format verified (content=null, finish_reason=tool_calls, function.arguments as JSON string)
  • End-to-end with Claude Code

🤖 Generated with Claude Code

Jack Neil and others added 6 commits April 10, 2026 18:07
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Unclosed tool call block (server fallback path)
- String containing colon (step-ordering guard)
- String with real newline and double quote (JSON escaping)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…e Code

Integration tests that verify the full pipeline (parser → server models →
JSON serialization) matches what Claude Code expects: tool_calls structure,
null content, function.arguments as JSON string, correct finish_reason.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
integrates Gemma 4 format as the first format tried in auto-detection,
adds streaming markers for tool call start/end. based on keegoid's
approach in waybarrios#254.
@waybarrios waybarrios force-pushed the pr/gemma4-tool-parser branch from 2d0c15d to 95865be Compare April 10, 2026 22:09
@waybarrios
Copy link
Copy Markdown
Owner

waybarrios commented Apr 10, 2026

@jackneil I rebased this on main (clean, no conflicts) and added one commit on top: Gemma 4 auto-detection in AutoToolParser, based on keegoid's approach from #254. This way Gemma 4 tool calls work out of the box with --enable-auto-tool-choice without needing --tool-call-parser gemma4.

What I added:

  • Gemma 4 as the first format tried in AutoToolParser.extract_tool_calls()
  • <|tool_call> and <tool_call|> in the streaming markers/end_markers
  • Imports Gemma4ToolParser and delegates to it when the marker is detected

All 121 tool parser tests pass, black and ruff clean.

Copy link
Copy Markdown
Owner

@waybarrios waybarrios left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebased, added auto-detection from #254, 121 tests pass. Ready to merge.

@waybarrios waybarrios requested a review from Thump604 April 10, 2026 22:16
@waybarrios
Copy link
Copy Markdown
Owner

@Thump604 would you mind taking a look at this one? It's the Gemma 4 tool call parser from Jack, plus I added auto-detection in AutoToolParser based on keegoid's approach from #254. This is the last piece to complete the Gemma 4 support alongside #268 and #256 that already landed.

@Thump604 Thump604 merged commit c50d7db into waybarrios:main Apr 10, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants