Studio: heal DiffusionGemma tool calls into structured tool_calls - #6851
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new property supports_tool_passthrough to the llama_cpp backend, which returns the underlying _supports_tools value. This property is then used in inference.py to determine tool passthrough and client tool support, falling back to supports_tools if the property is not present. This change ensures that client tool loops can bypass restrictions placed on supports_tools for specific models like DiffusionGemma. There are no review comments to address, and the changes look correct.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
@codex review |
|
Pushed a small follow-up so the OpenAI The passthrough branch was gated on It now gates on whether the server-side loop can actually claim the request ( |
for more information, see https://pre-commit.ci
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
1 similar comment
|
@codex review |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
3 participants
DiffusionGemma tool calls come back as raw text instead of a structured
tool_callsarray. A client that declares a tool gets<|tool_call>call:get_weather{...}<tool_call|>in the message content withfinish_reason: stop, so OpenAI-v1 clients like pi can't see the call.Closes #6732
(Thinking already works; only tool calls were affected.)
Problem
supports_toolsis off for diffusion, which keeps the agentic tool loop from running (it would drop the canvas frames that drive the visualization). But that same flag also switched off the client-tool passthrough, the path that turns the model's text tool call into a structuredtool_calls.So the tool call reached the client as raw text. The parser already understands DiffusionGemma's
<|tool_call>format (from #6801); it just wasn't being reached.Fix
Split the flag in two:
supports_toolsstill gates the agentic loop (stays off for diffusion), and a newsupports_tool_passthroughgates only the client-tool passthrough (on for diffusion).Non-diffusion models are unaffected, since both flags return the same value there.
Verification
Live on
unsloth/diffusiongemma-26B-A4B-it-GGUFwith a declared tool: OpenAI (streaming and non-streaming) and Anthropic/v1/messagesnow return propertool_calls/tool_use, and the diffusion visualization still streams. The existing passthrough and route suites pass unchanged.Note
This is the Studio half. The model only emits schema-correct arguments once the shim (unslothai/unsloth-zoo#864) and the visual server (llama.cpp #24423) forward the tool definitions too, so it's a no-op until a client sends tools.