fix(mcp): preserve explicit tool_choice for the initial responses request - #1963
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
github-actions
Bot
added
model-gateway
📝 WalkthroughWalkthroughMCP tool-loop payload construction now preserves and remaps explicit initial ChangesTool choice semantics
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@model_gateway/src/routers/openai/mcp/tool_loop.rs`:
- Around line 1944-1994: Add tests alongside
prepare_preserves_explicit_tool_choice and
prepare_defaults_missing_tool_choice_to_auto for tool_choice set to null and for
a structured specific-function choice. Verify prepare_mcp_tools_as_functions
changes null to "auto" while preserving the structured function selection
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 12da5b5b-c4db-448f-8ac5-e3da8a71236d
📒 Files selected for processing (1)
model_gateway/src/routers/openai/mcp/tool_loop.rs
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5ce36c43b8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if obj.get("tool_choice").is_none_or(Value::is_null) { | ||
| obj.insert("tool_choice".to_string(), Value::String("auto".to_string())); |
There was a problem hiding this comment.
When a Responses request uses MCP routing with a hosted/MCP-specific tool_choice object (for example {"type":"web_search_preview"} or {"type":"mcp","server_label":"wiki","name":"ask_wiki"}), this guard preserves that original object even though the preceding rewrite has replaced the tools array with synthesized function tools. The initial upstream /v1/responses request now carries a tool_choice whose type is no longer present in tools, so the model worker can reject it or fail to force the intended MCP/builtin call before the gateway gets a chance to dispatch it. Please normalize these non-function choices to the corresponding synthesized function choice (or fall back to auto) instead of preserving every non-null value.
Useful? React with 👍 / 👎.
…uest The OpenAI-router MCP tool loop rewrote every tool_choice to "auto" before the first model request, so "required"/"none"/specific choices were silently dropped. The initial request now keeps the client's explicit choice; resume payloads still force "auto" so a preserved "required" cannot loop until the tool-call limit. Matches the gRPC responses loop's existing per-iteration semantics. Closes #1945 Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
5ce36c4 to
e7de196
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7de196386
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| Some(name) | ||
| if tools | ||
| .iter() | ||
| .any(|tool| utils::function_tool_name(tool) == Some(name)) => |
There was a problem hiding this comment.
When a request mixes a caller-owned function tool with any MCP-routed tool and sets tool_choice to that function, this check now preserves the forced function choice because the retained user function is present in tools_json. The non-streaming tool loop later extracts every function_call and executes it via session.execute_tool_result, which only contains MCP-exposed tools, so a forced user function like get_weather is converted into an MCP execution error instead of being returned to the client. Either pass user-function calls through or avoid preserving forced choices for functions that are not MCP-exposed.
Useful? React with 👍 / 👎.
| Some(name) | ||
| if tools | ||
| .iter() | ||
| .any(|tool| utils::function_tool_name(tool) == Some(name)) => |
There was a problem hiding this comment.
Fresh evidence after the earlier comment is that the new remapper only compares the hosted tool_choice.type string to the rewritten function names. In normal builtin routing the exposed function name comes from builtin_tool_name (for example builtin_type: image_generation with builtin_tool_name: generate_image), so {"type":"image_generation"} misses here and is downgraded to auto; the initial request no longer forces the hosted tool call and models that rely on a forced choice can skip it. The hosted type needs to be resolved through the builtin routing/session mapping before falling back.
Useful? React with 👍 / 👎.
1 participant
Description
Problem
The OpenAI router's MCP tool loop rewrote every
tool_choiceto"auto"inprepare_mcp_tools_as_functionsbefore the initial model request, silently dropping explicit"required","none", and specific-function choices (#1945). A model that only calls tools under"required"could skip the MCP call entirely.Solution
Exactly the semantics the reporter proposed, which the gRPC responses loops already implement per-iteration:
tool_choice("required"/"none"/"auto") is preserved. Object choices are remapped through the MCP/hosted→function rewrite: they resolve by name against the rewritten function tools (a hosted type like{"type": "image_generation"}matches the function of the same name and becomes{"type": "function", "name": "image_generation"}); anything unmappable falls back to"auto", since the referenced tool no longer exists in the rewritten array. Naive verbatim preservation would 400 upstream — theopenai-responsese2e caught exactly that (Tool choice 'image_generation' not found in 'tools') on the first revision of this PR.build_resume_payload):tool_choiceis forced to"auto"— the tool call already happened, and a cloned"required"would otherwise loop until the tool-call limit."auto"(or nothing) behave exactly as before.The two other
tool_choicewrites in the OpenAI router are response-side echo normalization and are untouched — with this fix a preserved"required"now also echoes back faithfully. The gRPC responses loops (regular + Harmony) already had iteration-0-preserves/then-auto logic and need no change.Changes
routers/openai/mcp/tool_loop.rs—prepare_mcp_tools_as_functionsinserts"auto"only whentool_choiceis absent/null;build_resume_payloadsets"auto"alongside the resumed tools."auto", missing choice defaults to"auto", resume forces"auto"over a preserved"required".Test Plan
cargo test -p smg: lib 1258 passed (19 tool-loop tests incl. the 5 new); all integration binaries green (api 106, routing 94, spec 96, security 50, …; 0 failures).cargo clippy --all-targets -- -D warnings,cargo +nightly fmt --check.tool_choice: "required"with adeepwikiMCP tool now reaches the upstream model as"required"on the first request and"auto"on post-tool iterations.Checklist
cargo +nightly fmtpassescargo clippy --all-targets --all-features -- -D warningspassesSummary by CodeRabbit