fix(harmony): shrink BUILTIN_TOOLS to the gpt-oss-native tool set

[slin1237]

Summary

Remove image_generation and shell from harmony/builder.rs BUILTIN_TOOLS. Per the openai-harmony spec, gpt-oss is trained to emit only web_search_preview, web_search, code_interpreter/container, file_search, plus mcp / function / custom tools. The two extras (landed via T4 #1303 and T6 #1342 respectively) advertise hosted tools that gpt-oss was not trained for, producing undefined model behavior.

Why this matters

Today's path for a request like \{model: gpt-oss-120b, tools: [\{type: image_generation\}]}:

1. harmony/builder.rs projects ResponseTool::ImageGeneration → string "image_generation".
2. BUILTIN_TOOLS.contains("image_generation") → true → has_custom_tools() returns false → model is advertised the tool as a builtin in the system message.
3. gpt-oss has never seen this builtin during training. Behavior: hallucinated tool call in the wrong shape, ignored advertisement, or garbled output. No gateway handler exists downstream either — if the model does emit an image_generation_call, it's not dispatched anywhere.
4. If it reaches multi-turn replay: responses_to_chat and parse_response_item_to_harmony_message both return Err("Unsupported input item type"), but that only fires on replay, not first-turn emission.

Same story for shell on gpt-oss.

What this changes

• BUILTIN_TOOLS shrinks to [web_search_preview, web_search, code_interpreter, container, file_search] — the gpt-oss-native hosted-tool set.
• image_generation and shell tools fall through to the existing custom/function-tool path instead of being advertised as builtins.
• Multi-turn replay rejection at the conversion layer is unchanged (still returns Err for ImageGenerationCall / ShellCall items).
• Added doc comment explaining the invariant so future T-task merges don't silently re-expand the array.

Followup (tracked as R0 in the audit plan)

The proper fix is per-worker hosted-tool capability flags (HostedToolCapabilities bitflags or extension of ModelType), populated per model (gpt-oss = WEB_SEARCH_PREVIEW + WEB_SEARCH + CODE_INTERPRETER + FILE_SEARCH + MCP + CUSTOM; OpenAI gpt-5 = all; SGLang Llama = none). BUILTIN_TOOLS becomes per-request worker.hosted_tools() ∩ request.tools. Separate PR.

Test plan

• cargo test -p smg --lib — 621 passed, 0 failed.
• cargo clippy -p smg --lib --tests -- -D warnings — clean.
• cargo fmt --all — clean.
• No protocol types changed, no router behavior changed beyond the advertisement set.

Summary by CodeRabbit

• Bug Fixes
  • Corrected the default set of available tools to better align with hosted capabilities and prevent failures from untrained or misconfigured tools.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: d8828f84-7b61-4dd9-8969-1bf947aa4a18

📥 Commits

Reviewing files that changed from the base of the PR and between f394f60 and b6ede1a.

📒 Files selected for processing (1)

• model_gateway/src/routers/grpc/harmony/builder.rs

---

📝 Walkthrough

Walkthrough

The pull request modifies the BUILTIN_TOOLS constant in the gRPC harmony builder by removing image_generation and shell from the built-in tool list. Documentation is updated to clarify the hosted tools subset and describe failure modes for untrained tools.

Changes

Cohort / File(s)	Summary
Tool Configuration model_gateway/src/routers/grpc/harmony/builder.rs	Removed image_generation and shell from the BUILTIN_TOOLS constant and expanded documentation to define the gpt-oss hosted tools subset and describe failure modes for untrained tools.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• lightseekorg/smg#781: Modifies the developer-message guard that relies on with_custom_tools, which is directly affected by changes to the BUILTIN_TOOLS set.
• lightseekorg/smg#1311: Concurrently modifies the same BUILTIN_TOOLS constant in the builder, removing additional tool types from the built-in list.

Suggested labels

grpc, model-gateway

Suggested reviewers

• CatherineSue
• key4ng

Poem

🐰 With carrot-bright edits and careful precision,
We trim the tool list with strategic vision,
Image-gen and shells now rest in the past,
Built-in tools refined, reimagined at last!
Harmony blooms with cleaner design! 🌱

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(harmony): shrink BUILTIN_TOOLS to the gpt-oss-native tool set' directly and specifically describes the main change: reducing BUILTIN_TOOLS to only the gpt-oss-native tools by removing image_generation and shell.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/harmony-builtin-tools-gpt-oss-native

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request updates the BUILTIN_TOOLS list to align with the tools gpt-oss was trained to emit, specifically removing image_generation and shell. While the update is intended to prevent undefined behavior, it exposes several critical issues: is_builtin implementations are now inconsistent with the updated list, tools removed from the built-in set are being incorrectly filtered out rather than falling through to the custom path, and a logic error in the Responses API incorrectly disables the commentary channel when only built-in tools are used, which prevents the delivery of user-facing content.

[claude]

Clean, well-scoped fix. Removing image_generation and shell from BUILTIN_TOOLS is correct — these tools fall outside the gpt-oss training set and advertising them as built-in leads to undefined model behavior. The doc comment clearly explains the rationale and the future direction (per-worker capability flags + MCP dispatch). No issues found.