fix(agent): recover leaked Harmony tool calls for gpt-oss on chat endpoints

[zuowen7]

Problem

gpt-oss family models emit their native Harmony tool-call serialization (to=functions.<tool> {...} — Harmony is the gpt-oss family's native function-calling wire format, shared with the Codex Responses API) regardless of backend. The ChatGPT Codex Responses adapter already recovers this leak (_TOOL_CALL_LEAK_PATTERN in agent/codex_responses_adapter.py), but that path only runs for api_mode=codex_responses — the two sanitize_harmony_tokens= call sites in conversation_loop.py sit inside if agent.api_mode == "codex_responses": blocks.

Result: self-hosted gpt-oss (Ollama, vLLM, generic OpenAI-compatible endpoints) leaks raw tool-call markup into the visible assistant response as a confident-looking text answer, with no tool executed. Reproduced in the issue with gpt-oss:120b via Ollama (api_mode: chat_completions).

Approach

Gate leak recovery on the model family, not the backend:

1. run_agent.py: new AIAgent._model_uses_harmony_format(model) static helper (mirrors _model_requires_responses_api conventions) — true for gpt-oss* model ids, vendor prefix stripped.
2. agent/conversation_loop.py: at turn finalization (right beside the existing dropped-tool-call recovery), if a Harmony-format model returned leaked to=functions.<tool> text with no structured tool_calls:
   • discard the leaked markup,
   • append an ephemeral assistant + user-nudge pair (both flagged _harmony_leak_nudge, popped from the durable transcript like the dropped-tool-call pair),
   • re-prompt the model to emit a structured tool call, bounded to 3 consecutive leaks (budget resets on any genuine turn end),
   • on exhaustion, deliver an explicit failure message instead of the raw markup (mirrors the codex "remained incomplete after 3 continuation attempts" terminal).

The recovery is model-gated and transport-agnostic, so it also covers gpt-oss served through anthropic_messages/bedrock proxies, and leaves non-gpt-oss models byte-for-byte untouched.

Tests

New tests/run_agent/test_84158_harmony_leak_chat_completions.py (4 tests):

• test_model_uses_harmony_format — model-family gate unit coverage (incl. vendor-prefixed ids).
• test_harmony_leak_recovered_on_chat_completions — leak → re-prompt → clean second response; asserts the markup never surfaces in final_response or any persisted assistant message.
• test_harmony_leak_gives_up_with_clear_failure_after_3 — persistent leak → 4 API calls total, explicit failure delivered, transcript clean.
• test_harmony_leak_unchanged_for_non_gpt_oss_model — a non-Harmony model keeps today's verbatim-delivery behavior (proves the gate).

Sabotage verified: with the conversation_loop.py change reverted, the two recovery tests fail (leak delivered as-is, 1 API call) — the tests bite.

Local runs (platform: Windows 11 native, venv Python 3.11.15, PYTHONPATH unset so the dev checkout is imported; no platform-specific code touched — pure loop logic, cross-platform safe):

• new file: 4/4 passed
• tests/run_agent/test_dropped_tool_call_recovery.py + tests/run_agent/test_run_agent_codex_responses.py: 48/48 passed
• ruff check on all 3 changed files: clean

Approach comparison: this PR vs #46330

#46330 ("Support client-side Harmony format parsing and stateful streaming scrubbing") addresses the same gap from a different angle: it parses leaked to=functions.<tool> markup into structured tool_calls and executes them, and adds a stateful streaming scrubber. This PR instead performs bounded, model-gated recovery: the leaked markup is discarded and the model is re-prompted to emit a structured tool call (3 attempts max, then an explicit failure message).

Design trade-off: parsing recovers the intended tool call in a single shot, while re-prompting never executes content recovered from leaked text — a model that merely explains the tool-call format (e.g. "the format is to=functions.write_file {...}") cannot trigger an unintended execution, and the recovery stays consistent with the Codex Responses adapter's existing leak handling. The two PRs can coexist; maintainers may prefer either direction or a combination.

Risk / exclusions

• False-positive surface: detection requires the unambiguous to=functions.<tool> marker (same pattern the codex adapter ships) + Harmony-format model + zero structured tool calls. The JSON-envelope shape the reporter paraphrased ({"name": "bash", "cmd": [...]}) is not covered — consistent with the codex adapter's documented stable marker; noted as a boundary in the issue.
• No config, schema, or state changes; no new env vars; no new tools.
• Auxiliary-client paths (compression/title-gen/vision) are out of scope — the leak recovery targets the main conversation loop where visible responses are produced.
• api_mode=codex_responses is unaffected (its adapter already handles the leak before normalization; content arrives cleared).

Closes #84158

[alt-glitch]

This was generated by AI during triage.

Related: #46330 provides a broader client-side Harmony parser/scrubber. This PR instead adds bounded model-gated recovery at conversation finalization; maintainers should compare the transport-level and retry approaches.

[zuowen7]

@teknium1 — quick heads-up as a first-time contributor: my three open PRs (#84231, #83830, #83757) are all stuck at the fork-PR CI approval gate (action_required), so no checks have run on any of them yet. No rush — just flagging in case it needs a one-time approval. Happy to fix anything CI finds once it runs.

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

fix(agent): recover leaked Harmony tool calls for gpt-oss on chat endpoints

1. In the give-up path (3 retries exhausted), final_msg["content"] = "" is set but _harmony_leak_nudge is NOT set on final_msg (unlike the retry path a few lines above), and execution falls through to the scaffolding-pop block which only pops when messages[-1] carries one of the flags. If the blanked assistant message is then appended before finalization appends the failure final_response, history holds two consecutive assistant messages (empty + failure text) — a role-alternation violation on the next turn. Verify the give-up path's message is stripped/merged during finalization, or set the flag on final_msg there too so the pop handles it.

2. Leak detection runs _TOOL_CALL_LEAK_PATTERN.search(final_response) on any gpt-oss final text. A gpt-oss model that legitimately quotes to=functions.terminal {...} in a prose answer (e.g. explaining the tool-call format to the user) would be misdetected as a leak and re-prompted up to 3 times. Consider requiring the pattern to dominate the response (e.g. the response starts with / mostly consists of the markup) rather than any substring match.

3. The terminal failure message ("The model repeatedly emitted tool calls as raw text...") tells the user what happened but not what to do. A one-line actionable hint (switch model / retry) would make the dead-end recoverable.