Skip to content

fix: retry title generation without response_format on gateway 400 - #87508

Closed
honjiaxuan wants to merge 2 commits into
NousResearch:mainfrom
honjiaxuan:fix/title-generation-response-format-fallback
Closed

honjiaxuan wants to merge 2 commits into
NousResearch:mainfrom
honjiaxuan:fix/title-generation-response-format-fallback

Conversation

@honjiaxuan

Copy link
Copy Markdown

Problem

Session auto-titling fails with:

Auxiliary title generation failed: HTTP 400: Error from provider (Console Go):
Upstream request failed: [invalid_request_error] This response_format type is unavailable now

generate_title() sends a strict json_schema response_format on every request. Some gateways — e.g. OpenCode Zen ("Console Go") — reject that payload outright with HTTP 400 instead of ignoring it. The parsing fallbacks in _extract_title_text (loose JSON scan / prose) never get a chance because the API call itself fails, so every new session ends up untitled and the user sees a warning banner.

Fix

Retry once without response_format when the error message mentions response_format or is an invalid_request_error. The title prompt already demands a {"title": ...} object, and _extract_title_text falls back through a loose JSON scan, so the retry still yields a proper title — verified end-to-end against the OpenCode Zen gateway (first call 400, retry 200, title produced).

Non-response_format errors still propagate to the existing failure callback unchanged.

Tests

Added test_retries_without_response_format_when_gateway_rejects_json_schema — first call rejects with the exact "Console Go" error, retry succeeds, title is parsed. Full tests/agent/test_title_generator.py suite passes (34 tests).

@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference, author can ignore or act on any point.

fix: retry title generation without response_format on gateway 400

  1. agent/title_generator.py (~line 403): the retry decision is a substring match on the exception text ("response_format" in err or "invalid_request_error" in err). A provider that mentions response_format in an unrelated error (e.g. "unknown parameter response_format") would also trigger the retry. Harmless since the retry is a single no-op fallback, but matching on the HTTP 400 + invalid_request_error combination (or an exception type with a status code) would be more precise.
  2. The retry drops the structured constraint entirely (_request_title({})) and relies on _extract_title_text's loose JSON scan to recover {"title": ...}. The new test only covers the case where the fallback content is pure JSON; consider one more test where the retry returns plain prose (no braces) to pin the loose-parse recovery behavior.
  3. logger.debug for the retry means the fallback is invisible in default agent.log. Since this indicates a real gateway incompatibility that may affect other response_format users, a logger.warning (or info) would make it discoverable.
  4. Single retry only — good; no retry-storm risk. The guard is well scoped.

@honjiaxuan

honjiaxuan commented Aug 17, 2026 •

Copy link
Copy Markdown
Author

Thanks for the review — all three actionable points are addressed in new commit 313b981:

1. Retry condition narrowed. Now requires 400 or invalid_request_error alongside response_format in the error text:

if (
    "response_format" in err
    and ("invalid_request_error" in err or "400" in err)
):

An error that merely names response_format (e.g. an unknown-parameter typo) now re-raises and reaches the existing failure callback instead of being papered over. (Checking an exception type with a status code would be ideal, but by this point in the aux-client chain the error has been flattened into a generic RuntimeError carrying the upstream message, so the text signal is what's reliably available.)

2. Prose-fallback test added. test_prose_fallback_recovers_title_when_retry_returns_no_json pins the loose-parse recovery when the retry returns plain prose (no braces).

3. Logged at WARNING. The fallback now logs:
Title generation: provider rejected response_format (HTTP 400); retrying without structured output: ...
so the gateway incompatibility is visible in default agent.log.

4. Agreed — single retry, no risk of a storm (and the retry only fires on the 400/invalid_request_error signal, not on every failure).

Full tests/agent/test_title_generator.py suite: 36 passed.

generate_title() sends a strict json_schema response_format with every
request. Some gateways (OpenCode Zen "Console Go") reject that payload
outright with HTTP 400 instead of ignoring it, so the parsing fallbacks
in _extract_title_text never get a chance and every new session ends up
untitled with an "Auxiliary title generation failed" warning.

Retry once without response_format when the error mentions
response_format or is an invalid_request_error. The prompt already
demands a {"title": ...} object, and _extract_title_text falls back
through a loose JSON scan, so the retry still produces a title.
Address review comments on NousResearch#87508:
- Require 400/invalid_request_error alongside 'response_format' so an
  unrelated error that merely names response_format (e.g. unknown-parameter
  typo) re-raises instead of being papered over.
- Log the fallback at WARNING so the gateway incompatibility is
  discoverable in default agent.log.
- Add tests: prose (non-JSON) retry recovery, and non-retry on unrelated
  response_format mention.

Co-authored-by: NousResearch/hermes-agent CI review
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Aug 17, 2026
@alt-glitch

Copy link
Copy Markdown

This was generated by AI during triage.

Duplicate of #82890: both implement the same single schema-free retry for HTTP 400 response_format/json_schema rejection in title generation.

@honjiaxuan
honjiaxuan force-pushed the fix/title-generation-response-format-fallback branch from 313b981 to 9d072aa Compare August 17, 2026 11:19
@honjiaxuan

Copy link
Copy Markdown
Author

Note: the branch was rebased onto the latest upstream main (which now includes d516783, the answer-shaped auto-title guard) to resolve a merge conflict. The rebase kept both changes — my retry-without-response_format logic and upstream's answer-shaped guard coexist in generate_title. New head: 9d072aa. Same changes as 313b981 (review points 1–3 above remain addressed); full test suite 39 passed, PR mergeable.

@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference, author can ignore or act on any point.

Confirmed — Rebase coexisting with upstream answer-shaped guard confirmed. Good.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks @honjiaxuan for working on the gateway 400 on response_format. This landed on main through #89589 and #113966 (5cc8177), which covers the same symptom on the primary and fallback auxiliary paths and omits the field up front for providers known to reject it. Closing as superseded by the landed fix — the tracking issue (#83390 cluster) is closed with the same references.

@teknium1 teknium1 closed this Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants