fix(openai): model-aware api_mode for direct api.openai.com (GPT-4.1/4o use Chat Completions) - #47820
Conversation
Direct OpenAI (`openai-api` provider, and any api.openai.com base URL)
was hardcoded to the Responses API (`codex_responses`) for every model.
That breaks non-reasoning chat models, which the Responses API rejects:
- gpt-4.1 / gpt-4o → 400 "Encrypted content is not supported with this
model"
- other GPT-4.x / GPT-3.5 chat models → same class of failure
Reasoning families (GPT-5.x, o1/o3/o4, codex) genuinely require the
Responses API, so the host can't pick one transport for all models.
Add `openai_model_api_mode(model)` (mirrors the existing
`azure_foundry_model_api_mode` / `copilot_model_api_mode` precedent —
Azure Foundry hosts the same OpenAI families, so the Responses-only
prefix set is shared) and thread the effective model through the two
api_mode-governing paths:
- runtime_provider._detect_api_mode_for_url(base_url, model)
- providers.determine_api_mode(provider, base_url, model)
plus their OpenAI-resolving callers (pool, explicit, env-key, openrouter
runtime paths, and the two model_switch call sites).
Unknown/empty model preserves the historical Responses-API default for
api.openai.com, so the GPT-5.x family Hermes ships as the openai-api
default is unchanged. gpt-5/o-series behavior is unchanged.
Adds unit coverage for openai_model_api_mode, _detect_api_mode_for_url,
determine_api_mode, and an end-to-end resolve_runtime_provider test for
openai-api + gpt-4.1 / gpt-5.5 / o4-mini.
|
Related: competing fixes for #23450 (Responses-API "Encrypted content is not supported with this model"). This PR adds a model-aware api_mode for the direct OpenAI path via a new openai_model_api_mode() helper threaded through runtime_provider/providers/model_switch. Other open PRs attack the same bug from different layers: #23460 and #24234 strip the unsupported include parameter at the Chat Completions transport, and #23734 gates the codex_responses upgrade at agent init. Same goal, different mechanism — flagged so reviewers can compare the approaches and pick one. |
…lyze + implement, hermetic isolation, multi-provider auto-routing) Adds HermesAdapter (hermes chat headless) for analyze + full-edit implement across Gemini/Anthropic/OpenAI, verified end-to-end for both modes. - Process-group isolation (start_new_session) so Hermes can't signal-kill the orchestrator; git-diff PATCH attribution + stdout-parsed analyze success to tolerate Hermes's unreliable exit codes; failure classifier (incl. HTTP 404 model -> model_unavailable). - Hermetic workers: default --ignore-rules + toolsets that exclude memory/session_search, closing the cross-session memory/session_search leak (a worker could recall an unrelated prior session). Opt back in via --use-hermes-rules / custom --toolsets. - First-class wiring: platform_lock.KNOWN_ADAPTERS, MCP implement priority + hermes_command, `puppetmaster hermes` CLI subcommand, diagnostics configured detection. - Auto-routing: curated API-billed Hermes catalog stamps the required provider via payload_defaults; `models discover --source hermes` seeds it. Full suite 712 green (+10 tests). Upstream OpenAI api_mode fix tracked in NousResearch/hermes-agent#47820.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing the transport decision through the runtime resolver; the model-aware direction addresses a real current-main issue (hermes_cli/runtime_provider.py:126).
Problems
- In
724abf74b53d(hermes_cli/models.py:3299in the PR diff), the final fallback returnschat_completionsfor every nonempty unrecognized ID. That conflicts with the PR's documented unknown-model behavior and changes the historical Responses fallback for future/unrecognized model families. hermes_cli/cli_agent_setup_mixin.py:41-45still callsresolve_runtime_provider()withouttarget_model. The one-shot path passes it (hermes_cli/oneshot.py:375-379), but the interactive setup path can classify frommodel.defaultrather than the active selected model.
Suggested changes
- Return
Nonefor unrecognized nonempty model IDs and explicitly identify the GPT-3.5/GPT-4 chat families. - Thread the effective CLI model into the setup resolver and add a regression where it differs from
model.default.
Automated hermes-sweeper review.
| if raw.startswith(prefix): | ||
| return "codex_responses" | ||
| return "chat_completions" | ||
|
|
There was a problem hiding this comment.
This fallback classifies every nonempty unknown ID as Chat Completions, despite the documented unknown-model behavior preserving the Responses default. Please return None unless this is a recognized GPT-3.5/GPT-4 chat family; otherwise a future reasoning model such as gpt-6-preview is silently downgraded.
Problem
Direct OpenAI access (the built-in
openai-apiprovider, and anyapi.openai.combase URL) is hardcoded to the Responses API(
codex_responses) for every model. The Responses API rejectsnon-reasoning chat models, so they're currently unusable via Hermes:
gpt-4.1,gpt-4o→ HTTP 400 "Encrypted content is not supported with this model" (this is theinclude: ["reasoning.encrypted_content"]parameter the Responses path sends — see [Bug]: [Bug]: Hermes v0.13.0 sends unsupportedincludeparameter causing "Encrypted content is not supported with this model" error with OpenAI GPT-4o #23450)Reasoning families (GPT-5.x, o1/o3/o4, codex) genuinely require the
Responses API, so a single transport per host can't be correct — it has
to be chosen per model.
Repro
Fix
Hermes already solves this exact problem per-model for other providers —
azure_foundry_model_api_mode(Azure Foundry) andcopilot_model_api_mode(Copilot) both return the Responses API only for GPT-5+/o-series and Chat
Completions otherwise. This PR gives the direct OpenAI path the same
treatment:
openai_model_api_mode(model)inhermes_cli/models.py. It reuses theexisting
_AZURE_FOUNDRY_RESPONSES_PREFIXESset (Azure Foundry hosts thesame OpenAI families, so the Responses-only prefixes are identical):
reasoning families →
codex_responses, chat families →chat_completions,unknown/empty →
None.their OpenAI-resolving callers:
runtime_provider._detect_api_mode_for_url(base_url, model)— pool,explicit, env-key, and openrouter resolution paths.
providers.determine_api_mode(provider, base_url, model)— used by the/modelswitch path inmodel_switch.py.Behavior
gpt-4.1,gpt-4o,gpt-4-turbo,gpt-3.5*codex_responses(400)chat_completions✅gpt-5,gpt-5.5,o1/o3/o4,codex*codex_responsescodex_responses(unchanged)codex_responsescodex_responses(unchanged)The unknown-model case preserves the historical Responses-API default, so
the
gpt-5.xfamily Hermes ships as theopenai-apidefault keeps workingexactly as before. An explicit
model.api_mode/ custom-provider overridestill wins.
Note:
o4-mini's separate "organization must be verified to generatereasoning summaries" error is an OpenAI account-level setting, not a
transport bug — o4-mini correctly stays on the Responses API here.
Tests
python -m pytest \ tests/hermes_cli/test_detect_api_mode_for_url.py \ tests/hermes_cli/test_determine_api_mode_hostname.py \ tests/hermes_cli/test_api_key_providers.py \ tests/hermes_cli/test_runtime_provider_resolution.py \ tests/hermes_cli/test_user_providers_model_switch.py \ tests/cli/test_cli_provider_resolution.py -q # 387 passedNew coverage:
openai_model_api_modeunits, model-aware_detect_api_mode_for_url/determine_api_mode, and an end-to-endresolve_runtime_provider(requested="openai-api", target_model=…)testasserting
gpt-4.1→chat_completionswhilegpt-5.5/o4-ministaycodex_responses.Fixes #23450