Skip to content

fix: honor fallback api_mode override - #29749

Open
jmjeong wants to merge 1 commit into
NousResearch:mainfrom
jmjeong:fix/fallback-api-mode-override
Open

jmjeong wants to merge 1 commit into
NousResearch:mainfrom
jmjeong:fix/fallback-api-mode-override

Conversation

@jmjeong

@jmjeong jmjeong commented May 21, 2026

Copy link
Copy Markdown

Summary

  • honor fallback_providers[*].api_mode before applying fallback provider/base-url heuristics
  • keep existing heuristic behavior when no explicit fallback api_mode is configured
  • add regression coverage for Codex/Responses primary failing over to a direct OpenAI-compatible Chat Completions fallback

Why

When the primary agent runs via Codex/Responses and falls back to a plain OpenAI-compatible chat model, the current fallback activation path re-infers api_mode from https://api.openai.com/v1 and forces codex_responses. That can send Responses-only encrypted reasoning/include metadata to chat-only fallback models and fail with:

Encrypted content is not supported with this model

An explicit fallback entry such as:

fallback_providers:
  - provider: custom
    model: gpt-4.1-mini
    base_url: https://api.openai.com/v1
    api_key_env: OPENAI_API_KEY
    api_mode: chat_completions

should remain on chat_completions even if the primary runtime was codex_responses.

Tests

  • python -m pytest tests/agent/test_fallback_api_mode.py tests/run_agent/test_run_agent_codex_responses.py -q -o 'addopts='

Related: #23450

@jmjeong
jmjeong marked this pull request as ready for review May 21, 2026 09:22
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 21, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

Duplicate of #16346 — same bug where fallback provider activation ignores explicit api_mode config, causing Responses-only metadata to be sent to chat-only fallback models.

@jmjeong
jmjeong force-pushed the fix/fallback-api-mode-override branch from 76b22e1 to cc6eff9 Compare May 23, 2026 22:48
@jmjeong

jmjeong commented May 23, 2026

Copy link
Copy Markdown
Author

Thanks — agreed this is the same underlying bug as #16346.

I refreshed this PR to incorporate the useful coverage from #16346 while keeping it aligned with the current code layout:

  • applies the fix in agent/chat_completion_helpers.py, where fallback activation now lives on current main
  • validates and forwards explicit fallback api_mode into resolve_provider_client(..., api_mode=...)
  • keeps explicit api_mode ahead of provider/base-url/model heuristics
  • propagates the activated fallback api_mode into the context compressor update
  • adds regression coverage for the concrete Codex/Responses primary -> OpenAI-compatible chat fallback case, including that Responses-only payload fields are not emitted for chat_completions
  • adds tests for no override, invalid override, and compressor transport propagation

Local validation:

python -m py_compile agent/chat_completion_helpers.py tests/agent/test_fallback_api_mode.py
python -m pytest tests/agent/test_fallback_api_mode.py tests/run_agent/test_provider_fallback.py tests/run_agent/test_compressor_fallback_update.py -q -o 'addopts='
# 28 passed

I could also close this in favor of #16346 if maintainers prefer that one, but #16346 currently targets the older run_agent.py implementation path and is dirty against current main; this PR is intended as the current-layout version with the Codex -> chat fallback regression case preserved.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused regression coverage. The reported defect remains present on current main: try_activate_fallback() reads fallback base URL/key at agent/chat_completion_helpers.py:1428-1444, but never reads fb["api_mode"]; it then forces codex_responses for a direct OpenAI fallback at agent/chat_completion_helpers.py:1483-1484.

Problems

  • The PR is based on an older version of this helper. Current main added base_url_hostname(fb_base_url) == "api.anthropic.com" to the Anthropic fallback condition at agent/chat_completion_helpers.py:1470 in 18c61bb8c. Since this PR is currently dirty, accepting its older replacement conditional during salvage would lose that fix.

Suggested changes

  • Port the override gate around the current inference chain, preserving the current native-Anthropic-host check.
  • Keep the existing compressor propagation already present at agent/chat_completion_helpers.py:1629-1635 (dcbcdd652), rather than duplicating that hunk.

Automated hermes-sweeper review.

and base_url_host_matches(fb_base_url, "amazonaws.com")
):
fb_api_mode = "bedrock_converse"
if not fb_api_mode_hint:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During salvage, retain current main's additional or base_url_hostname(fb_base_url) == "api.anthropic.com" clause in this branch (agent/chat_completion_helpers.py:1470, 18c61bb8c); accepting this stale conditional unchanged would reintroduce the native-Anthropic fallback misrouting fix.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants