fix: honor fallback api_mode overrides

[y0shua1ee]

What does this PR do?

Honors explicit fallback_providers[].api_mode when Hermes activates a fallback provider.

Custom providers can declare api_mode: anthropic_messages, api_mode: chat_completions, or another transport mode, but fallback activation currently recomputes transport from provider/base URL/model heuristics. For Anthropic-compatible gateways whose URL does not end in /anthropic, the fallback can resolve credentials correctly while leaving the agent runtime in the wrong transport mode.

This PR makes the fallback entry's explicit api_mode authoritative in the in-agent failover path and applies the same per-entry precedence in the gateway auth fallback path.

Closest related PRs checked:

• fix(agent): respect fallback api_mode overrides #16346, fix(fallback): respect explicit api_mode on fallback_providers entries #24631, and fix: honor fallback api_mode override #29749 cover the same fallback api_mode class in the agent/runtime path.
• Respect explicit api_mode on fallback providers #33140 was closed as another duplicate of this class.
• This PR keeps the gateway auth fallback path coverage as the differentiator, so gateway/run.py::_try_resolve_fallback_provider() also preserves the per-entry api_mode hint.

Related Issue

Partially addresses #33062 for fallback provider entries that declare an explicit api_mode.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• agent/chat_completion_helpers.py
  • Passes the fallback entry's explicit api_mode through provider client resolution.
  • Keeps existing provider/base URL/model heuristics as the fallback behavior when no explicit api_mode is configured.
• gateway/run.py
  • Applies the same per-entry api_mode precedence while resolving gateway auth fallback providers.
• tests/run_agent/test_run_agent.py
  • Adds regression coverage for Anthropic Messages fallback activation.
• tests/gateway/test_auth_fallback.py
  • Adds regression coverage for gateway auth fallback preserving per-entry api_mode.

How to Test

1. Configure a fallback provider entry that explicitly sets a transport mode, for example:

fallback_providers:
  - provider: custom:anthropic-facade
    model: claude-compatible-model
    base_url: https://example.com/v1
    api_mode: anthropic_messages

2. Trigger fallback activation from a failing primary provider.
3. Verify the fallback runtime keeps the configured api_mode instead of recomputing transport from provider/base URL/model heuristics.
4. Run the targeted regression tests:

python -m pytest tests/run_agent/test_run_agent.py::TestFallbackAnthropicProvider \
                 tests/gateway/test_auth_fallback.py \
                 -q -o 'addopts='

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS with the targeted tests above; GitHub CI is green

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Targeted regression command:

$ python -m pytest tests/run_agent/test_run_agent.py::TestFallbackAnthropicProvider tests/gateway/test_auth_fallback.py -q -o 'addopts='

Current GitHub CI for this PR is green, including tests, e2e, ruff enforcement, nix checks, build jobs, attribution, and supply-chain scan.

[alt-glitch]

Duplicate of #16346 (fallback api_mode override ignored). Superset — also covers gateway auth fallback path. Prior duplicates: #29749, #24631, #33140.

[y0shua1ee]

Thanks for the duplicate-context note. I updated the PR body to use the project template and link #33062.

This overlaps with #16346 / #24631 / #29749 on the fallback_providers[*].api_mode bug class. The intended delta in this PR is the gateway auth fallback path: gateway/run.py::_try_resolve_fallback_provider() now preserves the per-entry api_mode, with regression coverage in tests/gateway/test_auth_fallback.py, alongside the agent/runtime fallback fix.

[fqx]

Ran into this exact issue today with a custom provider using api_mode: anthropic_messages configured in custom_providers, set as a fallback_providers entry.

Setup:

custom_providers:
- name: claude-api
  base_url: https://one.cmaster.org
  api_mode: anthropic_messages
  model: claude-haiku-4-5-20251001
  key_env: ANTHROPIC_API_KEY

fallback_providers:
- provider: claude-api
  model: claude-haiku-4-5-20251001
  key_env: ANTHROPIC_API_KEY
  base_url: https://one.cmaster.org

Symptom: When the primary model (a local Qwen via custom endpoint) went down, the fallback to claude-api produced empty responses every time. /model claude-api worked fine in the same session.

Root cause (traced through source):

• /model switch goes through _resolve_named_custom_runtime() → reads custom_provider.get("api_mode") → correctly gets anthropic_messages
• Fallback activation in try_activate_fallback() ignores the fallback entry's api_mode and re-derives it from provider name / URL heuristics
• claude-api ≠ "anthropic", one.cmaster.org doesn't end with /anthropic → falls through to fb_api_mode = "chat_completions"
• Agent sends OpenAI-wire /v1/chat/completions request instead of Anthropic-native /v1/messages
• One API proxy returns empty streaming chunks for this malformed path combination → hermes sees empty response and retries until exhausted

Workaround applied locally (essentially what this PR does):

# agent/chat_completion_helpers.py — inside try_activate_fallback()
# Before:
fb_api_mode = "chat_completions"

# After:
fb_api_mode = (fb.get("api_mode") or "").strip()
if not fb_api_mode:
    if fb_provider == "openai-codex":
        ...
    elif fb_provider == "anthropic" or ...:
        ...
    # (rest of heuristics indented inside if not fb_api_mode)
if not fb_api_mode:
    fb_api_mode = "chat_completions"

Also added api_mode: anthropic_messages to the fallback_providers entry in config.yaml.

This fix matters especially for users running Anthropic-compatible proxies (LiteLLM, One API, etc.) as fallbacks who need native Anthropic format for prompt caching — switching to the OpenAI-compatible endpoint loses cache_control support entirely.

Would be great to see this merged. The fix is straightforward and the behavior gap between /model switching and fallback activation is surprising.

[teknium1]

Thanks for tracing the explicit fallback transport gap. Current main still has the core issue: try_activate_fallback() resolves the fallback client without entry api_mode at agent/chat_completion_helpers.py:1441, then derives the transport heuristically at :1461; gateway auth fallback similarly returns runtime["api_mode"] at gateway/run.py:1967-1985.

Problems

• The added raw-hint gate would suppress heuristics for an invalid non-empty api_mode, while also passing that invalid value to resolve_provider_client. Validate once and treat an invalid value as absent before both operations.
• The same initial-auth fallback pattern remains in cron/scheduler.py:2937-2942, tui_gateway/server.py:4469-4474, hermes_cli/cli_agent_setup_mixin.py:60, and agent/agent_init.py:1035-1039; none applies fallback-entry api_mode.

Suggested changes

• Share validated per-entry runtime/transport resolution across these paths and add regressions for explicit anthropic_messages plus invalid-mode heuristic fallback.

Automated hermes-sweeper review.

[teknium1]

This checks the raw hint rather than the validated mode above. A typo such as api_mode: anthropic_message skips the existing URL/provider/model heuristics and leaves chat_completions; normalize to a validated value first, pass None for invalid input, and gate heuristics on that normalized value.

[y0shua1ee]

@teknium1 — I implemented the sweeper recommendations, rebased onto current main, and pushed head 34f68946f.

What changed:

• Added shared validated fallback-entry resolution across live failover, init-time fallback, Gateway, CLI, Cron, and TUI.
• A valid per-entry api_mode now wins consistently; an invalid value is treated as absent before both client/runtime resolution and transport selection.
• Preserved provider/URL/model heuristics, including exact-host checks for direct Anthropic/OpenAI/Bedrock endpoints and regressions for lookalike hosts.
• Routed fallback key_env and Ollama credentials through the active profile secret scope.
• Recomputed prompt-cache policy when init-time fallback switches to native Anthropic.

Validation:

• related 7-file regression group: 1007 passed
• final focused helper/init pass: 11 passed
• Ruff and git diff --check passed
• privacy/scope scan clean

Could you take another look?

[y0shua1ee]

Follow-up: the first CI run exposed two pre-existing gateway test doubles that still used the old resolver signature. I updated them to accept and assert the fallback target_model, then pushed head 2abf8958b.

The same run also hit an unrelated async-delegation timing race; this PR does not touch that module, and the exact failing file passed locally on rerun. Focused CI reproduction is now 30 passed (test_session_model_override_routing.py + test_async_delegation.py), with Ruff and diff checks still clean. A fresh CI run is starting.