Skip to content

fix(agent): respect fallback api_mode overrides - #16346

Open
Kayphoon wants to merge 2 commits into
NousResearch:mainfrom
Kayphoon:fix/fallback-provider-api-mode
Open

fix(agent): respect fallback api_mode overrides#16346
Kayphoon wants to merge 2 commits into
NousResearch:mainfrom
Kayphoon:fix/fallback-provider-api-mode

Conversation

@Kayphoon

Copy link
Copy Markdown

What does this PR do?

Makes provider fallback activation honor an explicit api_mode on fallback_providers entries before falling back to provider/base URL/model heuristics.

This fixes custom fallback providers that need Anthropic Messages routing (/v1/messages) but do not use a base URL ending in /anthropic. Before this change, those fallbacks were treated as OpenAI chat completions and could hit the wrong endpoint.

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • run_agent.py
    • Reads a valid api_mode override from fallback config entries.
    • Passes the override through resolve_provider_client(...).
    • Uses the explicit override before auto-detecting fallback API mode from provider/base URL/model heuristics.
    • Propagates the active fallback api_mode into the context compressor update.
  • tests/run_agent/test_fallback_model.py
    • Adds a regression test for provider: custom fallbacks that explicitly set api_mode: anthropic_messages while using a non-/anthropic base URL.

How to Test

  1. Configure a fallback provider like:
    fallback_providers:
      - provider: custom
        model: claude-compatible-model
        base_url: https://relay.example.com/claude-proxy
        api_key: test-key
        api_mode: anthropic_messages
  2. Trigger fallback activation.
  3. Verify Hermes switches to anthropic_messages mode and builds the native Anthropic client instead of keeping an OpenAI chat-completions client.

Test commands run locally:

.venv/bin/python -m py_compile run_agent.py tests/run_agent/test_fallback_model.py
scripts/run_tests.sh tests/run_agent/test_fallback_model.py -q
scripts/run_tests.sh tests/run_agent/test_fallback_model.py::TestTryActivateFallback::test_custom_fallback_honors_explicit_anthropic_api_mode -q
scripts/run_tests.sh tests/run_agent/test_provider_fallback.py -q

Results:

  • tests/run_agent/test_fallback_model.py: 29 passed
  • New focused regression test: 1 passed
  • tests/run_agent/test_provider_fallback.py: 19 passed

I also attempted the full wrapper command required by the local development guide:

scripts/run_tests.sh
scripts/run_tests.sh tests/

Notes:

  • scripts/run_tests.sh with no args failed locally with ARGS[@]: unbound variable before pytest collection.
  • scripts/run_tests.sh tests/ completed collection/execution but failed on unrelated local environment/platform issues, including missing optional modules such as faster_whisper and ACP/MCP/TTS/web/platform-specific tests. The new fallback tests above passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS (Darwin)

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

N/A

Screenshots / Logs

N/A

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating a real fallback-routing gap. The defect remains on current main: agent/chat_completion_helpers.py:1330-1346 does not forward a fallback entry's api_mode, and :1363-1411 then derives transport solely from heuristics.

Problems

  • The patch targets historical code. run_agent.py:4771-4774 is now only a forwarder; the live implementation is agent/chat_completion_helpers.py:1232. git apply --check confirms this branch cannot apply, including because tests/run_agent/test_fallback_model.py no longer exists.
  • The new fixed allowlist would reject valid codex_app_server; canonical validation includes it at hermes_cli/runtime_provider.py:332-352.
  • The sibling init-time and gateway-auth fallback paths also omit the entry override at agent/agent_init.py:1005-1013 and gateway/run.py:1966-1970.

Suggested changes

  • Salvage the precedence logic into agent/chat_completion_helpers.py, aligned with current validation, and move coverage to tests/run_agent/test_provider_fallback.py.
  • Include the init-time and gateway paths if the intended guarantee is that every fallback entry honors its explicit transport.

This is an automated hermes-sweeper review.

Comment thread run_agent.py Outdated
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
@Kayphoon
Kayphoon force-pushed the fix/fallback-provider-api-mode branch from e85a2c4 to 5fbf892 Compare July 13, 2026 21:16
@Kayphoon
Kayphoon force-pushed the fix/fallback-provider-api-mode branch from 5fbf892 to 5e97ca2 Compare July 14, 2026 07:30
@Kayphoon

Copy link
Copy Markdown
Author

Updated this PR after addressing the review feedback and cleaned up the commit history.

Commit structure:

  • c8f6200 preserves the original change supporting explicit fallback api_mode overrides.
  • 5e97ca2 addresses the review feedback by replacing the local fixed allowlist with the canonical _parse_api_mode validator, preserving codex_app_server, and applying the same handling to the init and gateway fallback paths.

Verification:

  • 36 targeted tests passed
  • Ruff passed
  • A manual AIAgent fallback probe confirmed codex_app_server normalization and propagation
  • The final diff remains limited to the 8 fallback-related files

The related inline review thread has been addressed and resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants