Skip to content

feat(providers): add ordered fallback provider chain (salvage #1761) - #3813

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-b8e0ec91
Mar 29, 2026
Merged

feat(providers): add ordered fallback provider chain (salvage #1761)#3813
teknium1 merged 1 commit into
mainfrom
hermes/hermes-b8e0ec91

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Salvage of #1761 by @uzaylisak. Extends the single fallback_model mechanism into an ordered provider chain. When the primary model fails (rate limit, 4xx, connection error), Hermes tries each fallback in sequence until one succeeds.

Closes #1734.

Config

# New list format — tried in order
fallback_providers:
  - provider: openrouter
    model: anthropic/claude-sonnet-4
  - provider: openai
    model: gpt-4o
  - provider: zai
    model: glm-4.7

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR

The original PR modified _try_activate_fallback() to use a chain index but did NOT update the call sites. Several call sites guarded with not self._fallback_activated, which prevented the chain from ever advancing past provider #1. This salvage replaces those guards with self._fallback_index < len(self._fallback_chain) so the chain actually works.

Additionally, when a provider in the chain fails to resolve (unconfigured, auth error, etc.), the chain now skips to the next entry instead of stopping.

Changes

File Change
run_agent.py _fallback_chain + _fallback_index replaces one-shot _fallback_model; call sites updated
cli.py Reads fallback_providers with legacy fallback_model compat
gateway/run.py Same
hermes_cli/config.py fallback_providers: [] in DEFAULT_CONFIG
tests/test_provider_fallback.py 12 new tests for chain init, advancement, skip behavior
tests/test_run_agent.py 5 existing test fixtures updated for new attributes
tests/test_compressor_fallback_update.py 1 fixture updated

Live test results

Tested with real OpenRouter API — primary model 404s, chain advances through fallbacks:

Primary: openai/fake-model-1 → 400 "not a valid model ID"
  ⚠️ Non-retryable error (HTTP 400) — trying fallback...
  🔄 switching to: openai/fake-model-2 (openrouter)
Fallback #1: openai/fake-model-2 → 400 "not a valid model ID"
  ⚠️ Non-retryable error (HTTP 400) — trying fallback...
  🔄 switching to: anthropic/claude-sonnet-4 (openrouter)
Fallback #2: anthropic/claude-sonnet-4 → ✓ "chain works"

Test results

6806 passed, 9 pre-existing failures, 0 regressions.

Extends the single fallback_model mechanism into an ordered chain.
When the primary model fails, Hermes tries each fallback provider in
sequence until one succeeds or the chain is exhausted.

Config format (new):
  fallback_providers:
    - provider: openrouter
      model: anthropic/claude-sonnet-4
    - provider: openai
      model: gpt-4o

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR: the call sites in the retry loop now use
_fallback_index < len(_fallback_chain) instead of the old one-shot
_fallback_activated guard, so the chain actually advances through
all configured providers.

Changes:
- run_agent.py: _fallback_chain list + _fallback_index replaces
  one-shot _fallback_model; _try_activate_fallback() advances
  through chain; failed provider resolution skips to next entry;
  call sites updated to allow chain advancement
- cli.py: reads fallback_providers with legacy fallback_model compat
- gateway/run.py: same
- hermes_cli/config.py: fallback_providers: [] in DEFAULT_CONFIG
- tests: 12 new chain tests + 6 existing test fixtures updated

Co-authored-by: uzaylisak <uzaylisak@users.noreply.github.com>
@teknium1
teknium1 merged commit 252fbea into main Mar 29, 2026
4 of 5 checks passed
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…earch#1761) (NousResearch#3813)

Extends the single fallback_model mechanism into an ordered chain.
When the primary model fails, Hermes tries each fallback provider in
sequence until one succeeds or the chain is exhausted.

Config format (new):
  fallback_providers:
    - provider: openrouter
      model: anthropic/claude-sonnet-4
    - provider: openai
      model: gpt-4o

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR: the call sites in the retry loop now use
_fallback_index < len(_fallback_chain) instead of the old one-shot
_fallback_activated guard, so the chain actually advances through
all configured providers.

Changes:
- run_agent.py: _fallback_chain list + _fallback_index replaces
  one-shot _fallback_model; _try_activate_fallback() advances
  through chain; failed provider resolution skips to next entry;
  call sites updated to allow chain advancement
- cli.py: reads fallback_providers with legacy fallback_model compat
- gateway/run.py: same
- hermes_cli/config.py: fallback_providers: [] in DEFAULT_CONFIG
- tests: 12 new chain tests + 6 existing test fixtures updated

Co-authored-by: uzaylisak <uzaylisak@users.noreply.github.com>
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…earch#1761) (NousResearch#3813)

Extends the single fallback_model mechanism into an ordered chain.
When the primary model fails, Hermes tries each fallback provider in
sequence until one succeeds or the chain is exhausted.

Config format (new):
  fallback_providers:
    - provider: openrouter
      model: anthropic/claude-sonnet-4
    - provider: openai
      model: gpt-4o

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR: the call sites in the retry loop now use
_fallback_index < len(_fallback_chain) instead of the old one-shot
_fallback_activated guard, so the chain actually advances through
all configured providers.

Changes:
- run_agent.py: _fallback_chain list + _fallback_index replaces
  one-shot _fallback_model; _try_activate_fallback() advances
  through chain; failed provider resolution skips to next entry;
  call sites updated to allow chain advancement
- cli.py: reads fallback_providers with legacy fallback_model compat
- gateway/run.py: same
- hermes_cli/config.py: fallback_providers: [] in DEFAULT_CONFIG
- tests: 12 new chain tests + 6 existing test fixtures updated

Co-authored-by: uzaylisak <uzaylisak@users.noreply.github.com>
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…earch#1761) (NousResearch#3813)

Extends the single fallback_model mechanism into an ordered chain.
When the primary model fails, Hermes tries each fallback provider in
sequence until one succeeds or the chain is exhausted.

Config format (new):
  fallback_providers:
    - provider: openrouter
      model: anthropic/claude-sonnet-4
    - provider: openai
      model: gpt-4o

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR: the call sites in the retry loop now use
_fallback_index < len(_fallback_chain) instead of the old one-shot
_fallback_activated guard, so the chain actually advances through
all configured providers.

Changes:
- run_agent.py: _fallback_chain list + _fallback_index replaces
  one-shot _fallback_model; _try_activate_fallback() advances
  through chain; failed provider resolution skips to next entry;
  call sites updated to allow chain advancement
- cli.py: reads fallback_providers with legacy fallback_model compat
- gateway/run.py: same
- hermes_cli/config.py: fallback_providers: [] in DEFAULT_CONFIG
- tests: 12 new chain tests + 6 existing test fixtures updated

Co-authored-by: uzaylisak <uzaylisak@users.noreply.github.com>
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…earch#1761) (NousResearch#3813)

Extends the single fallback_model mechanism into an ordered chain.
When the primary model fails, Hermes tries each fallback provider in
sequence until one succeeds or the chain is exhausted.

Config format (new):
  fallback_providers:
    - provider: openrouter
      model: anthropic/claude-sonnet-4
    - provider: openai
      model: gpt-4o

Legacy single-dict fallback_model format still works unchanged.

Key fix vs original PR: the call sites in the retry loop now use
_fallback_index < len(_fallback_chain) instead of the old one-shot
_fallback_activated guard, so the chain actually advances through
all configured providers.

Changes:
- run_agent.py: _fallback_chain list + _fallback_index replaces
  one-shot _fallback_model; _try_activate_fallback() advances
  through chain; failed provider resolution skips to next entry;
  call sites updated to allow chain advancement
- cli.py: reads fallback_providers with legacy fallback_model compat
- gateway/run.py: same
- hermes_cli/config.py: fallback_providers: [] in DEFAULT_CONFIG
- tests: 12 new chain tests + 6 existing test fixtures updated

Co-authored-by: uzaylisak <uzaylisak@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support multiple configured providers with ordered fallback sequence (like pi-mono)

1 participant