Skip to content

fix(gateway): reload fallback chain when config changes - #39022

Closed
Que0x wants to merge 1 commit into
NousResearch:mainfrom
Que0x:fix/gateway-reload-fallback-chain
Closed

fix(gateway): reload fallback chain when config changes#39022
Que0x wants to merge 1 commit into
NousResearch:mainfrom
Que0x:fix/gateway-reload-fallback-chain

Conversation

@Que0x

@Que0x Que0x commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

What & why

The gateway loaded the fallback provider chain once at __init__
(self._fallback_model) and never refreshed it. The agent-cache signature
also omitted the chain, so an operator who edited fallback_providers /
fallback_model in config.yaml kept getting routed on the stale chain —
even on a brand-new session — until a full gateway restart. This contradicts
the documented "changes take effect on the next agent session or gateway
restart" contract (web-dashboard.md), and means a provider auth/rate-limit
fix doesn't actually take hold.

How

All changes in gateway/run.py:

  • _load_fallback_model(cfg=None) — accepts a pre-loaded config so the
    chain can be re-resolved from the current turn's config (no extra disk read).
    Backward compatible; the __init__ call is unchanged.
  • _extract_cache_busting_config — folds the effective merged chain
    (fallback.chain) into the agent-cache signature, so a change rebuilds the
    cached agent and an unchanged chain causes no cache churn.
  • _run_agent / _run_background_task — re-resolve
    self._fallback_model from the freshly-loaded user_config before building
    the agent.

Legacy fallback_model + fallback_providers merge semantics are preserved
(get_fallback_chain). No behavior change for users without a fallback config
(empty chain → None).

How to test

pytest tests/gateway/test_agent_cache.py \
       tests/gateway/test_auth_fallback.py \
       tests/gateway/test_session_model_override_routing.py -q

Tests & results

Suite Result
test_agent_cache.py + test_auth_fallback.py + test_session_model_override_routing.py 76 passed
└ new TestFallbackChainCacheBusting (regression) 7 passed
test_background_command.py (covers edited _run_background_task) 22 passed

New regression tests assert: signature busts when the chain changes,
stays stable when unchanged (no churn), legacy fallback_model merge is
preserved, and _load_fallback_model(cfg) resolves from a passed config.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels Jun 4, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the careful fallback-chain/cache analysis. This is an automated hermes-sweeper review; current main now provides the requested behavior through a later, stronger implementation.

  • be1346cf2a0edb6e0f64bee3f5ce10d470e4fb33 reloads the configured fallback chain for fresh gateway agents, background tasks, and cached-agent reuse.
  • gateway/run.py:5016 now rereads the chain from config.yaml; gateway/run.py:18258 applies the refreshed chain to an already-cached agent outside the cache lock, preserving cache reuse rather than rebuilding solely for this configuration change.
  • 0a01b2087d0c0bb11bb7ed650b8de3e6b5205856 hardens that path by retaining the last known-good chain during transient parse/read failures and clearing stale unavailable-provider memoization after a real chain edit.
  • tests/gateway/test_fallback_chain_reload.py:18 covers live reload, cached-agent application, background/new-agent construction paths, cooldown handling, and torn-write behavior.

Closing as implemented on main.

@teknium1 teknium1 closed this Jul 14, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants