feat(agent): expose fallback model awareness to the agent - #26569
Closed
zccyman wants to merge 1 commit into
Closed
Conversation
Inject fallback notification into ephemeral_system_prompt when the primary model fails and a fallback activates. Also show fallback status in the /model picker when a fallback is active. Closes NousResearch#25852
zccyman
force-pushed
the
feat/fallback-model-awareness-25852
branch
from
May 18, 2026 07:48
585a8d7 to
e70032a
Compare
Contributor
|
Thanks for the clear fallback-awareness proposal. This is already implemented on current main via a cache-safe, broader fix. Automated hermes-sweeper review.
The linked request #25852 was also closed as implemented with this same commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When Hermes Agent's fallback mechanism activates (primary model fails, backup kicks in), the agent (LLM) now receives a notification in its system prompt indicating the active runtime model. This enables model-aware response behaviors like correct model code prefixes.
Closes #25852
Problem
The fallback mechanism was completely transparent to the agent. When a fallback kicked in, the system prompt still showed the configured primary model. Users who prefix responses with model codes (e.g., G31P-T for gemini-3.1-pro) would get wrong prefixes after fallback.
Solution
Three targeted changes:
run_agent.py - ephemeral_system_prompt injection: When _try_activate_fallback() succeeds, a note is appended to ephemeral_system_prompt (per-call, not cached) telling the agent which fallback model/provider is active. Using ephemeral_system_prompt avoids breaking prefix cache stability.
run_agent.py - cleanup on primary restoration: When _restore_primary_runtime() runs at turn start, the fallback note is stripped from ephemeral_system_prompt so subsequent turns don't carry stale messages.
cli.py - /model status display: When /model is invoked with no args during an active fallback, a status line shows the primary to fallback transition.
Files Changed
Test Results
tests/run_agent/test_fallback_model.py - 31 passed (162.92s)
All existing fallback tests pass without modification.
Design Decisions