feat: opt-in user-visible notification on provider fallback (#33174)

[ousiaresearch]

What does this PR do?

Adds opt-in user-visible fallback notification when the primary provider fails and a fallback takes over.

Fixes #33174

Type of Change

• ✨ New feature

Changes Made

• gateway/run.py: Added module-level _set_fallback_status_callback, _gateway_extra_flag, _notify_fallback_used functions
• Wired into _resolve_runtime_agent_kwargs to fire _notify_fallback_used when fallback activates
• Registered callback at gateway agent init via _set_fallback_status_callback
• Controlled by gateway.extra.fallback_notifications: true in config.yaml (opt-in, default off)

How to Test

1. Add gateway.extra.fallback_notifications: true to config.yaml
2. Configure primary provider to fail (e.g. invalid key) with valid fallback
3. Verify user receives notification when fallback activates

Checklist

• I have tested these changes locally

[corluxmaximus]

Thanks for picking this up! A couple of small thoughts on the current implementation — feel free to disregard if I'm missing context.

1. The notification might not fully close the gap on its own. Even with fallback_notifications enabled, I noticed that when a user directly asks the model "which model are you?" after a fallback, it still seems to answer with the primary model's identity — the response appears to be sourced from config.model.default rather than the provider actually serving the request. I was able to reproduce this in terminal mode (via the z flag). So a user who explicitly asks still gets the primary's name back, which felt like maybe a bigger gap than a missing notification. Could be worth looking at in the same area, if it's easy.

2. On the default value. The flag currently defaults to False (_gateway_extra_flag("fallback_notifications", False)). I was wondering whether it might make sense to default it to True — or at least when the fallback target is a paid third-party / different-vendor provider? Since the feature is essentially about letting users know where their requests are going, having it opt-in could slightly undercut that. Just raising it as a question though — totally open to there being a good reason for the current default, would be curious to hear your thinking.

Also, this looks like it only touches gateway mode at the moment — is the terminal/CLI path in scope here too, or is that tracked separately? Either way, thanks for working on this.

[teknium1]

Thanks for the contribution. Current main now provides the requested fallback visibility through the agent recovery path, so this PR is superseded. This is an automated hermes-sweeper review.

• agent/chat_completion_helpers.py:1652 records a one-shot notice containing the primary and fallback model/provider; agent/conversation_loop.py:5092 emits it after successful recovery.
• gateway/run.py:18359 supplies the gateway status callback used to deliver that notice to the active messaging adapter.
• agent/chat_completion_helpers.py:1638 also rewrites the prompt identity after fallback, addressing the related model-identity concern raised in the discussion.
• Commit 3fe7f6d27a005a0a75644c1d1d42cef8c8750c5f added this behavior and regression coverage in tests/run_agent/test_retry_status_buffer.py.