fix(agent): keep system-prompt model identity in sync across provider failover

[IamSanchoPanza]

Where to look first

The diff is 182 lines but the core logic is ~30. Suggested reading order:

1. The rewriter — rewrite_prompt_model_identity, chat_completion_helpers.py L1033: swaps the identity lines, last occurrence only, never persisted. The two design constraints in its docstring are the heart of the PR.
2. The in-flight sync — _sync_failover_system_message, conversation_loop.py L371: patches the request already mid-retry; without it the fix lands a turn too late (never, on gateway turns).
3. The trigger points — one call in try_activate_fallback after the runtime swap, one in restore_primary_runtime rewriting back, and the same 2-line sync at the seven failover branches in conversation_loop.py (repetition is deliberate — the sync needs run_conversation's locals, see inline comments).
4. Everything else is docstrings and the 9 regression tests.

What does this PR do?

Fixes the agent misreporting its own identity while a fallback provider is active. The session-stable system prompt embeds Model:/Provider: lines, but try_activate_fallback swaps the runtime without touching them — so the model that is actually answering reads (and repeats) the primary's name.

Reproduced live on a Codex plan-limit 429: every gateway turn failed over to a local gemma4:e2b-mlx, which answered "I am gpt-5.4-mini" when asked what model it was. Confusing for users, and it actively undermines trust in fallback behavior.

The fix rewrites the identity lines on the cached prompt when a fallback activates (and back when the primary is restored), and syncs the in-flight api_messages at every failover site in the conversation loop. Two deliberate design constraints:

• Only the last occurrence of each line is rewritten — earlier matches can be user content (memory snapshots, context files).
• The rewrite is never persisted to the session DB — the stored prompt keeps the primary's labels, so after restoration the prompt is byte-identical to the stored copy again and upstream prefix caches still hit. (Invalidating the cached prompt instead does not work: continuing sessions restore the stored prompt verbatim, resurrecting the stale line.)

The in-flight sync matters more than it looks: on gateway turns the primary is restored between messages, so while a usage limit lasts, every turn re-fails-over mid-turn — without the sync, the stale identity ships on every single message.

Related Issue

No existing issue — bug discovered and verified directly; happy to file one if preferred.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• agent/chat_completion_helpers.py — new rewrite_prompt_model_identity(); called from try_activate_fallback after the runtime swap
• agent/agent_runtime_helpers.py — restore_primary_runtime rewrites the lines back to the primary's
• agent/conversation_loop.py — new _sync_failover_system_message(); called at all seven failover sites so the current call block ships the corrected prompt
• tests/agent/test_failover_identity.py — 9 new tests: line swap, last-occurrence-only, byte-identical round-trip, ephemeral append, and no-op guards

How to Test

1. Configure a primary you can rate-limit (e.g. a usage-limited Codex subscription) and a local fallback in fallback_providers
2. Exhaust the primary's quota, then ask the agent "what model are you?" on any platform
3. Before: it answers with the primary's name while the fallback is generating. After: it reports the fallback model
4. pytest tests/agent/test_failover_identity.py -v for the unit coverage

Verified end-to-end against a live usage-limited primary by instrumenting _build_api_kwargs:

call 0: model=gpt-5.4-mini provider=openai-codex :: Model: gpt-5.4-mini | Provider: openai-codex   ← 429
call 1: model=gemma4:e2b-mlx provider=custom    :: Model: gemma4:e2b-mlx | Provider: custom        ← answers correctly

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature
• I've run the full suite via scripts/run_tests.sh tests/ — 30,040 passed; 33 environment-dependent failures (live local gateway/launchd state) reproduce identically on pristine origin/main, verified by running the failing subset on both branches: identical failure sets, zero failures introduced by this change
• I've added tests for my changes
• I've tested on my platform: macOS 15 (Apple Silicon)

Documentation & Housekeeping

• I've updated relevant documentation — N/A (behavior fix, no interface change)
• I've updated cli-config.yaml.example — N/A (no config keys changed)
• I've updated CONTRIBUTING.md/AGENTS.md — N/A
• I've considered cross-platform impact — pure-Python string handling, no platform-specific code
• I've updated tool descriptions/schemas — N/A

[liuhao1024]

Verification: reviewed the diff — this is a clean, well-structured fix.

Checked:

• rewrite_prompt_model_identity only touches the last occurrence of each Model:/Provider: line, preserving user content (memory snapshots, context files) that may contain earlier matches. ✅
• Round-trip property: failover → restore produces a byte-identical prompt, preserving primary's prefix cache. ✅
• _sync_failover_system_message correctly mutates api_messages[0] in place and appends ephemeral_system_prompt. ✅
• All 8 call sites in conversation_loop.py that invoke _try_activate_fallback() now also call _sync_failover_system_message(). ✅
• Regex rf"(?m)^{label}: .*$" uses multiline mode; $ matches before newline without consuming it — replacement preserves line structure. ✅
• Edge cases covered: None/empty prompt, empty model/provider values, first message not being system. ✅
• Test count: 9 tests across TestRewritePromptModelIdentity and TestSyncFailoverSystemMessage. ✅

No issues found.

[IamSanchoPanza]

Core of the fix. Two invariants here are load-bearing: (1) only the last occurrence of each line is rewritten — earlier matches can be user content from memory snapshots or context files; (2) the rewrite is never persisted to the session DB, so the stored prompt keeps the primary's labels and the prompt is byte-identical again after restore_primary_runtime — upstream prefix caches still hit. The round-trip is pinned by test_round_trip_restores_byte_identical_prompt.

[IamSanchoPanza]

Why the in-flight sync is required (not just nice-to-have): the current call block's api_messages were built before the failover, and on gateway turns the primary is restored between messages — so while a usage limit lasts, every turn re-fails-over mid-turn. Without this, the stale identity ships on every single gateway message. Simply invalidating _cached_system_prompt also doesn't work: continuing sessions restore the stored prompt verbatim from the session DB, resurrecting the stale line.

[IamSanchoPanza]

Hi maintainers — this is my first PR to the repo, so the GitHub Actions runs appear to be waiting on maintainer approval (action_required). Could someone approve the workflow run and review when you have a chance?

I’ve completed the PR checklist, kept the change focused to agent failover identity handling, added regression tests, and enabled maintainer edits on the branch. Happy to adjust anything.

[teknium1]

Merged via #49318 — your fix landed on main with your authorship preserved (commit c884ff64ea).

Your branch was ~915 commits behind main, so I reapplied the substantive work surgically onto current main rather than cherry-picking the stale branch (its diff had conflated already-merged helpers as additions). While salvaging I widened the in-flight sync to all 8 failover sites — main had grown one more (the empty-content path) since you wrote it.

Verified live against real OpenRouter failover, not just unit tests: with the fix the fallback call ships the corrected Model:/Provider: identity and the model self-reports correctly; the failover→restore round-trip is byte-identical (same SHA256 on a 13KB prompt) so the primary's prefix cache still hits; and the rewrite never touches the persisted session row. 9/9 unit tests + sibling failover/restore suites green.

Thanks for the clean writeup and the live repro — made this easy to validate.