fix(agent): fail closed when Codex account-model entitlement 400s exhaust the chain

[liuhao1024]

What does this PR do?

When both the primary model and every fallback_providers entry name an openai-codex model the ChatGPT-account credential is not entitled to, Hermes oscillates forever: the primary call returns the account-specific 400, the fallback activates, the fallback returns the same 400, and at the start of the next turn restore_primary_runtime switches back to the primary and announces "✅ Primary model restored" — a recovery that was never probed. Users see the two warnings alternate indefinitely with zero delivered answers while the service reports active (#106475).

This PR fails closed instead. On the non-retryable client-error path, a Codex ChatGPT-account entitlement 400 (The '<model>' model is not supported when using Codex with a ChatGPT account.) records the rejected (provider, model) pair as dead for the session. The fallback walk then skips rejected entries, and restore_primary_runtime refuses to switch back to a rejected primary — the session stays put and surfaces the terminal entitlement error once, with a user-visible pointer to /model, instead of looping.

The marker is deliberately scoped to the single-credential case: when the provider has a multi-credential pool, another account may be entitled to the same slug, so the decision is left to credential rotation (#71970 / #71973). Rejected pairs live only in memory and are cleared on restart, matching the session-scoped _unavailable_fallback_keys semantics.

Related Issue

Fixes #106475

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• agent/chat_completion_helpers.py — add _mark_entitlement_rejected_model (detect the normalized Codex entitlement 400 string, record the pair once, emit one user-visible notice) and _is_entitlement_rejected (matches the configured or normalized slug form); _should_skip_fallback_candidate now skips rejected entries.
• agent/turn_api_error.py — call the marker on the non-retryable client-error branch, before the fallback walk.
• agent/agent_runtime_helpers.py — restore_primary_runtime returns False (stays on the fallback, no restore notice) when the primary's slug was rejected as unentitled.
• tests/run_agent/test_entitlement_fail_closed.py — new: marker fires only on the Codex entitlement 400 (not 429/500/other 400s/403), multi-credential pools are left to rotation, rejected entries are skipped in both slug forms, restore is gated for a rejected primary and still restores normally for unrelated models.

How to Test

1. pytest tests/run_agent/test_entitlement_fail_closed.py -q — 8 passed.
2. pytest tests/run_agent/test_provider_fallback.py tests/run_agent/test_primary_runtime_restore.py tests/run_agent/test_24996_fallback_exhaustion_cooldown.py tests/run_agent/test_conversation_fallback_state.py tests/run_agent/test_fallback_credential_isolation.py tests/run_agent/test_fallback_api_mode_preservation.py tests/run_agent/test_nous_429_fallback_reentry.py tests/run_agent/test_nous_fallback_unavailable.py tests/run_agent/test_init_fallback_on_exhausted_pool.py tests/run_agent/test_32646_fallback_429_after_timeout.py tests/run_agent/test_image_rejection_fallback.py -q — all pass (no fallback/restore regressions).
3. Manual reproduction of the issue scenario: configure a primary and a fallback slug that the account is not entitled to, send a message. Observed result: first turn marks both slugs and fails with the terminal error naming the account-entitlement cause; a second message no longer emits "Primary model restored" and does not switch back to the rejected primary.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 15 (arm64)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

[Enough1122]

AI code review — automated review for reference; please use your judgment.

AI review — fix(agent): entitlement 400 fails closed

Summary: Single-credential Codex ChatGPT-account 400s ("model is not supported when using codex with a chatgpt account") now record (provider, model) as session-dead: fallback walk skips the slug (both raw and normalized forms) and restore_primary_runtime won't flip back with an unverified "restored" notice.

Strengths

• agent/chat_completion_helpers.py:32-87 — narrow trigger (400 + exact marker + single-credential pool); user-facing _buffer_status notice with remediation (/model); normalized-slug check covers openai/gpt-... vs gpt-... skew.
• agent/turn_api_error.py:110-113 — marking happens before the fallback walk, so the walk sees it immediately.
• agent/agent_runtime_helpers.py:10-15 — restore gate prevents oscillation; tests pin no-switch-back + no spurious emission, and normal restore for unrejected primaries.

Non-blocking

• Marker is an English server string match; a reworded upstream message silently disables the fail-closed path (fail-open to old oscillation). Consider also matching a stable error code if the API provides one, keeping the string as fallback.
• Mark-then-Copilot-self-heal ordering: the self-heal retry on the same (dead) slug will burn one call before the walk skips it. Consider skipping same-provider self-heal once marked.
• State is session-scoped (until restart) by design — right call; the notice tells the user how to escape.

Verdict: Non-blocking. Correct fail-closed direction.

[liuhao1024]

Thanks for the review. On the non-blocking notes: the 400 body doesn't expose a machine-readable code today, so the narrow string+trigger match is the tightest available — if a stable code surfaces it can replace the string. The same-provider self-heal skip is a good follow-up candidate once the fail-closed path has some field time.

[teknium1]

Incorporated into #106549, merged to main as 6c89cb032e. Cherry-picked with authorship preserved; the marker/skip logic moved off the chat_completion_helpers facade into agent/fallback_cooldown.py (which already owns the shared cooldown state) and tests trimmed 8→2.

Closed as part of the 2026-09-09 issue-queue salvage batch; if any hunk of yours that mattered was dropped, say so here and it gets its own follow-up.