feat(config): per-entry reasoning_effort in fallback_model chain (#21256)

[Kyzcreig]

Problem

reasoning_effort is a flat global (agent.reasoning_effort) — there's no way to set a different thinking depth per fallback tier. So a fallback chain like:

fallback_model:
  - provider: claude-api-proxy-f1
    model: claude-opus-4-8
  - provider: openai-codex
    model: gpt-5.5

runs every tier at the same global effort. You can't, e.g., keep the primary at medium but run a last-resort Codex tier at xhigh. This is the gap described in #21256.

Solution

Add an optional reasoning_effort key to each fallback_model entry. When that tier activates, its effort overrides the global; when the primary is restored, the original effort is reverted (turn-scoped, like the rest of the fallback runtime swap).

fallback_model:
  - provider: claude-api-proxy-f1
    model: claude-opus-4-8
  - provider: openai-codex
    model: gpt-5.5
    reasoning_effort: xhigh      # this tier only

Changes

• agent/chat_completion_helpers.py — try_activate_fallback reads entry.reasoning_effort, parses via the existing parse_reasoning_effort, and overrides agent.reasoning_config. Unknown level → warn + keep current; absent/blank → unchanged. none → reasoning disabled for that tier.
• agent/agent_runtime_helpers.py — _primary_runtime snapshot now captures reasoning_config; both restore paths (restore_primary_runtime + try_recover_primary_transport) revert it, guarded for older snapshots that predate the key.
• hermes_cli/config.py — config validation warns on an invalid per-entry reasoning_effort.

Scope / non-goals

This implements the fallback_model-chain case. The issue also mentions per-model defaults via custom_providers; that's a larger surface and left for a follow-up. Levels accepted: none, minimal, low, medium, high, xhigh (same vocabulary as agent.reasoning_effort).

Tests

5 new cases in tests/run_agent/test_primary_runtime_restore.py::TestFallbackReasoningEffort:

• override applied on fallback, restored on primary
• absent key leaves effort unchanged
• invalid level ignored
• none disables reasoning on the tier

Gold-standard verified: 3 of the 5 fail without the implementation (they exercise the new path); all pass with it. Full file 36/36; config-validation 21/21; provider-fallback + credential-isolation + gemini-fallback sweeps green.

Closes #21256 (fallback_model portion).

[liuhao1024]

Positive verification — reviewed the per-entry reasoning_effort override across agent_runtime_helpers.py, chat_completion_helpers.py, hermes_cli/config.py, and tests. Clean:

1. Snapshot/restore pattern — _primary_runtime["reasoning_config"] is captured before fallback activation and restored on primary recovery. The if "reasoning_config" in rt guard correctly handles older snapshots that predate this key (avoids clobbering the live value with None).

2. Override scoping — The reasoning_effort override in try_activate_fallback fires only when the fallback entry has an explicit non-blank value. Absent or blank → primary/global effort preserved. Invalid values (not in the valid set) are logged as warnings and ignored. This is the right fail-open behavior for a config typo.

3. Config validation — validate_config_structure now checks fallback_model[N].reasoning_effort against the valid set {none, minimal, low, medium, high, xhigh} and emits a warning (not error) for invalid values. Since the field is optional, a warning is the right severity.

4. parse_reasoning_effort return — The code correctly handles None return (unrecognized level) by skipping the override and logging a warning, rather than silently disabling reasoning.

5. Test coverage — Five test cases cover: override applied, override restored on primary recovery, absent key leaves reasoning unchanged, invalid level ignored, and "none" disables reasoning. The _primary_runtime snapshot setup in test_override_restored_on_primary is particularly well-constructed — it pre-populates the snapshot to test the full round-trip.

No issues found.

[teknium1]

Thanks for the focused fallback override implementation. The feature is still absent on current main, but two paths need correction before this is safe to salvage.

Problems

• The new snapshot is only written by switch_model, while ordinary agents create _primary_runtime independently in agent/agent_init.py:2105. Since each new turn calls restore at agent/turn_context.py:174, the proposed guarded restore leaves the fallback effort active for an agent that never invoked /model. The test manually inserts the missing key, so it does not exercise normal initialization.
• The validation change only visits legacy fallback_model. The documented/current manager path writes fallback_providers (website/docs/user-guide/features/fallback-providers.md:30-43), which is read first by hermes_cli/fallback_config.py:64-70; invalid values there receive no warning.

Suggested changes

• Snapshot reasoning_config in agent/agent_init.py as well as the model-switch rewrite, then test the normal init → fallback → restore flow without mutating _primary_runtime in the test.
• Apply shared validation to both fallback keys, add coverage for fallback_providers, and document the new entry field.

Automated hermes-sweeper review.

[teknium1]

Closing as superseded by PR #64458 (merged) — with credit due: you were the earliest submitter against #21256, and your PR correctly identified both the gap and the fallback-activation site as the place a per-tier effort has to be applied. Thanks for that groundwork, @Kyzcreig.

The merged design took a different config surface: a model-keyed agent.reasoning_overrides dict instead of per-entry keys inside the fallback_model chain. Fallback activation now re-resolves reasoning through that dict, so the use case in your PR body works today:

agent:
  reasoning_effort: "medium"        # primary tier
  reasoning_overrides:
    "gpt-5.5": "xhigh"              # last-resort tier thinks harder when it activates
fallback_model:
  - provider: claude-api-proxy-f1
    model: claude-opus-4-8
  - provider: openai-codex
    model: gpt-5.5

One case the model-keyed design intentionally doesn't cover: two chain tiers using the same model with different efforts. If that's a real need for you, please open a focused issue — it would be an additive extension to the current resolution rather than a competing surface.