fix(agent): keep smart routing from compressing session history

[Falicitas]

Closes #7798

Problem

Smart model routing rebuilds AIAgent on every route-signature change (cli.py:7081-7083), which creates a new ContextCompressor bound to the cheap model's context_length (run_agent.py:1314-1326). A trivial turn like "hi" routed to a small cheap model then triggers preflight compression against the cheap threshold and silently rewrites session history that was sized for the primary model.

Contract this PR establishes

Smart routing either uses the cheap model when it can safely handle the current request, or falls back to the primary. It never triggers destructive session compression.

Two coordinated changes

1. skip_preflight_compression kwarg on run_conversation() (run_agent.py).
CLI sets it True at the chat / single-query / /btw call sites when the turn carries a smart-routing label. Preflight is bypassed on those turns.

2. Refuse-to-route on history overflow (smart_model_routing.resolve_turn_route).
New kwargs current_request_tokens and max_history_ratio. CLI computes the request size via estimate_request_tokens_rough and passes the active compressor's threshold_percent (default 0.50) as the ratio. When est > cheap_ctx × ratio, smart routing returns the primary route.

The default max_history_ratio = 0.50 mirrors ContextCompressor.threshold_percent so tuning compression.threshold in config.yaml tunes both simultaneously — both settings express "how much pressure on the active model we tolerate before evasive action". Leaving the upper half of the cheap context free also gives room for typical read_file / web_extract / search_files outputs within the turn. A decoupled smart_routing.max_history_ratio config field remains a non-breaking future extension since resolve_turn_route already accepts it as a kwarg.

Behavior

Example: primary = claude-sonnet-4.6 (1M), ratio = 0.50.

Cheap model	History	Before	After
gemini-2.5-flash (1M)	any	safe	safe
gpt-4o-mini (128K)	50K	safe	safe
gpt-4o-mini (128K)	80K	preflight → session compressed	refuse → stay on primary
gpt-4o-mini (128K)	150K	preflight + in-loop → session compressed	refuse → stay on primary
claude-haiku-4.5 (200K)	120K	preflight → session compressed	refuse → stay on primary
any	primary also over threshold	primary compresses	primary compresses (unchanged)

If the cheap model genuinely can't accept a permitted request, existing in-loop compression on API error (413 / 400 context_length_exceeded) still catches it — no safety regression.

Covered CLI paths: chat() main REPL, single-query (hermes -- "..."), /btw. /background is already safe (no history passed to the fresh bg agent).

How to test

model: { default: anthropic/claude-sonnet-4.6, provider: openrouter }
smart_model_routing:
  enabled: true
  cheap_model: { provider: openrouter, model: openai/gpt-4o-mini }

1. Accumulate ~80K tokens in a hermes chat session
2. Send a trivial message ("hi")
3. Before: preflight fires on gpt-4o-mini, self.conversation_history gets replaced with a summary, session lineage splits in SQLite
4. After: smart routing refuses (80K > 128K × 0.50), stays on claude-sonnet-4.6, history intact

Automated tests:

• tests/run_agent/test_413_compression.py::TestPreflightCompression: test_skip_preflight_compression_flag, test_skip_preflight_false_still_compresses
• tests/agent/test_smart_model_routing.py: 5 new unit tests covering refuse/allow/backward-compat/custom ratio/unknown cheap context
• Locally: 462 related tests pass on af9caec4 (routing + compression + CLI suites). The 6 pre-existing failures in test_auxiliary_client.py / test_credential_pool.py / test_memory_user_id.py are unrelated and also fail on main.

Platforms: macOS (Darwin 24.6.0), Python 3.11.15.

Scope and known limitations

In scope: CLI call sites (chat / single-query / /btw). Gateway and batch_runner use the same smart-routing code path but are left for follow-up.

Known paths this PR does not cover:

• Mid-turn tool output explosion > 50% of cheap context. If a single tool result pushes the next API call past the cheap model's real limit, in-loop compression still fires. Fixing this would need a per-call override architecture (reusing _try_activate_fallback / _restore_primary_runtime).
• /model command (switch_model at run_agent.py:1474) has the same compressor-rebind pattern, but comments at 1484-1486 explicitly document it as a "permanent switch", so compression on downsize is likely intentional. Happy to discuss in a separate issue.

Related

• [Bug]: smart_model_routing triggers preflight compression against the cheap model's threshold #7798 — bug report this closes
• [Bug]: CLI chat status‑bar shows static default model even when smart routing uses a cheap model #4568 — UI-layer symptom of the same "smart routing mutates agent state" design
• fix: preserve config_context_length when switching models #6844 / fd3e855d — related /model switch fix (different code path)
• fix: clear conversation_history after mid-loop compression to prevent empty sessions #7001 / 871313ae — clears conversation_history after mid-loop compression (partial mitigation only)

[Falicitas]

Ready for review. Scope is intentionally minimal (fix A + A' on CLI only). Happy to iterate toward fix B (threshold override via the ContextEngine ABC) or fix C (per-call runtime override reusing _try_activate_fallback / _restore_primary_runtime) if a deeper refactor is preferred — or to extend the current approach to gateway / batch_runner in the same PR if that's the cleaner call.

[Falicitas]

Status update — the base code this PR modifies (agent/smart_model_routing.py, the rebuild path in _resolve_turn_agent_config, the /btw command) was removed wholesale by #12732 (merged 2026-04-20). That's why this PR is now Conflicting and can't be straight-forwardly rebased.

Three open PRs are reintroducing smart routing in different shapes:

PR	Approach	Race condition from #7798?
#18930	route-signature swap → _init_agent rebuilds AIAgent	reproduces — same path #7798/#7799 patches
#20327	inline runtime swap, reuses _try_activate_fallback / _restore_primary_runtime + context_compressor.update_model()	avoided by design (matches issue #7798's Proposed Fix C)
#20678	inline runtime swap, same shape as #20327	avoided by design

Three ways forward, happy with any:

1. Close this PR, I add a review comment on feat: add smart model routing #18930 pointing out the ContextCompressor rebuild risk and how feat(agent): smart model routing (pre-turn task-difficulty router) #20327/feat(agent): implement minimal conservative smart-2-tier model routing #20678 sidestep it. Fix design stays available in issue [Bug]: smart_model_routing triggers preflight compression against the cheap model's threshold #7798 for whichever direction lands.
2. Mark this Draft until a smart-routing reintroduction lands on main, then re-port if needed.
3. The maintainers fold the fix into whichever reintroduction PR is preferred — the algorithm is in issue [Bug]: smart_model_routing triggers preflight compression against the cheap model's threshold #7798's Proposed Fix sections A and A'.

Let me know the preferred path.

[teknium1]

Thanks for this careful analysis and well-structured PR, @Falicitas! Closing as the underlying feature has been removed.

This automated hermes-sweeper review found that the smart_model_routing feature this PR patches was entirely deleted from main in commit 424e9f36b (PR #12732, merged 2026-04-19) — 8 days after this PR was opened. The module agent/smart_model_routing.py, all related config keys (smart_model_routing.*, cheap_model), and the CLI/gateway orchestration scaffolding no longer exist on main.

• agent/smart_model_routing.py — deleted in 424e9f36b
• skip_preflight_compression kwarg — not present anywhere on current main
• All three changed non-test files (agent/smart_model_routing.py, cli.py routing hooks, run_agent.py kwarg) target code that was removed wholesale

The bug you identified (#7798) was real and the fix design was sound. If smart-routing (or a successor per-turn routing mechanism) is ever reintroduced, the skip_preflight_compression + overflow-guard pattern documented here would be a good reference.