Skip to content

fix: sync context_length on model switch and show updated limit - #17269

Closed
solink7 wants to merge 1 commit into
NousResearch:mainfrom
solink7:fix/context-length-sync-on-model-switch
Closed

fix: sync context_length on model switch and show updated limit#17269
solink7 wants to merge 1 commit into
NousResearch:mainfrom
solink7:fix/context-length-sync-on-model-switch

Conversation

@solink7

@solink7 solink7 commented Apr 29, 2026

Copy link
Copy Markdown

Problem

When switching models with /model, two things were wrong:

  1. Config not synced: /model <name> --global persisted model.default and model.provider to config.yaml, but not model.context_length. The next session would start with the old model's context window value, causing compression thresholds and token budgets to be wrong.

  2. No runtime feedback: The agent's compressor was correctly updated by switch_model() in run_agent.py, but there was no visual confirmation to the user — the startup 📊 banner was stale, and /model output only showed the resolved context length without telling the user the runtime compression settings had changed.

Root Cause

In _apply_model_switch_result() (cli.py) and _handle_model_switch_command() (gateway/run.py):

  • resolve_display_context_length() was called for display, but the result (ctx) was scoped inside a try/except and not available for the persist_global block
  • Neither path wrote model.context_length when persisting
  • Neither path printed the updated runtime context limit after the switch

Fix

CLI (cli.py)

  • Extract ctx to outer scope so it's available after the try/except block
  • When --global is specified, persist model.context_length via save_config_value()
  • After the switch, print 📊 Context limit updated line mirroring the startup banner, showing the new runtime context length and compression threshold

Gateway (gateway/run.py)

  • When --global is specified, write context_length into the model config dict before save_config()
  • Append the updated context limit line to the confirmation message sent to the messaging platform

Example

Before:

/model mimo-v2.5-pro --global
  ✓ Model switched: mimo-v2.5-pro
    Provider: xiaomi
    Context: 1,000,000 tokens
    Saved to config.yaml (--global)

Next session starts with context_length: 272000 (old model's value) ❌

After:

/model mimo-v2.5-pro --global
  ✓ Model switched: mimo-v2.5-pro
    Provider: xiaomi
    Context: 1,000,000 tokens
    Saved to config.yaml (--global)
    📊 Context limit updated: 1,000,000 tokens (compress at 50% = 500,000)

Next session starts with context_length: 1000000

Testing

  • All model switch tests pass (28/28)
  • Gateway command tests pass (58/58)

Two problems when switching models mid-session:

1. `/model <name> --global` persisted model.default and model.provider
   but NOT model.context_length — so the next session would start with
   the old model's context window in config.yaml while using a different
   model. This caused compression thresholds and token budgets to be
   wrong on restart.

2. After switching models, the runtime context_length was correctly
   updated in the compressor (run_agent.py switch_model() handles this),
   but the user had no visual confirmation — the startup banner was
   stale and there was no feedback about the new effective context limit
   and compression threshold.

Fixes:
- CLI: persist model.context_length when --global is specified
- CLI: print updated context limit line after switch, mirroring
  the startup banner so the user sees the new runtime values
- Gateway: same — write context_length to config on --global and
  append the updated context limit line to the confirmation message
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles labels Apr 29, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for identifying the persistence mismatch. The gap remains on current main: CLI persists only model.default/model.provider at cli.py:7945-7951, and the typed gateway path likewise omits context_length at gateway/slash_commands.py:1916-1923.

Problems

  • model.context_length is not neutral metadata: agent/model_metadata.py:2029-2053 resolves it as the highest-priority explicit override. Writing an auto-detected ctx there would pin future switches to that value, despite agent/agent_runtime_helpers.py:1811-1814 deliberately clearing stale overrides before re-resolving a target model.
  • The gateway target has moved: current picker and typed persistence live in gateway/slash_commands.py:1666-1673 and gateway/slash_commands.py:1916-1923; the PR changes the former gateway/run.py implementation. The TUI has a separate persistence helper at tui_gateway/server.py:2795-2812.
  • The diff adds no regression tests for persistent context semantics.

Suggested changes

  • Separate user-configured context caps from automatically discovered windows, then apply that policy consistently to CLI, gateway picker/typed paths, and TUI.
  • Add sequential-switch and pre-existing-override tests for each persistence surface.

This is an automated hermes-sweeper review.

Comment thread gateway/run.py
model_cfg["provider"] = result.target_provider
if result.base_url:
model_cfg["base_url"] = result.base_url
# Persist context_length so the next session starts with

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model.context_length is resolution step 0 (agent/model_metadata.py:2029-2053), so persisting an auto-resolved ctx here makes it a permanent override for later model switches. Please distinguish a user-selected cap from discovered metadata before writing this key.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 12, 2026
@solink7 solink7 closed this Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants