fix: sync context_length on model switch and show updated limit - #17269
Closed
solink7 wants to merge 1 commit into
Closed
fix: sync context_length on model switch and show updated limit#17269solink7 wants to merge 1 commit into
solink7 wants to merge 1 commit into
Conversation
Two problems when switching models mid-session: 1. `/model <name> --global` persisted model.default and model.provider but NOT model.context_length — so the next session would start with the old model's context window in config.yaml while using a different model. This caused compression thresholds and token budgets to be wrong on restart. 2. After switching models, the runtime context_length was correctly updated in the compressor (run_agent.py switch_model() handles this), but the user had no visual confirmation — the startup banner was stale and there was no feedback about the new effective context limit and compression threshold. Fixes: - CLI: persist model.context_length when --global is specified - CLI: print updated context limit line after switch, mirroring the startup banner so the user sees the new runtime values - Gateway: same — write context_length to config on --global and append the updated context limit line to the confirmation message
This was referenced May 12, 2026
teknium1
reviewed
Jul 12, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for identifying the persistence mismatch. The gap remains on current main: CLI persists only model.default/model.provider at cli.py:7945-7951, and the typed gateway path likewise omits context_length at gateway/slash_commands.py:1916-1923.
Problems
model.context_lengthis not neutral metadata:agent/model_metadata.py:2029-2053resolves it as the highest-priority explicit override. Writing an auto-detectedctxthere would pin future switches to that value, despiteagent/agent_runtime_helpers.py:1811-1814deliberately clearing stale overrides before re-resolving a target model.- The gateway target has moved: current picker and typed persistence live in
gateway/slash_commands.py:1666-1673andgateway/slash_commands.py:1916-1923; the PR changes the formergateway/run.pyimplementation. The TUI has a separate persistence helper attui_gateway/server.py:2795-2812. - The diff adds no regression tests for persistent context semantics.
Suggested changes
- Separate user-configured context caps from automatically discovered windows, then apply that policy consistently to CLI, gateway picker/typed paths, and TUI.
- Add sequential-switch and pre-existing-override tests for each persistence surface.
This is an automated hermes-sweeper review.
| model_cfg["provider"] = result.target_provider | ||
| if result.base_url: | ||
| model_cfg["base_url"] = result.base_url | ||
| # Persist context_length so the next session starts with |
Contributor
There was a problem hiding this comment.
model.context_length is resolution step 0 (agent/model_metadata.py:2029-2053), so persisting an auto-resolved ctx here makes it a permanent override for later model switches. Please distinguish a user-selected cap from discovered metadata before writing this key.
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When switching models with
/model, two things were wrong:Config not synced:
/model <name> --globalpersistedmodel.defaultandmodel.providertoconfig.yaml, but notmodel.context_length. The next session would start with the old model's context window value, causing compression thresholds and token budgets to be wrong.No runtime feedback: The agent's compressor was correctly updated by
switch_model()inrun_agent.py, but there was no visual confirmation to the user — the startup 📊 banner was stale, and/modeloutput only showed the resolved context length without telling the user the runtime compression settings had changed.Root Cause
In
_apply_model_switch_result()(cli.py) and_handle_model_switch_command()(gateway/run.py):resolve_display_context_length()was called for display, but the result (ctx) was scoped inside a try/except and not available for thepersist_globalblockmodel.context_lengthwhen persistingFix
CLI (
cli.py)ctxto outer scope so it's available after the try/except block--globalis specified, persistmodel.context_lengthviasave_config_value()📊 Context limit updatedline mirroring the startup banner, showing the new runtime context length and compression thresholdGateway (
gateway/run.py)--globalis specified, writecontext_lengthinto the model config dict beforesave_config()Example
Before:
Next session starts with
context_length: 272000(old model's value) ❌After:
Next session starts with
context_length: 1000000✅Testing