Skip to content

fix: preserve model.context_length on model switch - #62552

Closed
yingliang-zhang wants to merge 2 commits into
NousResearch:mainfrom
yingliang-zhang:fix/preserve-config-context-length-on-model-switch
Closed

fix: preserve model.context_length on model switch#62552
yingliang-zhang wants to merge 2 commits into
NousResearch:mainfrom
yingliang-zhang:fix/preserve-config-context-length-on-model-switch

Conversation

@yingliang-zhang

Copy link
Copy Markdown
Contributor

Summary

  • switch_model() cleared _config_context_length to None, causing get_model_context_length() to fall through to hardcoded catalog defaults instead of honoring the user's explicit model.context_length config
  • model.context_length is a global setting (endpoint capacity), not a per-model attribute — clearing it on switch is incorrect
  • Re-read model.context_length from config on every model switch so the override survives
  • When config has no model.context_length, fall back to None so per-model custom_providers resolution still works

Root Cause

agent_runtime_helpers.py line 1814 unconditionally set agent._config_context_length = None during model switch. This cleared the config-level override, causing get_model_context_length() step 0 (explicit config override) to be skipped, falling through to step 8 (hardcoded defaults).

For example, a user with model.context_length: 340000 and model.default: glm-5.2-heavy would see the context length jump to 1,048,576 (hardcoded catalog match) after any model switch, completely ignoring their config.

This differs from PR #41965 which addresses the Desktop model picker clearing context_length in web_server.py. This PR fixes the agent-side switch_model() path.

Testing

scripts/run_tests.sh tests/run_agent/test_switch_model_context.py -v --tb=short

2 tests passed, 0 failed

Updated test_switch_model_clears_previous_config_context_length to test_switch_model_preserves_config_context_length to verify config re-read behavior.

Closes #41944

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles labels Jul 11, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Companion to #41965 (which fixes the Desktop web_server.py model-picker path) — same root problem, different code path (agent-side switch_model). Related, not a duplicate. Fix for #41944.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — the runtime premise is confirmed: current main clears agent._config_context_length at agent/agent_runtime_helpers.py:1814, while get_model_context_length() honors that value before every detection/default path at agent/model_metadata.py:2051-2053.

Problems

  • agent/agent_runtime_helpers.py:1820 only preserves values already typed int or float. Startup accepts numeric strings via int(value) at agent/agent_init.py:1635-1651; therefore context_length: "131072" works initially but would be cleared after this PR's switch path.
  • The PR closes #41944 although the separate Desktop write path still removes context_length on current main at hermes_cli/web_server.py:1113; companion PR #41965 remains open.

Suggested changes

  • Match startup coercion at agent/agent_runtime_helpers.py:1820 and add a quoted-numeric regression test.
  • Reference #41944 without closing it unless #41965 lands together.

Automated hermes-sweeper review.

Comment thread agent/agent_runtime_helpers.py Outdated
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 11, 2026
switch_model() cleared _config_context_length to None, causing
get_model_context_length() to fall through to hardcoded catalog
defaults instead of honoring the user's explicit config.

model.context_length is a global setting (endpoint capacity), not a
per-model attribute. Re-read it from config on every model switch so
the override survives. When config has no model.context_length, fall
back to None so per-model custom_providers resolution still works.

Closes NousResearch#41944
@yingliang-zhang

Copy link
Copy Markdown
Contributor Author

Closing as fixed on main via a different mechanism: model switch now clears the stale per-config context_length override so the new model's window is resolved fresh (agent/agent_runtime_helpers.py_config_context_length = None on switch, including the custom-provider resolution path), which removes the stale-context-length issue this PR targeted. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Desktop] Model picker clears model.context_length when switching models in a fresh session

3 participants