Skip to content

fix(agent): preserve global model.context_length through switch_model and fallback - #32413

Open
TF0rd wants to merge 1 commit into
NousResearch:mainfrom
TF0rd:fix/preserve-global-context-length
Open

fix(agent): preserve global model.context_length through switch_model and fallback#32413
TF0rd wants to merge 1 commit into
NousResearch:mainfrom
TF0rd:fix/preserve-global-context-length

Conversation

@TF0rd

@TF0rd TF0rd commented May 26, 2026

Copy link
Copy Markdown

Problem

Both switch_model() and try_activate_fallback() unconditionally set agent._config_context_length = None before passing it to get_model_context_length(). This discards the global model.context_length from config.yaml, causing the context window to fall through to the 256K probe-down default even when the user has explicitly configured a larger value (e.g. 1048576).

The clearing was intended to prevent inheriting a stale per-model custom_providers override when switching models — but it also nukes the global config setting, which applies to ALL models.

Fix

Before clearing _config_context_length, re-read the global model.context_length from config.yaml and pass it to get_model_context_length(). The per-model override is still cleared (correct behavior), but the global setting is preserved.

Also consolidates the config load in switch_model() that was previously duplicated — once for the global context_length and once for custom_providers. Both now share a single load_config() call.

Files Changed

  • agent/agent_runtime_helpers.pyswitch_model(): preserve global context_length, deduplicate config load
  • agent/chat_completion_helpers.pytry_activate_fallback(): same fix

Testing

  • Existing tests pass: test_custom_provider_context_length, test_model_switch_context_display, test_session_info (28/28)
  • Manually verified: no more "defaulting to 256,000 tokens" after model switch with model.context_length: 1048576 in config

@TF0rd
TF0rd force-pushed the fix/preserve-global-context-length branch from df6ebfa to ac51419 Compare May 26, 2026 04:52
… and fallback

Both switch_model() and try_activate_fallback() unconditionally set
agent._config_context_length = None before passing it to
get_model_context_length(). This discards the global model.context_length
from config.yaml, causing the context window to fall through to the
256K probe-down default even when the user has explicitly configured
a larger value (e.g. 1048576).

Fix: before clearing _config_context_length, re-read the global
model.context_length from config.yaml and pass it to
get_model_context_length(). The per-model override is still cleared
(so switching from model A with a custom_providers per-model override
to model B doesn't inherit A's value), but the global setting — which
applies to all models — is preserved.

Also consolidates the config load in switch_model() that was previously
duplicated: once at the top (new) for the global context_length, and
once in the compressor block (removed) for custom_providers. Both now
share a single load_config() call.
@TF0rd
TF0rd force-pushed the fix/preserve-global-context-length branch from ac51419 to 5e4e78c Compare May 26, 2026 04:53
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 26, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for identifying the lost global override. The premise is confirmed on current main: switch_model() clears _config_context_length at agent/agent_runtime_helpers.py:1848 and passes it to the resolver at :2056; fallback does the same at agent/chat_completion_helpers.py:1505 and :1626. That conflicts with the documented precedence where model.context_length is highest priority (website/docs/integrations/providers.md:1139-1149).

Problems

  • The PR adds no regression tests. Existing tests/run_agent/test_switch_model_context.py:44-57 only verifies clearing a prior override and does not configure a global model.context_length, so it cannot protect the global-vs-per-model distinction.
  • GitHub currently reports this branch as conflicting. The current implementations are at agent/agent_runtime_helpers.py:1772 and agent/chat_completion_helpers.py:1330, not the original hunk locations.

Suggested changes

  • Salvage the two resolution changes into the current helper locations and add explicit switch and fallback tests for a global config override, while retaining the no-global case that clears a stale per-model override.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants