Skip to content

fix(config): re-read model.context_length from config after /model switch (#40979) - #41270

Open
rodboev wants to merge 4 commits into
NousResearch:mainfrom
rodboev:pr/config-context-length-model-switch
Open

fix(config): re-read model.context_length from config after /model switch (#40979)#41270
rodboev wants to merge 4 commits into
NousResearch:mainfrom
rodboev:pr/config-context-length-model-switch

Conversation

@rodboev

@rodboev rodboev commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Summary

When a user sets model.context_length in config.yaml, the override is read at session start in agent_init.py and stored as agent._config_context_length. This value controls the compression threshold, how aggressively Hermes compresses history to stay within the context window.

When /model is used to switch models mid-session, switch_model() in agent/agent_runtime_helpers.py clears _config_context_length to None, correctly discarding any stale value from the previous model. It then re-reads custom_providers per-model context lengths from config, but did not re-read the global model.context_length key. The override was silently lost: the session continued with auto-detected context lengths for all subsequent turns, ignoring the user's explicit setting.

This PR extends the config re-read block inside switch_model() to also extract model.context_length and re-apply it before calling get_model_context_length(). The follow-up fixes keep that same context state consistent across the runtime lifecycle: /model persists the refreshed _config_context_length and live _custom_providers into _primary_runtime, initialization snapshots those fields before any /model switch can happen, and restore_primary_runtime() restores both fields when the agent exits fallback.

Changes

  • agent/agent_runtime_helpers.py: re-read model.context_length during switch_model(), persist the refreshed _config_context_length and _custom_providers into _primary_runtime, and restore those fields in restore_primary_runtime()
  • agent/agent_init.py: include the init-time config_context_length and a deep copy of custom_providers in the primary runtime snapshot
  • tests/run_agent/test_switch_model_context.py: cover real temporary-HERMES_HOME config loading for the global override and refreshed custom-provider persistence after /model
  • tests/run_agent/test_primary_runtime_restore.py: cover init snapshot contents, copy isolation, fallback before any /model switch, restore success, fallback reset, and restored context state

Validation

Scenario Before After
model.context_length: 65536 set, no /model switch honored honored and included in the initial primary snapshot
model.context_length: 65536 set, /model new-model called lost, reverts to auto-detect re-read from config.yaml, passed to get_model_context_length, and persisted into _primary_runtime
model.context_length not set, /model called None passed None passed, unchanged
custom_providers per-model context_length, /model called honored during the immediate switch honored and persisted into the primary runtime snapshot
fallback before any /model switch restore could read missing snapshot keys initial snapshot restores _config_context_length and a copied _custom_providers list
/model switch followed by fallback/restore refreshed state lost on restore refreshed _config_context_length and _custom_providers restored correctly

Test plan

  • pytest tests/run_agent/test_switch_model_context.py tests/run_agent/test_primary_runtime_restore.py -v --timeout=0 — 41 focused tests pass

Why it's low-risk

The change stays scoped to /model, initialization, and runtime restore paths, and it only preserves state the branch already computes. Sessions without model.context_length set keep None; sessions without live custom providers snapshot and restore None.

Upstream

Closes #40979.
Reported by @bonaluo.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary (PR #41270)

Verdict: Approved

Looks Good

  • Fixes issue #40979: model.context_length from config.yaml is re-read after /model switch and persisted into primary runtime state
  • Handles both global model.context_length and per-model custom_providers context_length
  • restore_primary_runtime now restores _config_context_length and _custom_providers (deep copy)
  • Tests cover: restore after fallback, re-read on switch, custom_providers persistence

Reviewed by Hermes Agent

@daimon-nous daimon-nous Bot added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles P2 Medium — degraded but workaround exists labels Jun 7, 2026
@rodboev
rodboev force-pushed the pr/config-context-length-model-switch branch from 9c5daa7 to 2c4ab7a Compare June 28, 2026 19:56
@rodboev
rodboev force-pushed the pr/config-context-length-model-switch branch from aba4990 to ba94f5e Compare July 7, 2026 05:20

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the global override through the /model path; current main still clears _config_context_length at agent/agent_runtime_helpers.py:1848 and then passes the unset value at :2056.

Problems

  • The new restore assignments need corresponding initial snapshot entries. agent/agent_init.py:2105-2123 does not store config_context_length or custom_providers. Since fallback activation leaves _custom_providers intact (agent/chat_completion_helpers.py:1502-1636), restoring rt.get("custom_providers") after a fallback before any /model would replace it with None.
  • The added restore test pre-populates those keys, so it misses that initial-session path. The config test also mocks load_config; a temporary-HERMES_HOME integration test would cover the required propagation path.

Suggested changes

  • Snapshot both fields during initialization and add a fallback-without-/model preservation test.
  • Add a real config-loading regression test for the global override.

Automated hermes-sweeper review.

Comment thread agent/agent_runtime_helpers.py
@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

model.context_length global config override is not re-read after /model switch

3 participants