Skip to content

fix(context): re-probe context length on session reset (#31043) - #31492

Open
Tranquil-Flow wants to merge 1 commit into
NousResearch:mainfrom
Tranquil-Flow:fix/31043-cli-new-context-length-refresh
Open

fix(context): re-probe context length on session reset (#31043)#31492
Tranquil-Flow wants to merge 1 commit into
NousResearch:mainfrom
Tranquil-Flow:fix/31043-cli-new-context-length-refresh

Conversation

@Tranquil-Flow

Copy link
Copy Markdown
Contributor

What

CLI /new (and /reset) do not refresh context_compressor.context_length after provider config changes. After adjusting LM Studio's context length and reloading the model, /new reuses the stale cached value — the agent operates with the wrong context budget.

Fix

ContextCompressor.on_session_reset() now re-probes context length from the provider. For LM Studio specifically, it invalidates the short-lived endpoint metadata cache before probing, so /new reflects a just-reloaded model instead of a 5-minute-old /api/v1/models response.

  • agent/model_metadata.py — +9: invalidate_endpoint_model_metadata_cache(base_url) public helper
  • agent/context_compressor.py — +26: on_session_reset() re-probes + LM Studio cache invalidation

Why over #31067

A competing PR (#31067) identifies the same endpoint-cache issue, but:

  • Patches cli.py only (not the reset abstraction used by /new and /reset)
  • Imports private model_metadata cache internals directly
  • Recreates ContextCompressor instead of reusing the existing reset hook
  • Has no committed regression tests

This fix is narrower in the runtime reset abstraction and better covered.

Tests

python3 -m pytest tests/agent/test_context_compressor.py -q -o addopts=
87 passed in 25s

4 regression tests: re-probe on reset, no-op when unchanged, probe failure retains stale value, LM Studio cache invalidation.

Closes #31043

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard labels May 24, 2026
@Tranquil-Flow
Tranquil-Flow force-pushed the fix/31043-cli-new-context-length-refresh branch 2 times, most recently from 7a62780 to 5e152aa Compare May 25, 2026 11:02

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting the existing compressor lifecycle instead of replacing the context engine. The premise remains valid on current main: CLI /new retains the agent and calls reset_session_state() (cli.py:7073-7076), whose lifecycle calls on_session_reset() (run_agent.py:647-651); the built-in reset currently does not re-resolve context_length (agent/context_compressor.py:726-747).

Problems

  • The proposed resolver call omits config_context_length. Current initialization stores the resolved override (agent/agent_init.py:1747-1749), and resolver step 0 makes that value authoritative (agent/model_metadata.py:2051-2053). A reset could overwrite an explicit configured budget.
  • The proposed outer exception handler does not preserve the old value for normal probe-down behavior: failed custom/local probes fall through to catalog/default resolution (agent/model_metadata.py:2212-2258, 2404-2426).

Suggested changes

  • Rework the reset hook on current main to preserve configured overrides and dynamically refresh only auto-detected values.
  • Add coverage for explicit override preservation and provider-unavailable fallback, alongside the changed LM Studio value case.

Automated hermes-sweeper review.

# so /new reflects a just-reloaded model instead of a 5-minute-old
# /api/v1/models response.
if (self.provider or "").lower() == "lmstudio" and self.base_url:
invalidate_endpoint_model_metadata_cache(self.base_url)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_model_context_length() gives config_context_length precedence, but this reset call omits it. Preserve the resolved override used at agent initialization; otherwise /new can replace an explicit model.context_length or custom-provider override with endpoint metadata.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
@Tranquil-Flow
Tranquil-Flow force-pushed the fix/31043-cli-new-context-length-refresh branch from fd12768 to 0b502f1 Compare July 13, 2026 16:31
@teknium1 teknium1 added the area/sessions Session lifecycle, resume, persistence, history label Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions Session lifecycle, resume, persistence, history comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: CLI /new does not refresh context_compressor.context_length after provider config changes

3 participants