fix(context): re-probe context length on session reset (#31043) - #31492
fix(context): re-probe context length on session reset (#31043)#31492Tranquil-Flow wants to merge 1 commit into
Conversation
7a62780 to
5e152aa
Compare
teknium1
left a comment
There was a problem hiding this comment.
Thanks for targeting the existing compressor lifecycle instead of replacing the context engine. The premise remains valid on current main: CLI /new retains the agent and calls reset_session_state() (cli.py:7073-7076), whose lifecycle calls on_session_reset() (run_agent.py:647-651); the built-in reset currently does not re-resolve context_length (agent/context_compressor.py:726-747).
Problems
- The proposed resolver call omits
config_context_length. Current initialization stores the resolved override (agent/agent_init.py:1747-1749), and resolver step 0 makes that value authoritative (agent/model_metadata.py:2051-2053). A reset could overwrite an explicit configured budget. - The proposed outer exception handler does not preserve the old value for normal probe-down behavior: failed custom/local probes fall through to catalog/default resolution (
agent/model_metadata.py:2212-2258,2404-2426).
Suggested changes
- Rework the reset hook on current main to preserve configured overrides and dynamically refresh only auto-detected values.
- Add coverage for explicit override preservation and provider-unavailable fallback, alongside the changed LM Studio value case.
Automated hermes-sweeper review.
| # so /new reflects a just-reloaded model instead of a 5-minute-old | ||
| # /api/v1/models response. | ||
| if (self.provider or "").lower() == "lmstudio" and self.base_url: | ||
| invalidate_endpoint_model_metadata_cache(self.base_url) |
There was a problem hiding this comment.
get_model_context_length() gives config_context_length precedence, but this reset call omits it. Preserve the resolved override used at agent initialization; otherwise /new can replace an explicit model.context_length or custom-provider override with endpoint metadata.
fd12768 to
0b502f1
Compare
What
CLI
/new(and/reset) do not refreshcontext_compressor.context_lengthafter provider config changes. After adjusting LM Studio's context length and reloading the model,/newreuses the stale cached value — the agent operates with the wrong context budget.Fix
ContextCompressor.on_session_reset()now re-probes context length from the provider. For LM Studio specifically, it invalidates the short-lived endpoint metadata cache before probing, so/newreflects a just-reloaded model instead of a 5-minute-old/api/v1/modelsresponse.agent/model_metadata.py— +9:invalidate_endpoint_model_metadata_cache(base_url)public helperagent/context_compressor.py— +26:on_session_reset()re-probes + LM Studio cache invalidationWhy over #31067
A competing PR (#31067) identifies the same endpoint-cache issue, but:
cli.pyonly (not the reset abstraction used by/newand/reset)ContextCompressorinstead of reusing the existing reset hookThis fix is narrower in the runtime reset abstraction and better covered.
Tests
4 regression tests: re-probe on reset, no-op when unchanged, probe failure retains stale value, LM Studio cache invalidation.
Closes #31043