fix(config): preserve model.context_length on same-model re-pick - #59191
fix(config): preserve model.context_length on same-model re-pick#59191wesleysimplicio wants to merge 1 commit into
Conversation
…OST /api/model/set The _apply_main_model_assignment() function unconditionally drops context_length on every call, even when the model and provider are unchanged. For self-hosted OpenAI-compatible endpoints (llama.cpp, etc.), auto-detection is unreliable and the silently-lost override leaves users with a wrong context window and hard mid-session failures. Capture prev_model before overwriting it and only pop context_length when the model or provider actually changed, mirroring the same-provider preserve behavior already used for base_url and api_key. Closes NousResearch#59050
AmirF194
left a comment
There was a problem hiding this comment.
The fix itself is right. The unconditional model_cfg.pop("context_length", None) was wiping a user's manual context override on every re-pick, and gating it on new_provider != prev_provider or model != prev_model correctly preserves it on a same-model/same-provider re-pick while still clearing it whenever the model or provider actually changes. I checked the provider-change-with-same-model-name case too, and it drops as it should, and the change does not disturb the _denormalize_config_from_web path, which sets context_length explicitly after this helper runs.
Two things hold me back from a plain approval. First, this looks like a duplicate of #59125, which was opened a bit earlier, covers the same function with the same condition, and additionally ships regression tests and updates the docstring. Given the triage note pointing there, it probably makes sense to fold into that one. Second, if this PR moves forward instead, it needs a test: _apply_main_model_assignment's contract is pinned by test_apply_main_model_assignment_base_url_and_context_reconcile, and changing the behavior without extending that test leaves the new preserve path unguarded against a future regression of #59050. Worth fixing the docstring too, which still says the context_length override "is always dropped" and now contradicts the code.
I verified the fix is load-bearing in a clean Python 3.11 container matching CI with a scratch test: same-model re-pick preserves and different-model/different-provider clears, and reverting the source makes the two preserve cases fail. (The one unrelated test_get_status failure I saw also fails on pristine main, so that is environmental, not from this change.)
Minor: model is compared raw against a stripped prev_model. It is fine in the normalized request path, but stripping both sides (as #59125 does) removes the asymmetry.
|
Fechando como duplicada de #59125, conforme review/triagem. A versão canônica já cobre o mesmo ajuste com testes/docstring. |
|
Reopening — this was closed in error by the automated stale-PR check, which only inspected GitHub's |
|
Re-closing: this was mistakenly reopened citing the stale-close automation bug (that bug affects #59194/#59189, which is a separate issue), but this PR's actual close reason was different and still valid — it is a duplicate of #59125 (opened earlier, same fix for #59050, and #59125 additionally ships regression tests and a docstring update per @AmirF194's review). |
|
Thanks for the focused fix. The premise is confirmed on current main: Problems
Suggested changes
This is an automated hermes-sweeper review. |
|
Re-closing (again) — this PR is a duplicate of #59125 (opened earlier for the same #59050 root cause, and #59125 additionally ships regression tests + a docstring update per @AmirF194's and teknium1's reviews). Already confirmed in my 2026-07-15T06:34Z close comment; something reopened it again without new information. #59125 remains open and is the canonical fix to review/merge. Leaving closed. |
Closes #59050
_apply_main_model_assignment()unconditionally dropscontext_lengthon every call, even when the model and provider are unchanged. For
self-hosted OpenAI-compatible endpoints, auto-detection is unreliable
and the lost override leaves users with a wrong context window.
This captures
prev_modelbefore overwriting it and only popscontext_lengthwhen the model or provider actually changed, mirroringthe same-provider preserve behavior already used for
base_urlandapi_keyin the same function.Changes:
hermes_cli/web_server.py:_apply_main_model_assignment()