fix(lmstudio): use detected context for runtime preload when no override is set - #48884
Closed
lunarnexus wants to merge 1 commit into
Closed
fix(lmstudio): use detected context for runtime preload when no override is set#48884lunarnexus wants to merge 1 commit into
lunarnexus wants to merge 1 commit into
Conversation
16 tasks
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
model.context_lengthis not explicitly setmodel.context_lengthoverrides authoritativeProblem
Hermes resolved the larger LM Studio context window correctly for budgeting/display, but
_ensure_lmstudio_runtime_loaded()still preloaded LM Studio models with:When no explicit override was set, that forced the runtime load target to Hermes' 64K minimum even when
get_model_context_length(...)resolved a larger LM Studiomax_context_length.In practice this meant:
Fix
If
model.context_lengthis explicitly set, keep using it.Otherwise, resolve the model context via
get_model_context_length(...)and use that detected value as the LM Studio preload target, floored atMINIMUM_CONTEXT_LENGTHonly as a fallback.Why this is separate from #45037
PR #45037 fixes the autodetection/reporting path when LM Studio is already loaded at the default 64K.
This PR fixes the runtime preload/load-target path that was still forcing LM Studio to load at 64K when no explicit override was set.
Tests
Related
lmstudioprovider pre-loads models manually, bypassing JIT and trampling user context config #25989