Skip to content

fix(lmstudio): use detected context for runtime preload when no override is set - #48884

Closed
lunarnexus wants to merge 1 commit into
NousResearch:mainfrom
lunarnexus:fix/lmstudio-runtime-preload-context-clean
Closed

fix(lmstudio): use detected context for runtime preload when no override is set#48884
lunarnexus wants to merge 1 commit into
NousResearch:mainfrom
lunarnexus:fix/lmstudio-runtime-preload-context-clean

Conversation

@lunarnexus

Copy link
Copy Markdown
Contributor

Summary

  • use the detected LM Studio context window for runtime preload when model.context_length is not explicitly set
  • keep explicit model.context_length overrides authoritative
  • add regression tests covering runtime preload target selection

Problem

Hermes resolved the larger LM Studio context window correctly for budgeting/display, but _ensure_lmstudio_runtime_loaded() still preloaded LM Studio models with:

target_ctx = max(config_context_length or 0, MINIMUM_CONTEXT_LENGTH)

When no explicit override was set, that forced the runtime load target to Hermes' 64K minimum even when get_model_context_length(...) resolved a larger LM Studio max_context_length.

In practice this meant:

  • autodetection could report the correct larger context
  • but LM Studio was still actually loaded at 64K

Fix

If model.context_length is explicitly set, keep using it.

Otherwise, resolve the model context via get_model_context_length(...) and use that detected value as the LM Studio preload target, floored at MINIMUM_CONTEXT_LENGTH only as a fallback.

Why this is separate from #45037

PR #45037 fixes the autodetection/reporting path when LM Studio is already loaded at the default 64K.

This PR fixes the runtime preload/load-target path that was still forcing LM Studio to load at 64K when no explicit override was set.

Tests

python -m pytest tests/agent/test_lmstudio_runtime_load.py -q -o addopts=''
python -m pytest tests/agent/test_model_metadata_local_ctx.py -q -o addopts=''

Related

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/ollama Ollama / local models P3 Low — cosmetic, nice to have labels Jun 19, 2026
@lunarnexus

Copy link
Copy Markdown
Contributor Author

Closing this in favor of the consolidated PR #52188, which includes this runtime preload fix plus the related LM Studio autodiscovery/reporting fix from #45037 in one reviewable patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/ollama Ollama / local models type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: lmstudio provider pre-loads models manually, bypassing JIT and trampling user context config

2 participants