Skip to content

fix: use LM Studio max_context_length when model is loaded at default 64K - #45037

Closed
lunarnexus wants to merge 1 commit into
NousResearch:mainfrom
lunarnexus:fix/lmstudio-context-autodiscovery
Closed

fix: use LM Studio max_context_length when model is loaded at default 64K#45037
lunarnexus wants to merge 1 commit into
NousResearch:mainfrom
lunarnexus:fix/lmstudio-context-autodiscovery

Conversation

@lunarnexus

@lunarnexus lunarnexus commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • fall back to LM Studio max_context_length when the loaded instance context is only Hermes' 64K minimum/default
  • keep preferring the loaded instance value when it is above the minimum
  • add a regression test covering the default-64K JIT-load case

Problem

Hermes auto-discovers LM Studio context by checking loaded_instances[].config.context_length from /api/v1/models.
When Hermes JIT-loads a model without an explicit context override, LM Studio can load it at the default 64K. Hermes then reports 64K as the model context even when the model's advertised max_context_length is much higher.

Fix

If the loaded instance context is only Hermes' default/minimum load size, fall back to LM Studio's advertised max_context_length. If the loaded instance is already above that minimum, keep using the loaded runtime value.

Why this PR is separate from #36852

This fixes the LM Studio JIT-loaded autodiscovery path.
PR #36852 fixes a different layer: custom_providers per-model context_length lookup when LM Studio reports model ids as publisher/slug.

Related to #30178.

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Jun 12, 2026
@lunarnexus

Copy link
Copy Markdown
Contributor Author

Reviving this PR because it addresses a different LM Studio failure mode than #36852.

So this one is for the JIT/default-64K autodiscovery path, not the config-override path.

@lunarnexus

Copy link
Copy Markdown
Contributor Author

Closing this in favor of the consolidated PR #52188, which includes this autodiscovery/reporting fix plus the related LM Studio runtime preload fix from #48884 in one reviewable patch.

@lunarnexus lunarnexus closed this Jun 24, 2026
@lunarnexus
lunarnexus deleted the fix/lmstudio-context-autodiscovery branch July 28, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants