fix(custom_providers): resolve per-model context_length for publisher/slug model ids by banditburai · Pull Request #36852 · NousResearch/hermes-agent

banditburai · 2026-06-01T14:54:24Z

Summary

LM Studio reports model ids as publisher/slug (e.g. nvidia/nemotron-3-nano-4b), but custom_providers[].models: is keyed by the bare slug (nemotron-3-nano-4b). get_custom_provider_context_length matched only the exact id, so per-model context_length was missed and the model fell back to the 256K default — mis-sizing compression and the @context budget (#30178, regressed in 0.14.0). Fix: try the exact key, then the bare slug, in that one resolver, and route the remaining inline lookups through it.

Change

get_custom_provider_context_length (hermes_cli/config.py:3722-3748): after an exact models[model] miss on an id containing /, retry models[model.rsplit("/", 1)[1]]. Case-sensitive; exact match wins; ctx > 0 guard; debug logs tagged exact/slug plus a miss log.
Session hygiene (gateway/run.py:8957-8961): replace the inline lookup loop (24 lines) with a call to the resolver.
/info per-model (gateway/run.py:9766-9772): call the resolver; the match now requires base_url (the prior loop matched on model id alone).
/info legacy entry-level (gateway/run.py:9743-9750): retained for the top-level cp.model + context_length schema.
@context budget (gateway/run.py:8550-8568): pass custom_providers to get_model_context_length.
Invalid-value warning (agent/agent_init.py:1362-1365): apply the same slug fallback so a non-integer context_length (e.g. "256K") on a slug-keyed model still warns.
Unchanged: compression code, ContextCompressor.__init__, and the get_model_context_length step-2 256K short-circuit.

Resolution behavior

runtime id	matching config key	result
`pub/m`	`pub/m` (exact)	exact value
`pub/m`	`m` (bare slug)	slug value
`pub/m`	only `Pub/m` / wrong case	miss → probe/default
`m` (no slash)	`pub/m`	miss (no reverse match)
`pub/m@q4`	`m`	miss (quant suffix not stripped)

Testing

uv run --extra dev pytest -p no:randomly tests/hermes_cli/test_custom_provider_context_length.py tests/gateway/test_session_hygiene.py tests/gateway/test_session_info.py tests/gateway/test_context_expansion_custom_provider.py tests/run_agent/test_invalid_context_length_warning.py tests/run_agent/test_compression_feasibility.py → 77 passed. ruff check clean (the tree is not ruff format-managed).

test_custom_provider_context_length.py (19): exact-wins, slug fallback, multi-slash, case sensitivity, quant-suffix and reverse-direction unsupported, trailing slash, non-positive, empty inputs.
test_session_hygiene.py: slug-keyed context_length reaches get_model_context_length through _handle_message.
test_session_info.py: /info shows 1.0M and (config).
test_context_expansion_custom_provider.py: the @context budget uses the slug-resolved 1M through _prepare_inbound_message_text.
test_invalid_context_length_warning.py: "256K" on a slug-keyed model warns through _build_agent.
test_compression_feasibility.py::test_feasibility_resolves_slug_keyed_custom_provider_context: with get_model_context_length unmocked and a 500K threshold (above the 256K fallback tier), the feasibility check sees the slug-resolved 1M and does not lower the threshold; it fails if the slug fallback is removed.

Verification limit: exercised through config and the resolver chain, not against a live LM Studio endpoint.

Note

The /info per-model lookup now requires a base_url match. For a single provider the result is identical; for multiple providers sharing a model id, a non-matching provider's context_length is no longer shown. The displayed number was already correct via get_model_context_length step-0b; this affects the (detected)→(config) source label.

refactor(gateway): use get_custom_provider_context_length() helper + pass custom_providers to @context path #18844 proposed the @context threading and the resolver-based hygiene lookup; both are implemented here (gateway/run.py:8550-8568, :8957-8961).
Bug: compression runtime context_length overrides are not applied consistently #8785, auxiliary compression model does not read context_length from providers config #13807, Compression feasibility check ignores custom provider context_length for auxiliary compression #19539, _check_compression_model_feasibility doesn't pass custom_providers to get_model_context_length #20608, [Bug]: Auxiliary compression model does not inherit context_length from custom_providers #21919, [Bug]: Auxiliary compression model does not inherit context_length from custom_providers #21947, compression feasibility check ignores custom_providers context_length, causes false 256K warning #23779, [Bug]: auxiliary.* context_length ignores custom_providers[].models[].context_length #26548 report bare-keyed (non-slug) configs; those paths thread custom_providers into get_model_context_length as of 7becb19ea and b5bcffe16. [Bug]: Auxiliary compression model does not inherit context_length from custom_providers #21919 and [Bug]: Auxiliary compression model does not inherit context_length from custom_providers #21947 describe the same case.
get_model_context_length step 2 short-circuits to 128K default; resolve_display_context_length skips config_context_length #15563: the resolve_display_context_length half (hermes_cli/model_switch.py) is handled by 0dd373ec4; the get_model_context_length step-2 short-circuit is unchanged here.
fix(agent): propagate custom provider context_length to compression feasibility check #13540 and fix(agent): forward custom_providers to context-length probe on fallback activation #25554 proposed the custom_providers threading now in 7becb19ea/b5bcffe16. fix(compression): pass custom providers to context resolver #26843 proposed passing custom_providers to ContextCompressor.__init__; that constructor uses an already-resolved config_context_length (step-0 returns before step-0b), so the window is resolved before construction.

Closes #30178
Refs #18844

LM Studio reports model ids as publisher/slug but users key custom_providers models: entries with the bare slug. Exact-match-first, then bare-slug fallback (mirrors _model_id_matches). Adds debug logging of resolved-vs-miss. Closes NousResearch#30178

Replaces the inline custom_providers models.get(model) loop (which missed on publisher/slug ids) with get_custom_provider_context_length, picking up the slug fallback. Refs NousResearch#30178

Replaces the inline per-model lookup in _format_session_info with get_custom_provider_context_length (slug fallback) and fixes a secondary bug: the old loop never matched on base_url. The displayed number was already correct via step-0b; this corrects the source label from (detected) to (config) and drops the non-standard bare-int models entry schema. Refs NousResearch#30178

The invalid context_length warning loop missed on publisher/slug ids, silently skipping the warning for slug-keyed models. Mirror the helper's slug fallback. Refs NousResearch#30178

@context

The @context expansion ignored custom_providers per-model context_length. Thread it through get_model_context_length (step-0b -> slug-tolerant helper). Supersedes NousResearch#18844

Rename test_slug_collision_first_dict_key_wins → ..._and_exact_match_ precedence_with_prefixed_keys: the body only asserts deterministic exact-match outcomes, never a genuine bare-slug collision (which would be dict-order-dependent and brittle). Docstring now explains why the collision case is deliberately not asserted. Add test_multi_slash_id_strips_only_last_segment pinning 'org/team/model' → 'model' (rsplit('/',1)[1]). Refs NousResearch#30178

…bility seam The existing feasibility tests stub get_model_context_length, so they cannot observe a step-0b slug-resolution regression. Add a test that leaves the resolver chain unmocked: an LM Studio publisher/slug aux model id ('lmstudio/qwen3-coder-30b') against a bare-slug config key ('qwen3-coder-30b') must surface the full 1M window so the feasibility check does not auto-lower the threshold. Threshold is 500K — above DEFAULT_FALLBACK_CONTEXT (256K, the largest probe tier) — so only the slug-resolved 1M can clear it; proven RED when the slug branch is disabled (resolver→None→256K default<500K→warning). Hermetic: step-0b short-circuits before any localhost probe. Refs NousResearch#30178

teknium1

Thanks for tracing this through the shared resolver and its downstream gateway paths. The underlying defect remains on current main: hermes_cli/config.py:5150 only checks the exact models[model] key, while agent/model_metadata.py:2083-2096 relies on that helper before probing.

Problems

The added gateway @-context test uses the pre-refactor runner._model / runner._base_url setup. Current main resolves those values via _resolve_session_agent_runtime (gateway/run.py:10633-10645, introduced by 265ac7d812192d3b6f11888811512604681d3a32), so that test needs to be ported as part of salvage.

Suggested changes

Apply the exact-first, bare-slug fallback at the current helper seam (hermes_cli/config.py:5150), then adapt the hygiene (gateway/run.py:11162-11188), /info (gateway/run.py:12363-12414), and invalid-value warning (agent/agent_init.py:1712-1745) changes to current main.
Base the gateway regression on tests/gateway/test_context_ref_expansion_runtime.py:59-82 so it exercises session-runtime resolution.

Automated hermes-sweeper review.

teknium1 · 2026-07-13T21:14:03Z

+
+    captured = {}
+
+    async def fake_preprocess(message_text, *, cwd, context_length, allowed_root):


Current main resolves the model and endpoint through _resolve_session_agent_runtime rather than GatewayRunner._model/_base_url (gateway/run.py:10633-10645). When salvaging this test, stub that resolver and assert the slug-resolved context reaches the current async preprocessor path.

banditburai added 7 commits June 1, 2026 10:52

fix(gateway): route hygiene context-length lookup through shared helper

0f6b496

Replaces the inline custom_providers models.get(model) loop (which missed on publisher/slug ids) with get_custom_provider_context_length, picking up the slug fallback. Refs NousResearch#30178

fix(agent): slug-tolerant lookup in custom_providers invalid-ctx warning

a417c27

The invalid context_length warning loop missed on publisher/slug ids, silently skipping the warning for slug-keyed models. Mirror the helper's slug fallback. Refs NousResearch#30178

fix(gateway): thread custom_providers into @context budget probe

4c15441

The @context expansion ignored custom_providers per-model context_length. Thread it through get_model_context_length (step-0b -> slug-tolerant helper). Supersedes NousResearch#18844

alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have labels Jun 1, 2026

This was referenced Jun 19, 2026

fix: use LM Studio max_context_length when model is loaded at default 64K #45037

Closed

fix(lmstudio): respect applied runtime context #52188

Closed

briandevans mentioned this pull request Jul 13, 2026

fix(config): match custom-provider base_url case-insensitively for context_length lookup #63542

Closed

13 tasks

teknium1 reviewed Jul 13, 2026

View reviewed changes

GottZ mentioned this pull request Jul 27, 2026

[Bug]: lmstudio provider pre-loads models manually, bypassing JIT and trampling user context config #25989

Open

Navigation Menu

Uh oh!

fix(custom_providers): resolve per-model context_length for publisher/slug model ids - #36852

fix(custom_providers): resolve per-model context_length for publisher/slug model ids#36852
banditburai wants to merge 7 commits into
NousResearch:mainfrom
banditburai:fix/custom-provider-slug-context-length

banditburai commented Jun 1, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		captured = {}

		async def fake_preprocess(message_text, *, cwd, context_length, allowed_root):

Uh oh!

Conversation

banditburai commented Jun 1, 2026

Summary

Change

Resolution behavior

Testing

Note

Related

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants