fix(context-length): context_window alias, config override bypasses 64K guard, extend 32K underreport guard to MiniMax - #28634
Conversation
|
Thanks for the summary. Aware of the prior attempts - the reason I bundled these into one PR is that all four issues share the same root in the context-length resolution pipeline, and fixing them in isolation tends to leave edge cases open (e.g. fixing the 64K guard without the context_window alias still breaks users who set the wrong key). Happy to split into smaller PRs if that's easier to review, but the combined approach felt cleaner given the shared code path. |
|
Test Plan # context_window alias
./scripts/run_tests.sh tests/agent/test_model_metadata.py -q -k "context"
# 64K guard bypass with explicit config override
./scripts/run_tests.sh tests/agent/test_run_agent.py -q -k "context_length or minimum_context"
# MiniMax / Kimi 32K underreport guard
./scripts/run_tests.sh tests/agent/test_model_metadata.py -q -k "kimi or minimax or plausibly_wrong"Verified locally: |
|
Thanks for the thorough writeup here, Beyaz — you clearly mapped the context-length resolution path well. We're going to close this without merging, and I want to be straight about why across all four issues it targets: #24140 (MiniMax-M2.x → 32K) and #24268 (Kimi-k2.6 → 32K): both are already fixed on current #11096 (64K boot guard): we're declining the override by design. A model below the 64K minimum is genuinely unreliable for tool-calling workflows, and we'd rather not ship a config escape hatch that lets people footgun themselves into a broken setup. The guard stays unconditional on purpose. #8015 ( One heads-up for future PRs: the new None of this is a reflection on the effort — the analysis was solid and the issue-mapping was useful. It just didn't line up with where we want the resolution path and the guard to go. Keep them coming. |
Fixes #8015, #11096, #24140, #24268
Problem
Four related issues in the context-length resolution pipeline, all
stemming from the same area of code:
#8015,
context_windowsilently ignored in config.yamlUsers setting
model.context_window(a natural alias formodel.context_length) got no error and no effect. Hermes fell throughto auto-detection, hitting the 64K minimum guard on large-context models
configured with a small native window, or compressing unnecessarily on
1M+ models.
#11096, 64K boot guard fires even when
context_lengthis explicitly setThe guard was unconditional. A user running e.g. Qwen 3.5 35B-A3B (32K
native) with
model.context_length: 40000in config.yaml still got:#24268, kimi-k2.6 on Nous path resolves to 32K, blocks boot
OpenRouter's catalog reports 32,768 for kimi-k2.6. The existing
_model_name_suggests_kimi()guard was meant to catch this but theNous resolution path called the raw
ctx <= 32768 and _model_name_suggests_kimi()check inconsistently, allowing a staleOR cache entry to pass through and freeze 32K into the persistent cache.
#24140,MiniMax-M2.x resolves to 32K on OpenRouter
Same root cause as #24268 - OR catalog underreports MiniMax M2.x as
32,768 tokens. The
_model_name_suggests_kimi()guard didn't coverMiniMax, so these models hit the 64K boot guard every time.
Root causes
run_agent.pyandgateway/run.pyonly called.get("context_length")the
context_windowkey was never read.whether
_config_context_lengthwas set._model_name_suggests_kimi()only covered Kimi/Moonshot families.The scattered
ctx <= 32768 and _model_name_suggests_kimi(model)pattern was duplicated across 4 call sites with no shared abstraction.
Fix
run_agent.pycontext_windowalias + guard bypassmodel.context_windowas a fallback whenmodel.context_lengthis absent (Bug: context_window key in config.yaml silently ignored, falls back to 128K default #8015). Both keys now produce the same effect.
_config_context_length is not None(Three bugs in v0.9.0: context_length override, thinking-block sessions, config.yaml vs env precedence #11096). The user explicitly told Hermes the context size respect it.
gateway/run.pycontext_windowaliascontext_windowalias applied to both context-length read sitesin the gateway path (Bug: context_window key in config.yaml silently ignored, falls back to 128K default #8015).
agent/model_metadata.pyunified 32K underreport guard_model_name_suggests_kimi()extended to cover MiniMax M2.x andMiniMax-Text-01 (All models rejected with "context window below minimum 64,000 tokens" — Telegram completely down #24140), the second most frequently misreported
family in OR's catalog.
_or_ctx_is_plausibly_wrong(ctx, model)consolidates thescattered
ctx <= 32768 and _model_name_suggests_kimi()pattern.All 4 call sites now go through this single function, making future
additions (new model families) a one-line change.
What this does NOT change
explicit override is set, protecting users from tool-calling failures.
context_windowandcontext_lengthare treated identically;if both are set,
context_lengthwins (existing key takes priority).