fix(model-metadata): guard against models.dev underreports at step 5

[ignaciolagosruiz]

Summary

The get_model_context_length() resolution order trusts the models.dev lookup at step 5 over the curated DEFAULT_CONTEXT_LENGTHS table at step 8. When models.dev reports a stale/incorrectly-low value, the correct curated default is never reached.

Concrete case: models.dev reports minimax-m3-free on the opencode provider as 200K context, but DEFAULT_CONTEXT_LENGTHS['minimax-m3'] is 1M. Without this guard, the agent's effective context window is 5× too small and Hermes auto-compresses at 72% of 200K (≈144K) when the model actually accepts ≈720K of input.

How it complements existing work

• PR fix(minimax): drop stale ≤204,800 cache entries for MiniMax-M3 #36726 (Teknium, merged Jun 1 2026) drops stale ≤204,800 cache entries for the M3 family at step 1 — catches the symptom (stale cache from pre-catalog builds).
• This PR adds a step-5 guard that rejects models.dev live lookups contradicting a larger curated default — catches the root cause (any future fresh-lookup underreport, not just M3).

Together the two fixes make minimax-m3-free resolve to 1M regardless of whether the bug surfaces as a stale cache entry or a fresh models.dev probe.

Implementation

Adds the _curated_context_length(model) helper that mirrors the longest-key-first substring match used by step 8 (so the guard compares apples to apples), and a step-5 guard that logs and falls through to the curated value when the live lookup is a known underreport. Mirrors the existing Kimi guard in the OpenRouter path (step 6 below) — same pattern, generalised to any curated-vs-live drift.

Why no step-1 cache invalidation

I prototyped and reverted a generic curated > cached cache invalidation at step 1. It false-positived on:

• Codex gpt-5.5: cached 272K (correct Codex-OAuth cap) vs curated 1.05M (direct-API value)
• Nous qwen3.6-plus: cached 1M vs curated 1,048,576 (rounding noise, not a real underreport)

Cached values are persistent user data; auto-invalidating them on a heuristic is too risky. Users with a stale cached underreport can delete the entry from ~/.hermes/context_length_cache.yaml directly. The fresh-lookup guard at step 5 covers the in-the-wild case without that risk.

Tests

7 new tests in TestCuratedDefaultGuard:

• 3 helper tests (minimax-m3* → 1M, minimax-m2.5 → 204,800, unknown → None)
• 3 step-5 guard tests (underreport rejected, larger value accepted, equal value accepted)
• 1 end-to-end live-resolution test for the original bug

All 147 tests pass in test_model_metadata.py and test_minimax_provider.py. No regressions in the broader 7-file sweep (204 tests).

Reproduction

from agent.model_metadata import get_model_context_length
from unittest.mock import patch

# Without the fix: returns 200,000
# With the fix:    returns 1,000,000
with patch('agent.models_dev.lookup_models_dev_context', return_value=200_000), \
     patch('agent.model_metadata.fetch_model_metadata', return_value={}):
    ctx = get_model_context_length(
        'minimax-m3-free',
        base_url='https://opencode.ai/zen/v1',
        provider='opencode-zen',
    )
assert ctx == 1_000_000

Filed from the Hermes TUI gateway in production use. Confirmed working against https://opencode.ai/zen/v1/chat/completions (HTTP 200, normal token usage).

[teknium1]

Thanks for the detailed reproduction and for avoiding unsafe generic cache invalidation.

Automated hermes-sweeper review found that current main already implements the reported MiniMax-M3/models.dev resolution guarantee:

• agent/model_metadata.py:2373-2384 replaces a MiniMax-M3 models.dev value below the curated catalog value with DEFAULT_CONTEXT_LENGTHS["minimax-m3"] before returning it.
• agent/model_metadata.py:286 defines that curated MiniMax-M3 value as 1,000,000.
• This was shipped by 5a4297a11a83c38ac24eec7df0e4e41d6b3dbb9f (fix(model_metadata): prefer hardcoded 1M for MiniMax M3 over stale models.dev probe) and is included in v2026.6.19.

Closing as implemented on main.