Skip to content

fix: guard against cached context_length=0 poisoning resolution chain - #231

Merged
OmarB97 merged 1 commit into
mainfrom
fix/context-length-cache-zero-guard-fork
Jun 16, 2026
Merged

fix: guard against cached context_length=0 poisoning resolution chain#231
OmarB97 merged 1 commit into
mainfrom
fix/context-length-cache-zero-guard-fork

Conversation

@OmarB97

@OmarB97 OmarB97 commented Jun 16, 2026

Copy link
Copy Markdown
Owner

Why

The context window display in Hermes can show a broken `? (est)` or `ctx --` value when the persistent context-length cache contains a 0 entry. This happens because `get_model_context_length()` has a guard `if cached is not None` — but `0 is not None` evaluates to `True`, so the function returns 0 instead of falling through to the hardcoded defaults (e.g. 1M for `deepseek-v4-pro`).

The zero propagates to `ContextCompressor.context_length`, which then causes every downstream display code path to fail:

  • CLI status bar: `ctx --` (unknown denominator)
  • TUI status bar: falls back to showing total tokens without context percentage
  • Gateway `/usage`: shows pct=0% with total=0

While `save_context_length()` is never intentionally called with 0, a corrupted cache file (manual edit, YAML parse glitch, or a future buggy probe) can introduce a 0 entry and permanently poison the resolution chain.

What changed

Two-layer defense in `agent/model_metadata.py`:

  1. `get_model_context_length()`: After retrieving a cached value, check `cached <= 0` before checking the provider-specific invalidation chain. Non-positive values are logged as a warning, invalidated from the cache, and the function falls through to normal resolution (which ends at the hardcoded 256K fallback).

  2. `save_context_length()`: Added a guard that refuses to persist values `<= 0`, logging a warning. This prevents new poison entries from ever being written.

How to review

  • `agent/model_metadata.py` — the two changed functions (~40 lines of new comments/code, ~34 lines re-indented within an `else:` block)
  • The re-indented block is the existing Codex/Kimi/Nous cache invalidation chain — no logic changes there, just indented under the new `else:` branch

Evidence

Manual testing:

```

Test 1: save rejects 0

save_context_length('test-model', 'https://api.test.com', 0)
→ "Refusing to cache non-positive context length ..."
→ get_cached_context_length('test-model', 'https://api.test.com') → None ✓

Test 2: cached 0 handling

Manually wrote 0 to cache YAML
get_model_context_length('test-model', base_url='https://api.test-zero.com')
→ "Dropping non-positive cache entry ..."
→ Returns 256,000 (DEFAULT_FALLBACK_CONTEXT) ✓
→ Cache entry auto-invalidated ✓

Test 3: normal resolution still works

get_model_context_length('deepseek-v4-pro', base_url='https://api.deepseek.com', provider='deepseek')
→ 1,000,000 ✓
```

Test suite:

  • `tests/agent/test_model_metadata.py` — 100 passed, 0 failed ✓
  • `tests/hermes_cli/test_custom_provider_context_length.py` — all passing ✓
  • 8 pre-existing failures in `test_compression_feasibility.py` (aux compression model tests, unrelated to this change)

Verification

  • Manual: delete/reset context_length_cache.yaml and verify status bar shows correct context window after first API call
  • Manual: manually inject `context_length: 0` in cache YAML, restart, verify warning logged and context resolves correctly

Risks / gaps

  • Low risk: the guard is purely defensive — the existing code already handles the cache-miss path (falls through to hardcoded defaults)
  • The `else:` block re-indentation is the only structural change; the provider-specific invalidation logic (Codex, Kimi, Nous) is unchanged
  • If a future probe genuinely returns 0 for a real model, this guard would prevent caching but the model would still get DEFAULT_FALLBACK_CONTEXT (256K) — better than showing unknown

Fork counterpart of upstream PR NousResearch#25812. Merging this fork PR so the packaged app ships; upstream PR remains awaiting upstream review.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant