Skip to content

fix(model_metadata): drop stale cache entries for qwen3.6-plus, grok-4-fast, grok-4.20 - #37684

Closed
AhmetArif0 wants to merge 1 commit into
NousResearch:mainfrom
AhmetArif0:fix/stale-cache-guard-qwen-grok
Closed

fix(model_metadata): drop stale cache entries for qwen3.6-plus, grok-4-fast, grok-4.20#37684
AhmetArif0 wants to merge 1 commit into
NousResearch:mainfrom
AhmetArif0:fix/stale-cache-guard-qwen-grok

Conversation

@AhmetArif0

Copy link
Copy Markdown
Contributor

Summary

Three model families gained explicit DEFAULT_CONTEXT_LENGTHS entries after some users may have already cached lower (incorrect) values through earlier resolution paths. This PR adds stale-cache guards for each, mirroring the pattern used for Kimi (≤32K), MiniMax-M3 (≤204,800), and Grok-4.3 (≤256K) in prior PRs.

qwen3.6-plus (1M context)

The qwen3.6-plus entry was added on 2026-05-17 (d9abbe7fa). Prior builds resolved it via the generic qwen catch-all (131,072) and could have persisted that value via the context-overflow cache path. Guard threshold: cached ≤ 131,072.

grok-4-fast and grok-4.20 (2M context each)

Both entries were added on 2026-04-10 (b57769718). Before that date, these slugs had no explicit entry and fell through to DEFAULT_FALLBACK_CONTEXT (256,000) or lower probe tiers. Guard threshold: cached ≤ 256,000.

Changes

  • agent/model_metadata.py: add _model_name_suggests_qwen3_6_plus() and _model_name_suggests_grok_4_fast_or_4_20() helper predicates; add three elif branches in the step-1 stale-cache guard block inside get_model_context_length().
  • tests/agent/test_model_metadata.py: add TestQwen36PlusStaleCacheGuard and TestGrokFastAndGrok420StaleCacheGuard (12 tests total) — helper unit tests, stale-drop-and-re-resolve, correct-cache-preserved, no-clobber for sibling slugs.

Test plan

  • pytest tests/agent/test_model_metadata.py::TestQwen36PlusStaleCacheGuard tests/agent/test_model_metadata.py::TestGrokFastAndGrok420StaleCacheGuard -x -q — 12 tests pass

…4-fast, grok-4.20

Three model families gained explicit DEFAULT_CONTEXT_LENGTHS entries after users
may have already cached lower values through earlier lookup paths:

- qwen3.6-plus (1M) — added 2026-05-17 (d9abbe7). Prior builds resolved it
  via the generic 'qwen' catch-all (131,072) and could have persisted that value.
  Guard threshold: cached ≤ 131,072.

- grok-4-fast (2M) and grok-4.20 (2M) — both added 2026-04-10 (b577697).
  Before that, these slugs had no explicit entry and fell through to
  DEFAULT_FALLBACK_CONTEXT (256,000) or lower probe tiers, which can be
  persisted via the context-overflow cache path. Guard threshold: cached ≤ 256,000.

Mirrors the existing stale-cache guard pattern used for Kimi (≤32K), MiniMax-M3
(≤204,800), and Grok-4.3 (≤256K): add a _model_name_suggests_*() predicate and an
elif branch in get_model_context_length() that invalidates the stale entry and
lets the lookup fall through to the correct hardcoded default.

Add 12 regression tests (helper unit tests, stale-drop-and-re-resolve, correct-
cache-preserved, no-clobber for sibling slugs) for each guard.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/qwen Qwen / Alibaba Cloud (OAuth) provider/xai xAI (Grok) labels Jun 2, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused regression fix. The premise remains valid on current main: get_model_context_length() reads the persistent cache at agent/model_metadata.py:2110 and returns a non-exempt cache hit at agent/model_metadata.py:2173-2178, before it can reach the explicit qwen3.6-plus and Grok defaults at agent/model_metadata.py:277 and agent/model_metadata.py:311-312. Current main already uses the same targeted stale-cache-guard pattern for Kimi, MiniMax-M3, and Grok-4.3 at agent/model_metadata.py:2125-2158.

The PR is stale against its June base and will need a mechanical port into the moved guard/test sections, but its implementation approach and behavioral regression coverage remain appropriate.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 13, 2026

@GottZ GottZ left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

Summary

Three PRs address the stale persistent context-cache path: #37068 and #38996 implement the same Grok-4.3 ≤256K eviction guard, while #37684 extends that established fix to qwen3.6-plus, grok-4-fast, and grok-4.20 with model-specific thresholds and regression coverage.

Related pull requests

  • #37068 [closed] related — (+81/-0) — superseded by #38996: adds the Grok-4.3-specific ≤256K stale-cache guard and tests, directly correcting cache hits that otherwise bypass the newer 1M default; it remains relevant because its commit was cherry-picked with attribution into the merged implementation.
  • #37684 related — (+214/-0) — keep open and mechanically port to current main: adds targeted eviction of stale ≤131,072 qwen3.6-plus entries and ≤256,000 grok-4-fast/grok-4.20 entries, while preserving correct caches and sibling model slugs. This agrees with the keep_open review on #37684, which confirms that the cache-first return path and explicit defaults still leave the reported cause present on current main.
  • #38996 [merged] related — (+81/-0) — merged reference implementation: the cherry-picked successor to #37068 adds the same narrowly scoped Grok-4.3 guard and regression tests, resolving stale 256K cache entries without affecting plain Grok-4.

Duplicates

#37068 and #38996 are substantively identical; #38996 is the merged, current-main salvage of #37068. #37684 is complementary rather than duplicate because it covers different model families and stale thresholds.

Suggested consolidation

Merge #37684 after the documented mechanical port onto current main and targeted regression validation; its qwen3.6-plus, grok-4-fast, and grok-4.20 guards complement the pattern already merged in #38996. Keep #37068 closed as superseded by duplicate #38996; no additional PR should be closed as a duplicate of #37684.

Cross-PR triage: Reviewed 3 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 24 kB of PR diffs, 5 kB of issue/PR text, 2 kB of discussion (3 comments), 1 verify verdict. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

teknium1 added a commit that referenced this pull request Aug 13, 2026
Replaces the per-model _model_name_suggests_grok_4_3/_grok_4_6/
_minimax_m3 stale-cache predicates with one generic
_stale_pre_catalog_cache_entry() guard driven by
_PRE_CATALOG_STALE_KEYS. A cached context length is dropped when the
model resolves (longest-key-first, same as step 8) to a listed catalog
key and the cached value is at or below what the old resolution path
could have produced (largest shorter matching catch-all, or the 256K
fallback).

Also covers qwen3.6-plus, grok-4-fast, and grok-4.20 (the models
PR #37684 requested guards for), absorbing that PR.

_model_name_suggests_minimax_m3 is kept for its two non-cache callers
(models.dev underreport guard, cache-control gating in
agent_runtime_helpers).
teknium1 added a commit that referenced this pull request Aug 13, 2026
Replaces the per-model _model_name_suggests_grok_4_3/_grok_4_6/
_minimax_m3 stale-cache predicates with one generic
_stale_pre_catalog_cache_entry() guard driven by
_PRE_CATALOG_STALE_KEYS. A cached context length is dropped when the
model resolves (longest-key-first, same as step 8) to a listed catalog
key and the cached value is at or below what the old resolution path
could have produced (largest shorter matching catch-all, or the 256K
fallback).

Also covers qwen3.6-plus, grok-4-fast, and grok-4.20 (the models
PR #37684 requested guards for), absorbing that PR.

_model_name_suggests_minimax_m3 is kept for its two non-cache callers
(models.dev underreport guard, cache-control gating in
agent_runtime_helpers).
@teknium1

Copy link
Copy Markdown
Contributor

The guards you requested here (qwen3.6-plus, grok-4-fast, grok-4.20) are now on main via PR #85434 (merged 91e550b) — credited to you in the PR body. Rather than adding three more per-model predicates, we replaced the whole family with one generic _stale_pre_catalog_cache_entry() guard driven by a _PRE_CATALOG_STALE_KEYS set; all three of your models are listed and covered by TestGenericPreCatalogStaleGuard, including an E2E stale-drop-and-re-resolve for qwen3.6-plus.

Thanks for spotting the class — closing as absorbed by the generic fix.

@teknium1 teknium1 closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/qwen Qwen / Alibaba Cloud (OAuth) provider/xai xAI (Grok) sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants