Skip to content

fix(metadata): grok-4.20 context window is 1M, not 2M - #109898

Open
MaxFreedomPollard wants to merge 1 commit into
NousResearch:mainfrom
MaxFreedomPollard:fix/xai-grok-4.20-context-window
Open

MaxFreedomPollard wants to merge 1 commit into
NousResearch:mainfrom
MaxFreedomPollard:fix/xai-grok-4.20-context-window

Conversation

@MaxFreedomPollard

Copy link
Copy Markdown
Contributor

What does this PR do?

DEFAULT_CONTEXT_LENGTHS advertises a 2M window for Grok 4.20 (agent/model_metadata.py:374):

"grok-2-vision": 8192, "grok-4-fast": 2000000, "grok-4.20": 2000000,

xAI documents half that. From https://docs.x.ai/docs/models/grok-4.20, the "At a glance" panel:

Context window 1,000,000

The models index at https://docs.x.ai/docs/models agrees across the whole family — grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning and grok-4.20-multi-agent-0309 are each listed at 1M. All of those resolve through this one catalog key: the comment two lines above notes that "grok-4-fast / grok-4.20 also match their -(non-)reasoning and -multi-agent variants".

The xAI block exists precisely because /v1/models on api.x.ai returns no context_length, so this table is the window for these ids — there is no live value to correct it. Over-advertising by 2x is the dangerous direction: compression never triggers, and the first signal the user gets is the endpoint rejecting the request outright.

grok-4-fast is documented at 2M and keeps its own value; only the grok-4.20 row moves.

Related Issue

No issue. The _PRE_CATALOG_STALE_KEYS guard this touches came from #85434 (merged, "fix(xai): drop stale 256K grok-4.6 context cache + generalize pre-catalog guard").

grok-4.20 still belongs in that frozenset after the change: the docstring requires a key whose catalog value is strictly above every shorter matching key and the 256K fallback, and 1,000,000 clears the grok-4 catch-all (256,000), the grok catch-all (131,072) and the fallback. Only the shared inline comment needed updating, since grok-4-fast on the same line stays at 2M.

Open question, deliberately not changed here: "grok-build-latest": 500000 (line 373) carries the comment "grok-build-latest aliases grok-4.5". The docs' alias rule is <modelname>-latest is aliased to the latest version, i.e. the latest version of that model name — and the only grok-build model in the index is grok-build-0.1 at 256k, which is also what the neighbouring "grok-build": 256000 entry says. That reads like grok-build-latest should be 256K, not 500K (a 2x over-advertisement in the same direction as the bug fixed here), but the aliasing may be deliberate for the Grok Build CLI route, so I have left it alone rather than guess. Happy to follow up in a separate PR if a maintainer confirms.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • agent/model_metadata.py: "grok-4.20"1000000 in DEFAULT_CONTEXT_LENGTHS; the _PRE_CATALOG_STALE_KEYS inline comment (line 1361) now reads # 2M / 1M — matching the "grok-4.3", "grok-4.6", # 1M / 500K style on the line above — since the two keys on that line no longer share a window.
  • tests/agent/test_grok_420_context_window.py (new): get_model_context_length(..., provider="xai") returns 1M for grok-4.20 and for the three documented variants; the catalog entry is 1M; grok-4-fast still resolves to its own 2M.

How to Test

  1. On main, apply only the test file: scripts/run_tests.sh tests/agent/test_grok_420_context_window.py2 failed, 1 passed:

    E  AssertionError: grok-4.20
    E  assert 2000000 == 1000000
    E   +  where 2000000 = get_model_context_length('grok-4.20', provider='xai')
    
    E  assert 2000000 == 1000000
    
  2. On this branch: 3 passed.

  3. Wider sweep — all 10 test files referencing grok-4.20, grok-4-fast or _PRE_CATALOG_STALE_KEYS (includes test_model_metadata.py, test_xai_retirement.py, test_migrate_xai.py, test_codex_transport.py): 581 passed, 0 failed.

  4. Manually: /context on grok-4.20-0309-reasoning now reports 1,000,000; a session that previously ran past ~1M tokens without compressing now compresses at the documented threshold instead of failing at the endpoint.

Checklist

Code

Documentation & Housekeeping

  • I've updated relevant documentation — N/A (catalog data + its inline rationale comment)
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact — N/A (no file I/O or process changes)
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

docs.x.ai/docs/models/grok-4.20 lists "Context window 1,000,000", and the
models index carries grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning
and grok-4.20-multi-agent-0309 at 1M — all of which resolve through this
catalog key.

The 2M entry over-advertised the window by 2x, so compression only kicked in
after the endpoint had already started rejecting the request. grok-4-fast
keeps its own 2M row; the shared _PRE_CATALOG_STALE_KEYS comment now reads
"2M / 1M" and the key still qualifies (1M is strictly above the "grok-4"
catch-all at 256,000).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/xai xAI (Grok) labels Sep 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/xai xAI (Grok) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants