fix(bedrock): Fable reports 128K context — add Fable + Claude 4.6/4.7/4.8 1M entries to Bedrock table, invalidate stale cache - #44861
Conversation
…ble, drop stale cached values BEDROCK_CONTEXT_LENGTHS was missing entries for current 1M-context Claude models, and the resolution path in get_model_context_length() short-circuits to that table (step 1b) before DEFAULT_CONTEXT_LENGTHS is ever consulted, so the catalog's correct values could never apply on Bedrock: - claude-fable-5 (no entry at all) fell through to BEDROCK_DEFAULT_CONTEXT_LENGTH and reported 128K for a 1M model. - opus-4-7 / opus-4-8 substring-matched the generic 'anthropic.claude-opus-4' key and reported 200K. - opus-4-6 / sonnet-4-6 had explicit 200K entries predating their 1M windows. The practical symptom: the agent compresses context prematurely (at ~128K or ~200K of a 1M window) on every Bedrock-hosted current Claude model. Fixing the table alone is not enough for existing installs: a previously persisted 128K/200K value in the context-length cache wins at step 1 and masks the corrected table forever. Step 1 now reconciles Bedrock-context cache hits against the static table (the table is authoritative for Bedrock — there is no live probe to reconcile against), invalidating stale entries so existing users converge to the right window without manual cache surgery. Tests cover the new table entries (incl. inference-profile and versioned ID forms), the 128K-default regression for Fable, the stale-cache invalidation path, and that pre-4.6 models keep their 200K entries.
tonydwb
left a comment
There was a problem hiding this comment.
Good fix. Updates Bedrock Fable + Claude 4.6/4.7/4.8 context table entries from 200K to 1M to match AWS/Anthropic documentation. Also adds cache invalidation for stale entries that were seeded before the correction. Test updated. No issues found.
|
Thanks for covering both the static metadata and upgrade-cache path. Current Automated hermes-sweeper review. |
… fixtures - The #44861 stale-cache guard invalidated any cached value that differed from the static table, which would have discarded legitimate probe-derived windows larger than the table. Treat the table as a FLOOR: only drop under-reporting cache entries. - Update probe test fixtures that predated the 4.6+ 1M table flip (opus-4-6 fallback expectations 200K -> 1M).
|
Merged via PR #68006 — your commit was cherry-picked onto current main with your authorship preserved (rebase merge). The Claude 4.6-4.8 table rows had landed hours earlier via #67977 (salvaged from older PRs in the same cluster), but your Fable rows and especially the stale-cache invalidation were the surviving value — without it, anyone who ran Bedrock before the table fix would have kept a 200K cached window forever. One adjustment on top: the invalidation now treats the table as a floor (only under-reporting entries are dropped) so it composes with the live context-probe that landed after it. Thanks! |
… fixtures - The #44861 stale-cache guard invalidated any cached value that differed from the static table, which would have discarded legitimate probe-derived windows larger than the table. Treat the table as a FLOOR: only drop under-reporting cache entries. - Update probe test fixtures that predated the 4.6+ 1M table flip (opus-4-6 fallback expectations 200K -> 1M).
… fixtures - The NousResearch#44861 stale-cache guard invalidated any cached value that differed from the static table, which would have discarded legitimate probe-derived windows larger than the table. Treat the table as a FLOOR: only drop under-reporting cache entries. - Update probe test fixtures that predated the 4.6+ 1M table flip (opus-4-6 fallback expectations 200K -> 1M).
… fixtures - The NousResearch#44861 stale-cache guard invalidated any cached value that differed from the static table, which would have discarded legitimate probe-derived windows larger than the table. Treat the table as a FLOOR: only drop under-reporting cache entries. - Update probe test fixtures that predated the 4.6+ 1M table flip (opus-4-6 fallback expectations 200K -> 1M).
… fixtures - The NousResearch#44861 stale-cache guard invalidated any cached value that differed from the static table, which would have discarded legitimate probe-derived windows larger than the table. Treat the table as a FLOOR: only drop under-reporting cache entries. - Update probe test fixtures that predated the 4.6+ 1M table flip (opus-4-6 fallback expectations 200K -> 1M).
Symptom
On current
main, every Bedrock-hosted current Claude model gets a wrong context window fromget_model_context_length():*.anthropic.claude-fable-5→ 128K (no table entry; falls toBEDROCK_DEFAULT_CONTEXT_LENGTH) — for a 1M model*.anthropic.claude-opus-4-7/-4-8→ 200K (substring-match falls back to the genericanthropic.claude-opus-4key)*.anthropic.claude-opus-4-6/sonnet-4-6→ 200K (explicit stale entries)Practical effect: the agent compresses history prematurely — at 128K/200K of a 1M window.
Repro on main:
Why DEFAULT_CONTEXT_LENGTHS doesn't save it
#42991 correctly registers
claude-fable-5 → 1MinDEFAULT_CONTEXT_LENGTHS, but the Bedrock branch at step 1b inget_model_context_length()returnsget_bedrock_context_length()before that catalog is consulted. On Bedrock,BEDROCK_CONTEXT_LENGTHSis the only voice — so it must carry these entries itself.Fix (whole class, both layers)
bedrock_adapter.py): addanthropic.claude-fable-5/claude-fable/opus-4-8/opus-4-7→ 1M; correctopus-4-6/sonnet-4-6to 1M (AWS lists these as 1M-context Bedrock models). Pre-4.6 models keep 200K. Longest-substring matching keeps the genericopus-4→ 200K fallback safe.model_metadata.py): fixing the table alone never reaches existing installs — a previously persisted 128K/200K cache entry wins at step 1 and masks the corrected table forever. Bedrock-context cache hits are now reconciled against the static table (authoritative for Bedrock; there is no live probe), so existing users converge without manual cache surgery. Mirrors the existing stale-cache patterns (Codex ≥400K, Kimi 32K, MiniMax-M3, Grok-4.3) already in that function.Tests
us./global.) and versioned (-v1:0) ID formstest_bedrock_claude_fable_resolves_to_1m_not_128k_default)test_bedrock_claude_4_6_ignores_stale_200k_cache, real cache file intmp_path, no mocked resolution)test_claude_opus_4_base_stays_200k)pytest tests/agent/test_bedrock_adapter.py tests/agent/test_model_metadata.py→ 230 passed.Relationship to open PRs
#24059 / #26769 / #44419 each correct the 4.6/4.7 table entries but none covers Fable (today's 128K case, the worst one), 4.8, or the stale-cache layer that makes the fix actually land on existing installs. This PR intentionally does not touch the beta-header question (#44419's conditional
context-1mforwarding for regional profiles) — that's an orthogonal request-path concern; this PR is purely metadata resolution.Verified end-to-end on a live Bedrock setup (
global.anthropic.claude-fable-5): reports 1,000,000 after this change, 128,000 before.