Skip to content

fix(bedrock): Fable reports 128K context — add Fable + Claude 4.6/4.7/4.8 1M entries to Bedrock table, invalidate stale cache - #44861

Closed
avifenesh wants to merge 1 commit into
NousResearch:mainfrom
avifenesh:fix/bedrock-fable-context-table
Closed

fix(bedrock): Fable reports 128K context — add Fable + Claude 4.6/4.7/4.8 1M entries to Bedrock table, invalidate stale cache#44861
avifenesh wants to merge 1 commit into
NousResearch:mainfrom
avifenesh:fix/bedrock-fable-context-table

Conversation

@avifenesh

Copy link
Copy Markdown
Contributor

Symptom

On current main, every Bedrock-hosted current Claude model gets a wrong context window from get_model_context_length():

  • *.anthropic.claude-fable-5128K (no table entry; falls to BEDROCK_DEFAULT_CONTEXT_LENGTH) — for a 1M model
  • *.anthropic.claude-opus-4-7 / -4-8200K (substring-match falls back to the generic anthropic.claude-opus-4 key)
  • *.anthropic.claude-opus-4-6 / sonnet-4-6200K (explicit stale entries)

Practical effect: the agent compresses history prematurely — at 128K/200K of a 1M window.

Repro on main:

from agent.model_metadata import get_model_context_length
get_model_context_length('global.anthropic.claude-fable-5', provider='bedrock')
# → 128000, expected 1000000

Why DEFAULT_CONTEXT_LENGTHS doesn't save it

#42991 correctly registers claude-fable-5 → 1M in DEFAULT_CONTEXT_LENGTHS, but the Bedrock branch at step 1b in get_model_context_length() returns get_bedrock_context_length() before that catalog is consulted. On Bedrock, BEDROCK_CONTEXT_LENGTHS is the only voice — so it must carry these entries itself.

Fix (whole class, both layers)

  1. Table (bedrock_adapter.py): add anthropic.claude-fable-5 / claude-fable / opus-4-8 / opus-4-7 → 1M; correct opus-4-6 / sonnet-4-6 to 1M (AWS lists these as 1M-context Bedrock models). Pre-4.6 models keep 200K. Longest-substring matching keeps the generic opus-4 → 200K fallback safe.
  2. Stale cache invalidation (model_metadata.py): fixing the table alone never reaches existing installs — a previously persisted 128K/200K cache entry wins at step 1 and masks the corrected table forever. Bedrock-context cache hits are now reconciled against the static table (authoritative for Bedrock; there is no live probe), so existing users converge without manual cache surgery. Mirrors the existing stale-cache patterns (Codex ≥400K, Kimi 32K, MiniMax-M3, Grok-4.3) already in that function.

Tests

  • New table entries incl. inference-profile (us./global.) and versioned (-v1:0) ID forms
  • Fable 128K-default regression (test_bedrock_claude_fable_resolves_to_1m_not_128k_default)
  • Stale 200K cache entry is dropped and re-resolves to 1M (test_bedrock_claude_4_6_ignores_stale_200k_cache, real cache file in tmp_path, no mocked resolution)
  • Pre-4.6 models stay 200K (test_claude_opus_4_base_stays_200k)

pytest tests/agent/test_bedrock_adapter.py tests/agent/test_model_metadata.py → 230 passed.

Relationship to open PRs

#24059 / #26769 / #44419 each correct the 4.6/4.7 table entries but none covers Fable (today's 128K case, the worst one), 4.8, or the stale-cache layer that makes the fix actually land on existing installs. This PR intentionally does not touch the beta-header question (#44419's conditional context-1m forwarding for regional profiles) — that's an orthogonal request-path concern; this PR is purely metadata resolution.

Verified end-to-end on a live Bedrock setup (global.anthropic.claude-fable-5): reports 1,000,000 after this change, 128,000 before.

…ble, drop stale cached values

BEDROCK_CONTEXT_LENGTHS was missing entries for current 1M-context Claude
models, and the resolution path in get_model_context_length() short-circuits
to that table (step 1b) before DEFAULT_CONTEXT_LENGTHS is ever consulted, so
the catalog's correct values could never apply on Bedrock:

- claude-fable-5 (no entry at all) fell through to
  BEDROCK_DEFAULT_CONTEXT_LENGTH and reported 128K for a 1M model.
- opus-4-7 / opus-4-8 substring-matched the generic 'anthropic.claude-opus-4'
  key and reported 200K.
- opus-4-6 / sonnet-4-6 had explicit 200K entries predating their 1M windows.

The practical symptom: the agent compresses context prematurely (at ~128K or
~200K of a 1M window) on every Bedrock-hosted current Claude model.

Fixing the table alone is not enough for existing installs: a previously
persisted 128K/200K value in the context-length cache wins at step 1 and
masks the corrected table forever. Step 1 now reconciles Bedrock-context
cache hits against the static table (the table is authoritative for Bedrock
— there is no live probe to reconcile against), invalidating stale entries
so existing users converge to the right window without manual cache surgery.

Tests cover the new table entries (incl. inference-profile and versioned ID
forms), the 128K-default regression for Fable, the stale-cache invalidation
path, and that pre-4.6 models keep their 200K entries.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/bedrock AWS Bedrock (boto3, IAM) provider/anthropic Anthropic native Messages API labels Jun 12, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix. Updates Bedrock Fable + Claude 4.6/4.7/4.8 context table entries from 200K to 1M to match AWS/Anthropic documentation. Also adds cache invalidation for stale entries that were seeded before the correction. Test updated. No issues found.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for covering both the static metadata and upgrade-cache path. Current main still has the reported defect: agent/bedrock_adapter.py:1298-1326 assigns Claude 4.6 200K and lacks Fable/Opus 4.7/4.8 entries, while agent/model_metadata.py:2105-2178 can return a persistent cached value before the Bedrock table at :2180-2198 runs. The proposed table update plus Bedrock cache reconciliation addresses both verified paths without changing the resolution order or prompt-cache behavior.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
teknium1 added a commit that referenced this pull request Jul 20, 2026
… fixtures

- The #44861 stale-cache guard invalidated any cached value that differed
  from the static table, which would have discarded legitimate
  probe-derived windows larger than the table. Treat the table as a
  FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
  (opus-4-6 fallback expectations 200K -> 1M).
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #68006 — your commit was cherry-picked onto current main with your authorship preserved (rebase merge). The Claude 4.6-4.8 table rows had landed hours earlier via #67977 (salvaged from older PRs in the same cluster), but your Fable rows and especially the stale-cache invalidation were the surviving value — without it, anyone who ran Bedrock before the table fix would have kept a 200K cached window forever. One adjustment on top: the invalidation now treats the table as a floor (only under-reporting entries are dropped) so it composes with the live context-probe that landed after it. Thanks!

#68006

teknium1 added a commit that referenced this pull request Jul 20, 2026
… fixtures

- The #44861 stale-cache guard invalidated any cached value that differed
  from the static table, which would have discarded legitimate
  probe-derived windows larger than the table. Treat the table as a
  FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
  (opus-4-6 fallback expectations 200K -> 1M).
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
… fixtures

- The NousResearch#44861 stale-cache guard invalidated any cached value that differed
  from the static table, which would have discarded legitimate
  probe-derived windows larger than the table. Treat the table as a
  FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
  (opus-4-6 fallback expectations 200K -> 1M).
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
… fixtures

- The NousResearch#44861 stale-cache guard invalidated any cached value that differed
  from the static table, which would have discarded legitimate
  probe-derived windows larger than the table. Treat the table as a
  FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
  (opus-4-6 fallback expectations 200K -> 1M).
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
… fixtures

- The NousResearch#44861 stale-cache guard invalidated any cached value that differed
  from the static table, which would have discarded legitimate
  probe-derived windows larger than the table. Treat the table as a
  FLOOR: only drop under-reporting cache entries.
- Update probe test fixtures that predated the 4.6+ 1M table flip
  (opus-4-6 fallback expectations 200K -> 1M).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API provider/bedrock AWS Bedrock (boto3, IAM) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants