Skip to content

fix(bedrock): serve 1M context window for Claude Opus 4.6+/Sonnet 4.6 - #66551

Closed
sudhraja wants to merge 1 commit into
NousResearch:mainfrom
sudhraja:fix/bedrock-claude-1m-context-length
Closed

fix(bedrock): serve 1M context window for Claude Opus 4.6+/Sonnet 4.6#66551
sudhraja wants to merge 1 commit into
NousResearch:mainfrom
sudhraja:fix/bedrock-claude-1m-context-length

Conversation

@sudhraja

Copy link
Copy Markdown

Summary

Bedrock Claude Opus 4.6/4.7/4.8 and Sonnet 4.6 serve a 1M context window, but Hermes was budgeting them at 200K — so long sessions compressed at ~half the real capacity.

The two halves of the code disagreed:

  • On the wire: build_anthropic_bedrock_client (agent/anthropic_adapter.py) attaches the context-1m-2025-08-07 beta header to every Bedrock Claude request, unlocking the 1M window at the API.
  • In the budget: get_model_context_length resolves Bedrock via the static BEDROCK_CONTEXT_LENGTHS table (agent/bedrock_adapter.py), which capped every Claude model at 200K.

opus-4-8 had no entry at all, so global.anthropic.claude-opus-4-8 substring-matched the generic anthropic.claude-opus-4 → 200K key. Result: 1M unlocked on the wire, 200K enforced by the agent.

Fix

Set the 1M-capable Bedrock Claude models to 1_000_000 in BEDROCK_CONTEXT_LENGTHS, matching the beta header and the existing DEFAULT_CONTEXT_LENGTHS entries in agent/model_metadata.py:

Model Before After
anthropic.claude-opus-4-8 200K (fell through to opus-4) 1M
anthropic.claude-opus-4-7 200K (fell through) 1M
anthropic.claude-opus-4-6 200K 1M
anthropic.claude-sonnet-4-6 200K 1M
Sonnet 4.5 / Opus 4 / Sonnet 4 / Haiku 4.5 / 3.x 200K 200K (unchanged)

Longer keys win the substring match in get_bedrock_context_length, so the generic anthropic.claude-opus-4 fallback still catches unversioned IDs at 200K.

Test Plan

  • Updated the 3 tests in tests/agent/test_bedrock_adapter.py that encoded the stale 200K assumption (opus-4-6, sonnet-4-6, inference-profile), added an opus-4-8 case and a Sonnet-4.5-stays-200K guard.
  • scripts/run_tests.sh tests/agent/test_bedrock_adapter.py137 passed, 0 failed.

The AnthropicBedrock client attaches the context-1m-2025-08-07 beta
header on every Bedrock Claude request (build_anthropic_bedrock_client
in agent/anthropic_adapter.py), unlocking the 1M context window for
Opus 4.6/4.7/4.8 and Sonnet 4.6. But BEDROCK_CONTEXT_LENGTHS still
capped every Claude model at 200K, so get_model_context_length()
budgeted these models at 200K even though 1M was unlocked on the wire.

opus-4-8 had no entry at all and substring-matched the generic
anthropic.claude-opus-4 -> 200K key, so global.anthropic.claude-opus-4-8
sessions were capped at 200K.

Set the 1M-capable models to 1_000_000 to match the beta header and
DEFAULT_CONTEXT_LENGTHS in agent/model_metadata.py. Older models
(Sonnet 4.5, Opus 4, Sonnet 4, Haiku 4.5, 3.x) stay at 200K. Longer
keys win the substring match, so the generic opus-4 fallback still
catches unversioned IDs at 200K.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/bedrock AWS Bedrock (boto3, IAM) P3 Low — cosmetic, nice to have sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades duplicate This issue or pull request already exists labels Jul 17, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of open #24059: the live diff makes the same Bedrock Claude 4.6/4.7/4.8 and Sonnet 4.6 context-table change. #24059 remains the earlier canonical PR.

@sudhraja

Copy link
Copy Markdown
Author

Closing — incorrect approach. The 1M long-context beta on Bedrock is a per-account/org entitlement, not a model-level property. BEDROCK_CONTEXT_LENGTHS is a global static table shipped to all users, so hardcoding these models to 1M would make Hermes budget to 1M for accounts without the entitlement and hit Bedrock ValidationExceptions near the top of the window. The correct, entitlement-safe mechanism already exists: users who have the beta set model.context_length in their own config, which wins at step 0 of get_model_context_length(). No global change needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have provider/bedrock AWS Bedrock (boto3, IAM) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants