fix(bedrock): serve 1M context window for Claude Opus 4.6+/Sonnet 4.6 - #66551
Closed
sudhraja wants to merge 1 commit into
Closed
fix(bedrock): serve 1M context window for Claude Opus 4.6+/Sonnet 4.6#66551sudhraja wants to merge 1 commit into
sudhraja wants to merge 1 commit into
Conversation
The AnthropicBedrock client attaches the context-1m-2025-08-07 beta header on every Bedrock Claude request (build_anthropic_bedrock_client in agent/anthropic_adapter.py), unlocking the 1M context window for Opus 4.6/4.7/4.8 and Sonnet 4.6. But BEDROCK_CONTEXT_LENGTHS still capped every Claude model at 200K, so get_model_context_length() budgeted these models at 200K even though 1M was unlocked on the wire. opus-4-8 had no entry at all and substring-matched the generic anthropic.claude-opus-4 -> 200K key, so global.anthropic.claude-opus-4-8 sessions were capped at 200K. Set the 1M-capable models to 1_000_000 to match the beta header and DEFAULT_CONTEXT_LENGTHS in agent/model_metadata.py. Older models (Sonnet 4.5, Opus 4, Sonnet 4, Haiku 4.5, 3.x) stay at 200K. Longer keys win the substring match, so the generic opus-4 fallback still catches unversioned IDs at 200K.
Collaborator
Author
|
Closing — incorrect approach. The 1M long-context beta on Bedrock is a per-account/org entitlement, not a model-level property. BEDROCK_CONTEXT_LENGTHS is a global static table shipped to all users, so hardcoding these models to 1M would make Hermes budget to 1M for accounts without the entitlement and hit Bedrock ValidationExceptions near the top of the window. The correct, entitlement-safe mechanism already exists: users who have the beta set model.context_length in their own config, which wins at step 0 of get_model_context_length(). No global change needed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bedrock Claude Opus 4.6/4.7/4.8 and Sonnet 4.6 serve a 1M context window, but Hermes was budgeting them at 200K — so long sessions compressed at ~half the real capacity.
The two halves of the code disagreed:
build_anthropic_bedrock_client(agent/anthropic_adapter.py) attaches thecontext-1m-2025-08-07beta header to every Bedrock Claude request, unlocking the 1M window at the API.get_model_context_lengthresolves Bedrock via the staticBEDROCK_CONTEXT_LENGTHStable (agent/bedrock_adapter.py), which capped every Claude model at 200K.opus-4-8had no entry at all, soglobal.anthropic.claude-opus-4-8substring-matched the genericanthropic.claude-opus-4→ 200K key. Result: 1M unlocked on the wire, 200K enforced by the agent.Fix
Set the 1M-capable Bedrock Claude models to
1_000_000inBEDROCK_CONTEXT_LENGTHS, matching the beta header and the existingDEFAULT_CONTEXT_LENGTHSentries inagent/model_metadata.py:anthropic.claude-opus-4-8opus-4)anthropic.claude-opus-4-7anthropic.claude-opus-4-6anthropic.claude-sonnet-4-6Longer keys win the substring match in
get_bedrock_context_length, so the genericanthropic.claude-opus-4fallback still catches unversioned IDs at 200K.Test Plan
tests/agent/test_bedrock_adapter.pythat encoded the stale 200K assumption (opus-4-6, sonnet-4-6, inference-profile), added an opus-4-8 case and a Sonnet-4.5-stays-200K guard.scripts/run_tests.sh tests/agent/test_bedrock_adapter.py→ 137 passed, 0 failed.