fix(bedrock): bearer-token auth (anthropic 0.88) + 1M context for Mythos Claude models - #63650
fix(bedrock): bearer-token auth (anthropic 0.88) + 1M context for Mythos Claude models#63650vinayshah1998 wants to merge 3 commits into
Conversation
Competing/overlapping with the anthropic-SDK-bump cluster for the same Bedrock bearer-token root cause (#29309): #54431 bumps to |
… auth anthropic 0.88.0 is the first release whose AnthropicBedrock client reads AWS_BEARER_TOKEN_BEDROCK. On 0.87.0 the client is SigV4-only, so BYOK bearer-token auth to Bedrock fails at request time with "could not resolve credentials from session". The bump is minimal (single minor), additive (no breaking changes to the Messages/Bedrock APIs), and preserves the CVE-2026-34450 / CVE-2026-34452 fixes that motivated the 0.87.0 pin. Moves all three pins that must stay in lockstep: - pyproject.toml [anthropic] extra - tools/lazy_deps.py provider.anthropic (runtime lazy-installer pin) - uv.lock anthropic package + specifier (uv lock --check passes)
989b5ae to
6538328
Compare
|
Sure I'm okay with just going with #26769 since it's broader and adds support for the 1M context window |
Fable 5, Sonnet 5, and Opus 4.7/4.8 were absent from BEDROCK_CONTEXT_LENGTHS, so get_bedrock_context_length() fell through to the 128K default and Hermes compacted/truncated their context far below the 1M that the context-1m-2025-08-07 beta already unlocks. build_anthropic_bedrock_client sends that beta on every Bedrock Claude request, so the cap was purely the static table lagging behind the newer models. Add the four at 1M.
|
I updated this in case you wanted a narrower update to the SDK along with the support for the 1M context windows. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for keeping the SDK pin, lazy-installer pin, and lockfile synchronized. The bearer-token premise remains live on current main: Claude Bedrock models route through AnthropicBedrock (hermes_cli/runtime_provider.py:1975-1988), while the SDK remains pinned to 0.87.0 (pyproject.toml:146, tools/lazy_deps.py:99). The context-table premise is also live because Bedrock resolution uses the static table first (agent/model_metadata.py:2208-2224).
Problems
- The four new 1M table entries have no regression coverage. Existing table tests in
tests/agent/test_bedrock_adapter.py:1165-1196do not cover any added ID. - The new comment generalizes the beta-header guarantee beyond current code's documented Opus 4.6/4.7 and Sonnet 4.6 scope (
agent/anthropic_adapter.py:840-845). The PR does not provide repository-backed validation for the added Fable 5, Sonnet 5, and Opus 4.8 ceilings. - No test exercises bearer-token behavior through the AnthropicBedrock client path.
Suggested changes
- Add targeted context-resolution tests for the added IDs and inference-profile variants.
- Verify the Bedrock ceilings per model before retaining the 1M mappings.
- Add a bearer-vs-SigV4 regression test for the upgraded SDK path.
Automated hermes-sweeper review.
| # sends on every Bedrock Claude request. Without an entry here these fell | ||
| # through to BEDROCK_DEFAULT_CONTEXT_LENGTH (128K), silently capping the | ||
| # window far below what the beta already enables. | ||
| "anthropic.claude-fable-5": 1_000_000, |
There was a problem hiding this comment.
Please add table-resolution coverage for these new IDs (including a regional inference-profile form). Bedrock bypasses generic model metadata and always uses this static table, while the current Bedrock table tests do not cover any of these entries.
There was a problem hiding this comment.
Added in 047dce9: TestBedrockContextLength now covers all four new IDs at 1M, plus their geo/global inference-profile forms (us./eu./jp./au./global., matching the ID tables on the AWS Bedrock model cards), and non-shadowing checks that the new sonnet-5 / opus-4-8 keys leave the existing 200K sonnet-4-6 / opus-4-6 resolutions untouched.
Address review feedback on NousResearch#63650: - Add context-resolution tests for the four new 1M table entries (fable-5, sonnet-5, opus-4-8, opus-4-7), including geo/global inference-profile forms (us./eu./jp./au./global.) and non-shadowing checks against the 200K sonnet-4-6 / opus-4-6 keys. - Add bearer-vs-SigV4 regression tests exercising the real installed SDK through build_anthropic_bedrock_client: AWS_BEARER_TOKEN_BEDROCK is picked up as api_key, absence falls back to the SigV4/boto3 chain, and the installed anthropic version is asserted >= 0.88.0 so a pin downgrade fails loudly in CI instead of at request time. - Re-anchor the 1M table comment on the AWS Bedrock model cards (which document 1M-token context windows for all four models, default rather than beta-gated) instead of overgeneralizing the context-1m beta-header scope documented in anthropic_adapter.py.
|
Addressed all three review points in 047dce9: 1. Regression coverage for the four new 1M table entries. 2. Ceiling verification + comment scope. The 1M values are now anchored on the official AWS Bedrock model cards, each of which documents "Context window: 1M tokens" as the model default (not beta-gated): Fable 5, Sonnet 5, Opus 4.8, Opus 4.7. The in-code comment was rewritten to cite these cards and no longer generalizes the 3. Bearer-token regression test through the AnthropicBedrock path. New Full |
|
@teknium1 Checking in on this one as well - all three review points from July 16 remain addressed in 047dce9 (regression coverage for the four 1M table entries including geo/global inference-profile forms, bearer-token fallback ordering, and the version-gate test). The branch is still green and I am happy to rebase if main has moved. Ready for another look whenever convenient. |
Two related fixes for Anthropic Claude models on AWS Bedrock. Happy to split into two PRs if preferred.
1. Bump
anthropic0.87.0 → 0.88.0 — Bedrock bearer-token authWhy.
0.88.0is the first release whoseAnthropicBedrockclient reads theAWS_BEARER_TOKEN_BEDROCKbearer token. On0.87.0,AnthropicBedrockis SigV4-only, so bring-your-own-key bearer-token auth to Bedrock (short-termABSK…keys) fails at request time with:Hermes routes Claude-on-Bedrock through
AnthropicBedrock(agent/anthropic_adapter.py::build_anthropic_bedrock_client, constructed with no explicitapi_key, relying on the SDK's env/credential chain), so on0.87.0the entire bearer-token Bedrock path for Claude is unusable.0.88.0'sAnthropicBedrock.__init__addsapi_key = os.environ.get("AWS_BEARER_TOKEN_BEDROCK"), so the existing call site works with zero code changes.Safety. Minimal (single minor), additive (0.88 adds Bedrock bearer auth +
stop_details; no breaking Messages/Bedrock API changes), and security-preserving (0.88 ≥ 0.87, so CVE-2026-34450 / CVE-2026-34452 fixes are retained).Pins moved in lockstep (a mismatch makes the runtime lazy-installer in
tools/lazy_deps.pysilently downgrade a manually-upgraded install back to 0.87.0):pyproject.toml—[anthropic]extratools/lazy_deps.py—provider.anthropicuv.lock—anthropicpackage + specifier (uv lock --checkpasses)Tested. Against Bedrock
us-east-1with a bearer token,AnthropicBedrock(0.88.0) picks upAWS_BEARER_TOKEN_BEDROCKand bothus.anthropic.claude-fable-5andus.anthropic.claude-sonnet-5return successful completions. New regression tests intests/agent/test_bedrock_bearer_auth.pyexercise the real installed SDK throughbuild_anthropic_bedrock_client: bearer env var is picked up asapi_key, absence falls back to the SigV4/boto3 chain, and the installedanthropicversion is asserted>= 0.88.0so a future pin downgrade fails in CI rather than at request time.2. 1M context window for Mythos-class Claude on Bedrock
What. Add
anthropic.claude-fable-5,anthropic.claude-sonnet-5,anthropic.claude-opus-4-8,anthropic.claude-opus-4-7toBEDROCK_CONTEXT_LENGTHS(agent/bedrock_adapter.py) at1_000_000.Why. For
provider: bedrock,get_model_context_length()short-circuits to the staticBEDROCK_CONTEXT_LENGTHStable. These newer models are absent, so they fall through toBEDROCK_DEFAULT_CONTEXT_LENGTH(128K) — capping their usable window far below the documented ceiling.Basis for 1M (verified against the AWS Bedrock model cards). Each model's official Bedrock model card documents Context window: 1M tokens as the model default (not beta-gated):
The in-code comment now cites these model cards instead of generalizing the
context-1m-2025-08-07beta-header behavior (whose documented scope inagent/anthropic_adapter.pyis Opus 4.6/4.7 and Sonnet 4.6).Tests.
tests/agent/test_bedrock_adapter.py::TestBedrockContextLengthnow covers all four bare IDs, their geo/global inference-profile forms (us./eu./jp./au./global.per the model cards), and non-shadowing checks thatsonnet-5/opus-4-8entries do not disturb the existing 200Ksonnet-4-6/opus-4-6resolutions.