fix(bedrock): translate body-level betas into additionalModelRequestFields.anthropic_beta for Converse - #28096
Conversation
Squash-merged by litellm-agent from FabrizioCafolla's PR.
…erriAI#27927) Squash-merged by litellm-agent from Cyberfilo's PR.
Squash-merged by litellm-agent from tomdee's PR.
Squash-merged by litellm-agent from escon1004's PR.
…conversations (BerriAI#28080) Squash-merged by litellm-agent from Divyansh8321's PR.
|
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes a
Confidence Score: 4/5The change is a narrowly scoped transformation fix that only affects how beta flags flow into the Bedrock Converse request body; existing header-only and non-Anthropic-model paths are unchanged and covered by tests. The implementation is correct and well-tested across all four relevant scenarios. The only gaps are a missing explicit test for the duplicate-beta deduplication path and a lack of observability when betas are silently dropped for non-Anthropic models — neither affects correctness on the fixed path. No files require special attention;
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/chat/converse_transformation.py | Adds _pop_body_anthropic_betas() static helper and updates _process_tools_and_beta() to merge body-level betas/anthropic_beta with header-derived betas, then deduplicate with dict.fromkeys(); non-Anthropic models have the field silently stripped. |
| tests/test_litellm/llms/bedrock/test_anthropic_beta_support.py | Adds 4 new unit tests covering body betas, body anthropic_beta, merged header+body, and non-Anthropic model stripping; all tests call transformation methods directly with no real network calls. |
| .gitignore | Adds .local/ to gitignore and fixes missing newline at end of file. |
Reviews (1): Last reviewed commit: "fix(bedrock): translate body-level betas..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| assert "betas" not in additional_fields | ||
|
|
||
| def test_messages_transformation_anthropic_beta(self): |
There was a problem hiding this comment.
betas) and an anthropic-beta header. The existing test_converse_transformation_betas_body_and_header_merged test only checks distinct values; without a duplicate-entry case, the dict.fromkeys() dedup logic has no explicit regression coverage.
| assert "betas" not in additional_fields | |
| def test_messages_transformation_anthropic_beta(self): | |
| assert "betas" not in additional_fields | |
| def test_converse_transformation_betas_body_and_header_deduplicated(self): | |
| """Test that duplicate betas from body and header are collapsed into a single entry.""" | |
| config = AmazonConverseConfig() | |
| headers = {"anthropic-beta": "context-1m-2025-08-07"} | |
| result = config._transform_request_helper( | |
| model="anthropic.claude-opus-4-7-20251101-v1:0", | |
| system_content_blocks=[], | |
| optional_params={"betas": ["context-1m-2025-08-07"]}, | |
| messages=[{"role": "user", "content": "Test"}], | |
| headers=headers, | |
| ) | |
| additional_fields = result["additionalModelRequestFields"] | |
| assert additional_fields["anthropic_beta"].count("context-1m-2025-08-07") == 1 | |
| def test_messages_transformation_anthropic_beta(self): |
| body_betas: List[str] = [] | ||
| for body_beta_key in ("betas", "anthropic_beta"): | ||
| value = additional_request_params.pop(body_beta_key, None) | ||
| if value is None: | ||
| continue | ||
| if isinstance(value, list): | ||
| body_betas.extend(str(beta) for beta in value) | ||
| elif isinstance(value, str): | ||
| body_betas.append(value) | ||
| return body_betas |
There was a problem hiding this comment.
betas/anthropic_beta in the request body are silently dropped for non-Anthropic models. A user who mistakenly routes a beta-flagged request to a Nova or Llama model will get no indication that their betas were ignored. A single logger.debug call would make this much easier to diagnose without affecting the hot path.
| body_betas: List[str] = [] | |
| for body_beta_key in ("betas", "anthropic_beta"): | |
| value = additional_request_params.pop(body_beta_key, None) | |
| if value is None: | |
| continue | |
| if isinstance(value, list): | |
| body_betas.extend(str(beta) for beta in value) | |
| elif isinstance(value, str): | |
| body_betas.append(value) | |
| return body_betas | |
| body_betas: List[str] = [] | |
| for body_beta_key in ("betas", "anthropic_beta"): | |
| value = additional_request_params.pop(body_beta_key, None) | |
| if value is None: | |
| continue | |
| if isinstance(value, list): | |
| body_betas.extend(str(beta) for beta in value) | |
| elif isinstance(value, str): | |
| body_betas.append(value) | |
| if value is not None: | |
| import logging | |
| logger = logging.getLogger(__name__) | |
| logger.debug( | |
| "Stripped body-level '%s' from additionalModelRequestFields; " | |
| "will re-emit as anthropic_beta for Anthropic models only.", | |
| body_beta_key, | |
| ) | |
| return body_betas |
|
🤖 litellm-agent: Auto-merge skipped — the staging branch Please rebase your branch onto |
1c8632b to
da8a0b2
Compare
…ields.anthropic_beta for Converse
Adds two unit tests to AmazonConverseConfig._transform_request_helper coverage: - test_converse_transformation_betas_body_string_value: exercises the isinstance(value, str) branch in _pop_body_anthropic_betas, which the previous list-only tests left at 0% coverage (Codecov flag on PR). - test_converse_transformation_betas_deduplicated: explicitly verifies that the same beta arriving via both the anthropic-beta header and the body betas array collapses to a single entry in additionalModelRequestFields.anthropic_beta. Greptile review noted the dedup path was only implicitly covered by an existing computer-use test. No production changes.
da8a0b2 to
09bf878
Compare
|
Heads up — the failing "Code Quality Checks" workflow is flagging Once a follow-up adds the test (or the script is updated) on the staging |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
7 participants
Relevant issues
Fixes #28081
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Before (failure)
The reproducer from #28081 returns HTTP 400 from AWS Bedrock:
{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: betas: Extra inputs are not permitted\"}", "code": "400"}}After (fix)
The body-level
betasfield is now translated intoadditionalModelRequestFields.anthropic_beta, which AWS Bedrock Converse accepts. The request body LiteLLM now sends to AWS becomes:{ "additionalModelRequestFields": { "anthropic_beta": ["context-1m-2025-08-07"] }, ... }Test output
Full Bedrock chat suite (
tests/test_litellm/llms/bedrock/chat/+test_anthropic_beta_support.py): 269 passed, 0 failed.Type
🐛 Bug Fix
Changes
Problem. When an Anthropic 1M-context request was routed through
bedrock_converse, LiteLLM forwarded the user-suppliedbetas: ["context-1m-2025-08-07"]array as a top-level body field. AWS Bedrock's Converse API rejects unknown top-level fields and returnedHTTP 400 — betas: Extra inputs are not permitted. The same payload worked againstanthropic(native),bedrock(Invoke), andvertex_ai, so Converse was the only blocked route to 1M context.Root cause.
AmazonConverseConfig._process_tools_and_beta()inlitellm/llms/bedrock/chat/converse_transformation.pyonly collected beta values from theanthropic-betarequest header. Body-levelbetas(Anthropic Messages API style) was untouched and passed straight through to AWS inadditionalModelRequestFields.Fix. Extract body-level
betas(and the alternateanthropic_betakey) inside_process_tools_and_beta(), merge with header-derived values, and emit a single deduplicatedadditionalModelRequestFields.anthropic_betalist — the shape AWS Bedrock Converse accepts. Non-Anthropic models on Bedrock (Nova, Llama, etc.) still have the field stripped without re-adding it, preserving existing behavior.Files changed
litellm/llms/bedrock/chat/converse_transformation.py_pop_body_anthropic_betas()static helper stripsbetas/anthropic_betafromadditional_request_params._process_tools_and_beta()now merges header- and body-derived betas; final list is deduplicated withdict.fromkeys()(order-preserving) so the body, headers, and auto-detected sources like computer-use can't double-list the same beta.tests/test_litellm/llms/bedrock/test_anthropic_beta_support.pytest_converse_transformation_betas_from_body— bodybetas→additionalModelRequestFields.anthropic_beta, top-levelbetasremoved.test_converse_transformation_anthropic_beta_from_body— same for the alternateanthropic_betabody key.test_converse_transformation_betas_body_and_header_merged— bodybetas+ headeranthropic-betamerged into a single list.test_converse_transformation_betas_body_non_anthropic_model— bodybetasstripped for non-Anthropic models (Nova) without re-addinganthropic_beta.Verification
uv run ruff checkanduv run black .both clean.