fix(anthropic): auto-inject compact beta header for context_management by Jwrede · Pull Request #27593 · BerriAI/litellm

Jwrede · 2026-05-10T15:37:16Z

Summary

Fix Bedrock InvokeModel rejecting context_management with "Extra inputs are not permitted"
get_anthropic_beta_list() did not detect context_management in optional_params, so the required compact-2026-01-12 beta was never added to the request body's anthropic_beta array
The direct Anthropic path already handled this via _ensure_context_management_beta_header (HTTP headers), but the Bedrock path builds betas from get_anthropic_beta_list() which feeds the body

Changes

litellm/llms/anthropic/common_utils.py: detect context_management edits in get_anthropic_beta_list() and add compact-2026-01-12 for compact edits or context-management-2025-06-27 for other edits
tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py: 3 tests covering compact edit, non-compact edit, and no context_management

Test plan

test_compact_edit_adds_compact_beta -- context_management with compact_20260112 edit includes compact-2026-01-12
test_non_compact_edit_adds_context_management_beta -- non-compact edits include context-management-2025-06-27
test_no_context_management_no_extra_betas -- no context_management does not add extra betas

get_anthropic_beta_list() did not detect context_management in optional_params, so Bedrock InvokeModel requests with compaction never received the required "compact-2026-01-12" in anthropic_beta. Bedrock rejects context_management without the beta header with "Extra inputs are not permitted". The direct Anthropic path already handled this via _ensure_context_management_beta_header (HTTP headers), but the Bedrock path builds its beta list from get_anthropic_beta_list() which feeds the request body's anthropic_beta array. Fixes BerriAI#27532

Jwrede · 2026-05-10T15:37:21Z

@greptileai review

greptile-apps · 2026-05-10T15:40:04Z

Greptile Summary

This PR fixes a bug where Bedrock InvokeModel rejected requests using context_management because get_anthropic_beta_list() never injected the required beta header values into the request body. The direct Anthropic path already handled this via HTTP headers; the fix extends the same logic to the shared get_anthropic_beta_list() used by Bedrock and Vertex.

common_utils.py: detects context_management.edits in optional_params and appends either compact-2026-01-12 or context-management-2025-06-27 to the beta list, mirroring the logic in _ensure_context_management_beta_header.
The new implementation only handles the Anthropic dict format ({\"edits\": [...]}), while the existing _ensure_context_management_beta_header also supports the OpenAI list format ([{\"type\": \"compaction\", ...}]); this gap means the Bedrock/Vertex path would still miss betas for list-format inputs.
Three unit tests are added covering compact edits, non-compact edits, and the no-context_management baseline; the OpenAI list format is not tested.

Confidence Score: 3/5

The fix is correct for the Anthropic dict format but leaves the OpenAI list format unhandled, which could reproduce the original rejection for callers using that shape.

The core fix works for the reported case (dict-format context_management on Bedrock), but the new code diverges from the parallel implementation in _ensure_context_management_beta_header by not handling context_management passed as a plain list. Any caller routing through get_anthropic_beta_list with a list-shaped input will silently receive no beta header, reproducing the original Bedrock rejection for that input shape.

litellm/llms/anthropic/common_utils.py — the list-format branch is missing from the new context_management detection block.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/common_utils.py	Adds context_management beta detection to get_anthropic_beta_list(), but only handles the Anthropic dict format — the OpenAI list format supported by the direct Anthropic path is not covered, leaving a gap for that input shape.
tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py	Adds three new unit tests for the context_management beta logic; all use mock-only patterns correctly, but the OpenAI list format for context_management is not exercised.

_{Reviews (1): Last reviewed commit: "fix(anthropic): auto-inject compact beta..." | Re-trigger Greptile}

greptile-apps · 2026-05-10T15:40:08Z

+        if optional_params:
+            context_management = optional_params.get("context_management")
+            if isinstance(context_management, dict) and "edits" in context_management:
+                for edit in context_management.get("edits", []):
+                    edit_type = edit.get("type", "")
+                    if edit_type in ("compact_20260112", "compaction"):
+                        betas.append(
+                            ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value
+                        )
+                    else:
+                        betas.append(
+                            ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
+                        )


The new code only handles context_management when it's a dict with an "edits" key, but _ensure_context_management_beta_header (the direct Anthropic path) also supports the OpenAI list format (context_management as a plain list of edit objects). If a caller passes context_management as a list to the Bedrock/Vertex path, the beta headers will silently not be added, causing the same "Extra inputs are not permitted" rejection that this PR aims to fix.

Suggested change

if optional_params:

context_management = optional_params.get("context_management")

if isinstance(context_management, dict) and "edits" in context_management:

for edit in context_management.get("edits", []):

edit_type = edit.get("type", "")

if edit_type in ("compact_20260112", "compaction"):

betas.append(

ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value

)

else:

betas.append(

ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value

)

if optional_params:

context_management = optional_params.get("context_management")

edits: list = []

if isinstance(context_management, dict) and "edits" in context_management:

edits = context_management.get("edits", [])

elif isinstance(context_management, list):

edits = context_management

for edit in edits:

edit_type = edit.get("type", "")

if edit_type in ("compact_20260112", "compaction"):

betas.append(

ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value

)

else:

betas.append(

ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value

)

Good catch -- fixed in dcba26c. Now handles both the Anthropic dict format and the OpenAI list format, matching the logic in _ensure_context_management_beta_header. Added a test for the list path too.

greptile-apps · 2026-05-10T15:40:09Z

+
+
+class TestGetAnthropicBetaListContextManagement:
+    """
+    Regression tests for https://github.com/BerriAI/litellm/issues/27532
+    get_anthropic_beta_list must include compact/context-management betas
+    when context_management is in optional_params, so Bedrock InvokeModel
+    receives the required anthropic_beta field.
+    """
+
+    def test_compact_edit_adds_compact_beta(self):
+        from litellm.llms.anthropic.common_utils import AnthropicModelInfo
+
+        info = AnthropicModelInfo()
+        betas = info.get_anthropic_beta_list(
+            model="claude-sonnet-4-6",
+            optional_params={
+                "context_management": {
+                    "edits": [{"type": "compact_20260112"}],
+                }
+            },
+        )
+        assert "compact-2026-01-12" in betas
+
+    def test_non_compact_edit_adds_context_management_beta(self):
+        from litellm.llms.anthropic.common_utils import AnthropicModelInfo
+
+        info = AnthropicModelInfo()
+        betas = info.get_anthropic_beta_list(
+            model="claude-sonnet-4-6",
+            optional_params={
+                "context_management": {
+                    "edits": [{"type": "summarize"}],
+                }
+            },
+        )
+        assert "context-management-2025-06-27" in betas
+
+    def test_no_context_management_no_extra_betas(self):
+        from litellm.llms.anthropic.common_utils import AnthropicModelInfo
+
+        info = AnthropicModelInfo()
+        betas = info.get_anthropic_beta_list(
+            model="claude-sonnet-4-6",
+            optional_params={"max_tokens": 100},
+        )
+        assert "compact-2026-01-12" not in betas
+        assert "context-management-2025-06-27" not in betas


Missing test for OpenAI list format

The three new tests only cover context_management as an Anthropic dict ({"edits": [...]}). The direct Anthropic path (_ensure_context_management_beta_header) also handles the OpenAI list format where context_management is a plain list of edit objects (e.g., [{"type": "compaction", ...}]). A test for this format would catch the gap identified in get_anthropic_beta_list where the list branch is absent.

Added in dcba26c -- test_openai_list_format_adds_compact_beta covers the list branch.

codecov · 2026-05-10T15:40:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The direct Anthropic path supports context_management as either a dict with an "edits" key or a plain list of edit objects. Mirror that logic in get_anthropic_beta_list so Bedrock/Vertex paths also inject the correct beta headers when the OpenAI list format is used.

jgowdy-godaddy

Nice fix! I noticed one edge case that might explain the 3/5 score from the bot:

Issue: When there are mixed edit types (e.g., one compact + one non-compact), the current loop adds a beta for each edit. This could result in BOTH compact-2026-01-12 AND context-management-2025-06-27 being added. Based on Anthropic's docs, if ANY edit is compact, only the compact beta should be used.

I've got a quick refactor + test case that should fix this and likely bump the bot score. Want me to open a PR against your branch, or would you prefer I just share the diff here for you to apply?

The changes are:

Scan all edits first to check if any are compact
If compact found, use only compact beta (more efficient, breaks early)
Otherwise use context-management beta
Add test for mixed edits

Happy to help get this to 5/5! 🚀

jgowdy-godaddy · 2026-05-12T18:19:18Z

Here's the diff if you want to apply it directly:

Changes to common_utils.py

if optional_params:
    context_management = optional_params.get("context_management")
    edits: list = []
    if isinstance(context_management, dict) and "edits" in context_management:
        edits = context_management.get("edits", [])
    elif isinstance(context_management, list):
        edits = context_management

    # Check if ANY edit is compact - if so, use compact beta exclusively
    has_compact = False
    has_other_edits = False

    for edit in edits:
        edit_type = edit.get("type", "")
        if edit_type in ("compact_20260112", "compaction"):
            has_compact = True
            break  # Compact takes precedence
        elif edit_type:
            has_other_edits = True

    if has_compact:
        betas.append(ANTHROPIC_BETA_HEADER_VALUES.COMPACT_2026_01_12.value)
    elif has_other_edits:
        betas.append(
            ANTHROPIC_BETA_HEADER_VALUES.CONTEXT_MANAGEMENT_2025_06_27.value
        )

New test case to add

def test_mixed_edits_uses_compact_beta(self):
    """When mixed edit types are present, compact should take precedence"""
    from litellm.llms.anthropic.common_utils import AnthropicModelInfo

    info = AnthropicModelInfo()
    betas = info.get_anthropic_beta_list(
        model="claude-sonnet-4-6",
        optional_params={
            "context_management": {
                "edits": [
                    {"type": "summarize"},
                    {"type": "compact_20260112"},
                ],
            }
        },
    )
    assert "compact-2026-01-12" in betas
    # Should NOT include both betas when compact is present
    assert "context-management-2025-06-27" not in betas

When mixed edit types are present (e.g. summarize + compact), only emit the compact beta header. Previously both betas were appended independently per edit.

Jwrede · 2026-05-12T18:22:54Z

@jgowdy-godaddy Good catch, fixed in 9e85c8b. When mixed edit types are present, compact now takes precedence and the context-management beta is not emitted alongside it. Added test_mixed_edits_uses_compact_beta covering that case.

Jwrede · 2026-05-16T23:03:52Z

Friendly ping @ishaan-jaff @krrish-berri-2 -- this fixes Bedrock compaction failing with "Extra inputs are not permitted" (missing compact beta header). @jgowdy-godaddy's edge case feedback has been addressed. Ready for review.

github-actions · 2026-08-15T00:04:32Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

greptile-apps Bot reviewed May 10, 2026

View reviewed changes

jgowdy-godaddy reviewed May 12, 2026

View reviewed changes

compact beta takes precedence over context-management beta

9e85c8b

When mixed edit types are present (e.g. summarize + compact), only emit the compact beta header. Previously both betas were appended independently per edit.

Jwrede mentioned this pull request May 13, 2026

fix(router): allow APIConnectionError to trigger cooldown for failover #27423

Closed

3 tasks

github-actions Bot added the stale label Aug 15, 2026

github-actions Bot closed this Aug 23, 2026

Navigation Menu

Uh oh!

Uh oh!

fix(anthropic): auto-inject compact beta header for context_management - #27593

fix(anthropic): auto-inject compact beta header for context_management#27593
Jwrede wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
Jwrede:fix/bedrock-compaction-beta-header

Jwrede commented May 10, 2026

Uh oh!

Jwrede commented May 10, 2026

Uh oh!

greptile-apps Bot commented May 10, 2026

Important Files Changed

Uh oh!

greptile-apps Bot May 10, 2026

Uh oh!

Jwrede May 10, 2026

Uh oh!

greptile-apps Bot May 10, 2026

Uh oh!

Jwrede May 10, 2026

Uh oh!

codecov Bot commented May 10, 2026 •

edited

Loading

Uh oh!

jgowdy-godaddy left a comment

Uh oh!

jgowdy-godaddy commented May 12, 2026

Uh oh!

Jwrede commented May 12, 2026 •

edited

Loading

Uh oh!

Jwrede commented May 16, 2026

Uh oh!

github-actions Bot commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

Jwrede commented May 10, 2026

Summary

Changes

Test plan

Uh oh!

Jwrede commented May 10, 2026

Uh oh!

greptile-apps Bot commented May 10, 2026

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Uh oh!

greptile-apps Bot May 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jwrede May 10, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 10, 2026

Choose a reason for hiding this comment

Uh oh!

Jwrede May 10, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jgowdy-godaddy left a comment

Choose a reason for hiding this comment

Uh oh!

jgowdy-godaddy commented May 12, 2026

Uh oh!

Jwrede commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jwrede commented May 16, 2026

Uh oh!

github-actions Bot commented Aug 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented May 10, 2026 •

edited

Loading

Jwrede commented May 12, 2026 •

edited

Loading