Skip to content

Refactor: Filtering beta header after transformation #23715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sameerlite merged 1 commit into from
Mar 16, 2026
Merged

Refactor: Filtering beta header after transformation #23715

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions litellm/anthropic_beta_headers_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,42 @@ def update_headers_with_filtered_beta(
return headers


def update_request_with_filtered_beta(
headers: dict,
request_data: dict,
provider: str,
) -> tuple[dict, dict]:
"""
Update both headers and request body beta fields based on provider support.
Modifies both dicts in place and returns them.

Args:
headers: Request headers dict (will be modified in place)
request_data: Request body dict (will be modified in place)
provider: Provider name

Returns:
Tuple of (updated headers, updated request_data)
"""
headers = update_headers_with_filtered_beta(headers=headers, provider=provider)

existing_body_betas = request_data.get("anthropic_beta")
if not existing_body_betas:
return headers, request_data

filtered_body_betas = filter_and_transform_beta_headers(
beta_headers=existing_body_betas,
provider=provider,
)
Comment on lines +389 to +396
Copy link
Contributor

@greptile-apps greptile-apps bot Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing type guard for anthropic_beta string value

filter_and_transform_beta_headers expects a List[str], but request_data["anthropic_beta"] could be a plain comma-separated string if the caller passes it through optional_params directly. If it is a string, for header in beta_headers: will iterate over individual characters ("f", "i", "l", "e", …) — none of which exist in the provider mapping — silently dropping all beta headers.

This is unlikely to occur in the current Vertex AI path (where transform_request always stores it as a list), but the function is a public utility and accepts dict without type constraints, making it brittle for future callers.

    existing_body_betas = request_data.get("anthropic_beta")
    if not existing_body_betas:
        return headers, request_data

    # Normalise: handle both list and comma-separated string
    if isinstance(existing_body_betas, str):
        existing_body_betas = [b.strip() for b in existing_body_betas.split(",") if b.strip()]

    filtered_body_betas = filter_and_transform_beta_headers(
        beta_headers=existing_body_betas,
        provider=provider,
    )


if filtered_body_betas:
request_data["anthropic_beta"] = filtered_body_betas
else:
request_data.pop("anthropic_beta", None)

return headers, request_data


def get_unsupported_headers(provider: str) -> List[str]:
"""
Get all beta headers that are unsupported by a provider (have null values in mapping).
Expand Down
16 changes: 9 additions & 7 deletions litellm/llms/anthropic/chat/handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@
import litellm.litellm_core_utils
import litellm.types
import litellm.types.utils
from litellm.anthropic_beta_headers_manager import (
update_request_with_filtered_beta,
)
from litellm.constants import RESPONSE_FORMAT_TOOL_NAME
from litellm.litellm_core_utils.core_helpers import map_finish_reason
from litellm.llms.custom_httpx.http_handler import (
Expand Down Expand Up @@ -58,9 +61,6 @@

from ...base import BaseLLM
from ..common_utils import AnthropicError, process_anthropic_headers
from litellm.anthropic_beta_headers_manager import (
update_headers_with_filtered_beta,
)
from .transformation import AnthropicConfig

if TYPE_CHECKING:
Expand Down Expand Up @@ -339,10 +339,6 @@ def completion(
litellm_params=litellm_params,
)

headers = update_headers_with_filtered_beta(
headers=headers, provider=custom_llm_provider
)

config = ProviderConfigManager.get_provider_chat_config(
model=model,
provider=LlmProviders(custom_llm_provider),
Expand All @@ -360,6 +356,12 @@ def completion(
headers=headers,
)

headers, data = update_request_with_filtered_beta(
headers=headers,
request_data=data,
provider=custom_llm_provider,
)

## LOGGING
logging_obj.pre_call(
input=messages,
Expand Down
27 changes: 27 additions & 0 deletions tests/test_litellm/test_anthropic_beta_headers_filtering.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import litellm
from litellm.anthropic_beta_headers_manager import (
filter_and_transform_beta_headers,
update_request_with_filtered_beta,
)


Expand Down Expand Up @@ -116,6 +117,32 @@ def test_unknown_headers_filtered_out(self, provider):
unknown not in filtered
), f"Unknown header '{unknown}' should be filtered out for {provider}"

def test_update_request_with_filtered_beta_vertex_ai(self):
"""Test combined filtering for both HTTP headers and request body betas."""
headers = {
"anthropic-beta": "files-api-2025-04-14,context-management-2025-06-27,code-execution-2025-05-22"
}
request_data = {
"anthropic_beta": [
"files-api-2025-04-14",
"context-management-2025-06-27",
"code-execution-2025-05-22",
]
}

filtered_headers, filtered_request_data = update_request_with_filtered_beta(
headers=headers,
request_data=request_data,
provider="vertex_ai",
)

assert (
filtered_headers.get("anthropic-beta") == "context-management-2025-06-27"
)
assert filtered_request_data.get("anthropic_beta") == [
"context-management-2025-06-27"
]
Comment on lines +120 to +144
Copy link
Contributor

@greptile-apps greptile-apps bot Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test covers utility, not the actual bug path

The new test validates update_request_with_filtered_beta in isolation, which is useful. However, the root cause described in the PR is that auto-detected beta headers (injected by get_anthropic_beta_list inside VertexAIAnthropicConfig.transform_request) were never filtered. A more targeted regression test would simulate the real scenario:

  • Call litellm.completion (or the relevant handler directly) with a vertex_ai model and messages that contain a file_id source block.
  • Assert that neither data["anthropic_beta"] nor headers["anthropic-beta"] in the captured HTTP request contain files-api-2025-04-14 or code-execution-2025-05-22.

Without this, a future refactor that re-introduces the same omission would not be caught by the test suite.


@pytest.mark.asyncio
async def test_anthropic_messages_http_headers_filtering(self):
"""Test that Anthropic messages API filters HTTP headers correctly."""
Expand Down
Loading