fix(anthropic): register thinking-binding-controls-2026-08-01 in beta headers config by clonylu · Pull Request #41203 · BerriAI/litellm

clonylu · 2026-09-15T07:19:36Z

TLDR

Problem this solves:

thinking.block_binding (preserved thinking, Claude Fable 5.1) needs beta thinking-binding-controls-2026-08-01
The proxy forwards the body field but strips the header (unknown to the config), so Bedrock/Vertex 400

How it solves it:

Registers the header in anthropic_beta_headers_config.json for anthropic, bedrock, bedrock_converse, vertex_ai, databricks
azure_ai left null (not verified on Foundry)
One parametrized test asserting the header survives filtering for those providers

User Flow

Before: a developer sending Anthropic's documented preserved-thinking opt-in to a Claude Fable 5.1 deployment behind the gateway gets a 400 from the provider

They send POST https://litellm-domain/v1/messages with header anthropic-beta: thinking-binding-controls-2026-08-01 and "thinking": {"type": "adaptive", "block_binding": {"prefix_mismatch_behavior": "drop_block"}} against a vertex_ai/claude-fable-5-1 or bedrock/us.anthropic.claude-fable-5-1 deployment
They get HTTP 400 invalid_request_error with the provider text thinking.adaptive.block_binding: Extra inputs are not permitted
Dropping block_binding makes the request succeed but they lose input_transformations and any way to enforce the prefix check through the proxy

After: the same request succeeds and the API reports what it dropped

They send the same POST https://litellm-domain/v1/messages with the same header and body
They get HTTP 200 with content plus a top-level input_transformations: []
On a later turn with an edited earlier message, input_transformations lists the dropped blocks with reason: "prefix_binding_mismatch" (or the documented 400 under "error")

Relevant issues

Fixes #41202

Affected release

Linear ticket

Resolves LIT-7852

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
uv run pytest tests/test_litellm/test_anthropic_beta_headers_filtering.py -v passes locally (27 passed; the 5 new cases fail on unpatched main). tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py still passes (112)
My PR passes all required CI/CD checks
My PR's scope is as isolated as possible; it only solves 1 specific problem
Greptile confidence score 5/5

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

QA run at this PR's tip c19a19999f (after) against its merge base 15bd8b0e4a (before). Each leg is its own worktree and venv, booted with python litellm/proxy/proxy_cli.py --config config.yaml --port <random free port> --num_workers 2 --detailed_debug (2 uvicorn workers per leg, before on 40307, after on 25461) with LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True in the env so each leg reads its own bundled anthropic_beta_headers_config.json. Without that flag the proxy downloads the config from main at boot, which is why the after leg kept returning 400 on its first boot: it was running main's config, not this branch's. Every request below hit real Bedrock (us-east-1), first-party Anthropic, Databricks and Vertex AI, all paid. The proof cases use the Fable 5.1 entries, plus the Vertex Sonnet 5 entry for the reason given in the Vertex AI section; the blast radius section further down uses the rest

model_list:
  - model_name: bedrock-fable
    litellm_params:
      model: bedrock/us.anthropic.claude-fable-5-1
      aws_region_name: us-east-1
  - model_name: bedrock-invoke-fable
    litellm_params:
      model: bedrock/invoke/us.anthropic.claude-fable-5-1
      aws_region_name: us-east-1
  - model_name: bedrock-sonnet45
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
      aws_region_name: us-east-1
  - model_name: vertex-fable
    litellm_params:
      model: vertex_ai/claude-fable-5-1
      vertex_project: os.environ/VERTEXAI_PROJECT
      vertex_location: us-east5
  - model_name: vertex-fable-global
    litellm_params:
      model: vertex_ai/claude-fable-5-1
      vertex_project: os.environ/VERTEXAI_PROJECT
      vertex_location: global
  - model_name: vertex-fable-eu
    litellm_params:
      model: vertex_ai/claude-fable-5-1
      vertex_project: os.environ/VERTEXAI_PROJECT
      vertex_location: europe-west1
  - model_name: vertex-sonnet5-global
    litellm_params:
      model: vertex_ai/claude-sonnet-5
      vertex_project: os.environ/VERTEXAI_PROJECT
      vertex_location: global
  - model_name: anthropic-fable
    litellm_params:
      model: anthropic/claude-fable-5-1
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: anthropic-sonnet45
    litellm_params:
      model: anthropic/claude-sonnet-4-5-20250929
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: databricks-opus5
    litellm_params:
      model: databricks/databricks-claude-opus-5
      api_base: os.environ/DATABRICKS_API_BASE
      api_key: os.environ/DATABRICKS_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

$LITELLM_KEY is the master key from general_settings. The /v1/messages cases send the beta as an anthropic-beta header the way the Anthropic SDK does; the /v1/chat/completions and /v1/responses cases send it through extra_headers, which is how OpenAI-format clients reach it

Before (merge base `15bd8b0e4a`, port 40307)

/v1/messages -> bedrock-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"bedrock-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "invalid_request_error", "message": "litellm.BadRequestError: BedrockException - {\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"}. Received Model Group=bedrock-fable\nAvailable Model Group Fallbacks=None"}}
HTTP 400

/v1/messages -> vertex-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"vertex-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "rate_limit_error", "message": "litellm.RateLimitError: litellm.RateLimitError: vertex_aiException - {\n  \"error\": {\n    \"code\": 429,\n    \"message\": \"Quota exceeded for aiplatform.googleapis.com/online_prediction_input_tokens_per_minute_per_base_model with base model: anthropic-claude-fable-5-1. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai.\",\n    \"status\": \"RESOURCE_EXHAUSTED\"\n  }\n}\n. Received Model Group=vertex-fable\nAvailable Model Group Fallbacks=None"}}
HTTP 429

/v1/messages -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"anthropic-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "invalid_request_error", "message": "litellm.BadRequestError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_011CfBPDR9NRXkTGm9NJVeZJ\"}. Received Model Group=anthropic-fable\nAvailable Model Group Fallbacks=None"}}
HTTP 400

/v1/chat/completions -> bedrock-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"bedrock-fable","max_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: thinking.adaptive.block_binding: Extra inputs are not permitted\"}. Received Model Group=bedrock-fable\nAvailable Model Group Fallbacks=None", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400

/v1/chat/completions -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"anthropic-fable","max_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"error": {"message": "litellm.BadRequestError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_011CfBPDYqj2pLB7pcQxVLQZ\"}. Received Model Group=anthropic-fable\nAvailable Model Group Fallbacks=None", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400

/v1/responses -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/responses \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"anthropic-fable","max_output_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"input":"Reply with the single word ok"}'

{"error": {"message": "litellm.BadRequestError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_011CfBPDbJoVuKuhVhwoZy8P\"}. Received Model Group=anthropic-fable\nAvailable Model Group Fallbacks=None", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400

After (`c19a19999f`, port 25461)

/v1/messages -> bedrock-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"bedrock-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"id": "msg_bdrk_k7qnfagytyg6ktvak76jxwqcqlitwcpymxt7yepkdxgt5kxteuia", "model": "bedrock-fable", "stop_reason": "end_turn", "input_transformations": [], "content_types": ["text"], "text": "ok"}
HTTP 200

/v1/messages -> vertex-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"vertex-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "rate_limit_error", "message": "litellm.RateLimitError: litellm.RateLimitError: vertex_aiException - {\n  \"error\": {\n    \"code\": 429,\n    \"message\": \"Quota exceeded for aiplatform.googleapis.com/online_prediction_input_tokens_per_minute_per_base_model with base model: anthropic-claude-fable-5-1. Please submit a quota increase request. https://cloud.google.com/vertex-ai/docs/generative-ai/quotas-genai.\",\n    \"status\": \"RESOURCE_EXHAUSTED\"\n  }\n}\n. Received Model Group=vertex-fable\nAvailable Model Group Fallbacks=None"}}
HTTP 429

/v1/messages -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"anthropic-fable","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"id": "msg_011CfBPDSErto3ykiBgLB6WX", "model": "anthropic-fable", "stop_reason": "end_turn", "input_transformations": [], "content_types": ["text"], "text": "ok"}
HTTP 200

/v1/chat/completions -> bedrock-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"bedrock-fable","max_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"id": "chatcmpl-c1745096-38ac-49ca-bd8a-3a2164b9331f", "model": "bedrock-fable", "finish_reason": "stop", "content": "ok", "has_reasoning": false}
HTTP 200

/v1/chat/completions -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"anthropic-fable","max_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"id": "chatcmpl-50a7f0ec-f099-4325-b8a2-2197a4edae6b", "model": "anthropic-fable", "finish_reason": "stop", "content": "ok", "has_reasoning": false}
HTTP 200

/v1/responses -> anthropic-fable

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/responses \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -d '{"model":"anthropic-fable","max_output_tokens":256,"extra_headers":{"anthropic-beta":"thinking-binding-controls-2026-08-01"},"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"input":"Reply with the single word ok"}'

{"id": "resp_VaZIp9ygZ-g_mCOehn68jVLRjFimR4JMnKDE9JLC6qfh8TjfehuNdIDNfn6kOZXQWhkyCxQ9Zi44rsp-WeS0NjRwWl0re75TrKZtJYAleul9hGIHRn4ducgtiN2vx12po-4Q9TUYboB1j45_nOstfitJaU6F8UyczGfjb-wo-EowWUdQKHsTpbaaPsRnRNtRIHqvHGRqsWBrAnyLUJgsCRJZp1blR7VsvQ11kKiLNDkSobMi8UGgLcvFKg_c0fx0ZcQSLRGwCeRgU4-dOazwznyykj_bRXbjZ086C1922CjG7e9Tn-pksGb-PA1Xa7C1TfIFwWidafWLQ3EInYMLzj34fqWUWA-49n1Cl0zEzL2UlxyxTyG-kRJpedSdYxsdQPtS-wT4EyuQg-yQM7HlejVBjdKXSm2kJdE38Z8jLqXJyLbK8XHt2KGsyPv_kLTHQAWBE2rGlOhtIw==", "model": "anthropic-fable", "status": "completed", "output_types": ["message"], "text": ["ok"]}
HTTP 200

Vertex AI

Both legs got HTTP 429 from Vertex AI for Claude Fable 5.1 for the whole run, in us-east5 (above), global and europe-west1: Quota exceeded for aiplatform.googleapis.com/online_prediction_input_tokens_per_minute_per_base_model with base model: anthropic-claude-fable-5-1 and the global_online_prediction_requests_per_base_model variant. A direct call to Vertex without the proxy answers the same 429 in under a second, so the project has no Fable 5.1 quota. It does have Claude Sonnet 5 quota in global, and Sonnet 5 shows the same symptom directly against Vertex: HTTP 400 thinking.adaptive.block_binding: Extra inputs are not permitted without the header and HTTP 200 with input_transformations: [] with it. So the vertex_ai leg runs on vertex_ai/claude-sonnet-5 at global (the vertex-sonnet5-global entry above), on the same two worktrees rebooted on the same ports

Before (merge base `15bd8b0e4a`, port 40307): `/v1/messages` -> vertex-sonnet5-global

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"vertex-sonnet5-global","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "invalid_request_error", "message": "litellm.BadRequestError: Vertex_aiException BadRequestError - {\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"thinking.adaptive.block_binding: Extra inputs are not permitted\"},\"request_id\":\"req_vrtx_011CfBUkGtUMW8GHEKRz31ef\"}. Received Model Group=vertex-sonnet5-global\nAvailable Model Group Fallbacks=None"}}
HTTP 400

After (`c19a19999f`, port 25461): same request

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model":"vertex-sonnet5-global","max_tokens":256,"thinking":{"type":"adaptive","block_binding":{"prefix_mismatch_behavior":"drop_block"}},"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

{"id": "msg_vrtx_011CfBUkg2WvmKAnKbiNTyWT", "model": "vertex-sonnet5-global", "stop_reason": "end_turn", "input_transformations": [], "content_types": ["text"], "text": "ok"}
HTTP 200

Claude Code

Claude Code v2.1.277 pointed at each proxy through ANTHROPIC_BASE_URL with model bedrock-fable, as a regression check on the largest /v1/messages client. Both legs answer. Through a custom base URL Claude Code sends neither thinking-binding-controls-2026-08-01 in its anthropic-beta list nor block_binding in the body, so it cannot show this symptom; these panes only show the PR leaves its path alone

Before (15bd8b0):

After (c19a199):

Blast radius

Same two proxies and ports, driven through every filter_and_transform_beta_headers call site with the beta on models that do and do not implement it; the checkbox at the bottom of this description is that pass's verdict. The one behavior change beyond the fix is Bedrock's own reply to the forwarded header on a model that lacks it (Claude Sonnet 4.5 here), where the merge base silently dropped the header and answered 200

Before: `/v1/messages` -> bedrock-sonnet45 with the beta header (dropped by the filter)

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model": "bedrock-sonnet45", "max_tokens": 256, "messages": [{"role": "user", "content": "Reply with the single word ok"}]}'

{"id": "msg_bdrk_011CfBRVofdJXwjrCKdZ7b6x", "model": "bedrock-sonnet45", "stop_reason": "end_turn", "content_types": ["text"], "text": "ok"}
HTTP 200 (2.0s)

After: same request (header forwarded, Bedrock answers 400)

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: thinking-binding-controls-2026-08-01' \
  -d '{"model": "bedrock-sonnet45", "max_tokens": 256, "messages": [{"role": "user", "content": "Reply with the single word ok"}]}'

{"type": "error", "error": {"type": "invalid_request_error", "message": "litellm.BadRequestError: BedrockException - {\"message\":\"invalid beta flag\"}. Received Model Group=bedrock-sonnet45\nAvailable Model Group Fallbacks=None"}}
HTTP 400 (0.7s)

Before: `/v1/chat/completions` -> bedrock-sonnet45 with `extra_headers`

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:40307/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json'  \
  -d '{"model": "bedrock-sonnet45", "max_tokens": 256, "messages": [{"role": "user", "content": "Reply with the single word ok"}], "extra_headers": {"anthropic-beta": "thinking-binding-controls-2026-08-01"}}'

{"id": "chatcmpl-ed50fd7f-0aeb-41ec-a4bf-da5105b70083", "model": "bedrock-sonnet45", "finish_reason": "stop", "content": "ok", "has_reasoning": false}
HTTP 200 (1.8s)

After: same request

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json'  \
  -d '{"model": "bedrock-sonnet45", "max_tokens": 256, "messages": [{"role": "user", "content": "Reply with the single word ok"}], "extra_headers": {"anthropic-beta": "thinking-binding-controls-2026-08-01"}}'

{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: invalid beta flag\"}. Received Model Group=bedrock-sonnet45\nAvailable Model Group Fallbacks=None", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400 (0.7s)

After: other listed betas the model lacks stay 200 on Bedrock

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: compact-2026-01-12,effort-2025-11-24,computer-use-2025-11-24' \
  -d '{"model": "bedrock-sonnet45", "max_tokens": 256, "messages": [{"role": "user", "content": "Reply with the single word ok"}]}'

{"id": "msg_bdrk_011CfBTCaaqUUNXWsDHdqzPo", "model": "bedrock-sonnet45", "stop_reason": "end_turn", "content_types": ["text"], "text": "ok"}
HTTP 200 (1.9s)

After: the CI e2e suite's request shape (every non-null `anthropic` header of this config on Sonnet 4.5, first-party)

curl -sS -w '\nHTTP %{http_code}\n' -X POST http://localhost:25461/v1/messages \
  -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' -H 'anthropic-version: 2023-06-01' -H 'anthropic-beta: advisor-tool-2026-03-01,advanced-tool-use-2025-11-20,code-execution-2025-08-25,compact-2026-01-12,computer-use-2025-01-24,computer-use-2025-11-24,context-1m-2025-08-07,context-management-2025-06-27,effort-2025-11-24,fast-mode-2026-02-01,files-api-2025-04-14,fine-grained-tool-streaming-2025-05-14,interleaved-thinking-2025-05-14,mcp-client-2025-11-20,mcp-client-2025-04-04,oauth-2025-04-20,output-128k-2025-02-19,prompt-caching-scope-2026-01-05,skills-2025-10-02,structured-outputs-2025-11-13,thinking-binding-controls-2026-08-01,token-efficient-tools-2025-02-19,web-fetch-2025-09-10,web-search-2025-03-05' \
  -d '{"model": "anthropic-sonnet45", "max_tokens": 10, "messages": [{"role": "user", "content": "Reply with the single word ok"}], "tools": [{"type": "code_execution_20250825", "name": "code_execution"}]}'

{"id": "msg_011CfBRXt94hkAsDipfqZFUZ", "model": "anthropic-sonnet45", "stop_reason": "end_turn", "input_transformations": [], "content_types": ["text"], "text": "ok"}
HTTP 200 (1.3s)

Every other case answered the same on both legs: bedrock/invoke/ chat (which forwards client betas unfiltered on both trees), Databricks Opus 5 (accepts the header, rejects block_binding in the body on both trees), first-party Sonnet 4.5 (accepts the header), an unknown beta (still dropped) and the no-beta regression guards. The merged tree (origin/main at a2626726a2 plus this tip, clean merge) answered identically to the tip on all 17 scenarios, the Vertex Sonnet 5 case included

Observations

After leg 400s without local flag: config comes from main
Remote-config proxies get it at next boot or /reload/anthropic_beta_headers
Local-flag deployments need a release; PR leaves alone
/v1/chat/completions and /v1/responses fixed too via extra_headers
Vertex Fable 5.1 quota 429 everywhere; proven on Sonnet 5
Claude Code omits binding beta over base URL; PR unrelated
Bedrock 400s binding beta on Sonnet 4.5; before was 200
Other listed betas on Sonnet 4.5 still 200 on Bedrock
bedrock/invoke/ chat forwards client betas unfiltered; PR unrelated
Databricks Opus 5 rejects block_binding body on both legs
Merged tree (origin/main plus tip) matches tip

Verdict: PASS. Before is HTTP 400 thinking.adaptive.block_binding: Extra inputs are not permitted and after is HTTP 200 with input_transformations: [] on Bedrock and first-party Anthropic across /v1/messages, /v1/chat/completions and /v1/responses, and the same 400 to 200 flip on Vertex AI /v1/messages with Claude Sonnet 5 at global, since the project has no Fable 5.1 quota on Vertex

Type

🐛 Bug Fix

Caveats (if any)

Medium

Out of scope here, filed as [Bug]: /v1/messages drops signature-only thinking blocks (Claude Opus 4.8 / Fable 5 / Fable 5.1 default shape), losing reasoning and defeating preserved-thinking checks #41361: the proxy's request-side sanitizer (is_empty_thinking_block, run on every /v1/messages request) drops thinking blocks whose text is empty even when they carry a valid signature. Claude Fable 5.1 returns exactly that shape by default (display: "omitted"), so replayed thinking never reaches the provider and the prefix check has nothing to bind, which is why the runs above show dropped: 0 even when an earlier turn is edited. This PR is the prerequisite for that fix: once the block is forwarded, block_binding needs the header or the provider 400s

Low

Bedrock now answers HTTP 400 invalid beta flag when a client sends anthropic-beta: thinking-binding-controls-2026-08-01 to a Bedrock model that does not implement it (Claude Sonnet 4.5 in the blast radius run above), where the merge base answered HTTP 200 after silently dropping the header. The 400 is Bedrock's own reply to a beta the client asked for: first-party Anthropic accepts the same header on Sonnet 4.5, Databricks ignores it, Bedrock still accepts the other listed betas the model lacks (compact-2026-01-12, effort-2025-11-24, computer-use-2025-11-24 all 200), and neither Claude Code nor the Anthropic SDK sends this beta on its own. Left as is: a per-model allowlist for one header would be the only per-model rule in the config, would go stale the day Bedrock accepts the flag on more models, and would keep silently downgrading a feature the client explicitly asked for. Deployments on the default remote config pick this up at their next boot after the merge, or earlier through POST /reload/anthropic_beta_headers, together with the fix
azure_ai is mapped to null (dropped, as today) because I could not verify the header on Foundry; Anthropic states the same beta names apply on Bedrock and Google Cloud, which are the providers mapped here
The Vertex AI leg of the proof runs on Claude Sonnet 5 at global rather than the Claude Fable 5.1 deployment the issue names, because the QA project has no Fable 5.1 quota on Vertex (HTTP 429 in three regions, direct calls included). The mapping is per provider, not per model, so the same vertex_ai lookup is what forwarded the header; what this run did not observe is Vertex's own reply to the header on Fable 5.1, which the issue reporter's 400 and the author's earlier run in this PR's history both show

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
c19a199 passes /live-pr-risk

CLAassistant · 2026-09-15T07:19:55Z

All committers have signed the CLA.

greptile-apps · 2026-09-15T07:21:58Z

Confidence Score: 5/5

The PR appears safe to merge because the configuration change is focused and covered across all enabled providers

Summary

Registers the thinking-binding-controls-2026-08-01 Anthropic beta header for supported providers so preserved-thinking requests retain the required opt-in during filtering

Forwards the header for Anthropic, Bedrock, Bedrock Converse, Vertex AI, and Databricks
Continues filtering the unverified Azure AI mapping
Adds a parametrized regression test covering each enabled provider

_{Reviews (2) · Last reviewed commit: "fix(anthropic): register thinking-bindin..."}

codspeed · 2026-09-15T07:22:04Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing clonylu:fix/anthropic-thinking-binding-beta-header (c19a199) with main (feab83a)¹}

No successful run was found on main (81863c1) during the generation of this report, so feab83a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

codecov · 2026-09-15T07:28:38Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

… headers config Anthropic's preserved-thinking controls (`thinking.block_binding`, Claude Fable 5.1) are only accepted alongside the beta header `thinking-binding-controls-2026-08-01`. The proxy forwards the body field untouched but `filter_and_transform_beta_headers` drops the header because it has no entry in `anthropic_beta_headers_config.json`, so Bedrock and Vertex reject the request with "thinking.adaptive.block_binding: Extra inputs are not permitted". Map the header for anthropic, bedrock, bedrock_converse, vertex_ai and databricks (same beta name on all of them per Anthropic's docs). azure_ai is left null pending verification on Foundry.

mateo-berri · 2026-09-18T18:14:09Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c19a199. Configure here.}

mateo-berri · 2026-09-18T18:36:02Z

@greptileai

mateo-berri

LGTM. Thanks for the contribution!

clonylu requested a review from a team September 15, 2026 07:19

clonylu force-pushed the fix/anthropic-thinking-binding-beta-header branch from 1b50ef8 to c19a199 Compare September 15, 2026 07:34

cursor Bot reviewed Sep 18, 2026

View reviewed changes

mateo-berri approved these changes Sep 18, 2026

View reviewed changes

mateo-berri merged commit a7a3f68 into BerriAI:main Sep 18, 2026
93 of 94 checks passed

yuneng-berri mentioned this pull request Sep 19, 2026

test(e2e): stop sending model-scoped anthropic betas at models without them #41927

Closed

5 of 15 tasks

devin-ai-integration Bot mentioned this pull request Sep 19, 2026

fix(test): unbreak the integration-cost and proxy_e2e_anthropic_messages CircleCI jobs on main #42048

Merged

3 of 5 tasks

clonylu mentioned this pull request Sep 24, 2026

fix(responses): emit the reasoning item on streaming /v1/responses for signature-only thinking #42871

Closed

4 of 6 tasks

Archit-Kumar-16 mentioned this pull request Sep 24, 2026

fix(aws-bedrock,aws-bedrock-mantle,google-vertex): allow thinking-binding-controls beta truefoundry/models#3078

Merged

deepanshululla mentioned this pull request Oct 6, 2026

[Feature]: additive registration of anthropic-beta headers via config, instead of shadowing the whole registry #44846

Open

Navigation Menu

Repository navigation

fix(anthropic): register thinking-binding-controls-2026-08-01 in beta headers config - #41203

fix(anthropic): register thinking-binding-controls-2026-08-01 in beta headers config#41203
mateo-berri merged 1 commit into
BerriAI:mainfrom
clonylu:fix/anthropic-thinking-binding-beta-header

clonylu commented Sep 15, 2026 •

edited by mateo-berri

Loading

Uh oh!

CLAassistant commented Sep 15, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 15, 2026

Uh oh!

mateo-berri commented Sep 18, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

mateo-berri commented Sep 18, 2026

Uh oh!

mateo-berri left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

clonylu commented Sep 15, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (merge base 15bd8b0e4a, port 40307)

/v1/messages -> bedrock-fable

/v1/messages -> vertex-fable

/v1/messages -> anthropic-fable

/v1/chat/completions -> bedrock-fable

/v1/chat/completions -> anthropic-fable

/v1/responses -> anthropic-fable

After (c19a19999f, port 25461)

/v1/messages -> bedrock-fable

/v1/messages -> vertex-fable

/v1/messages -> anthropic-fable

/v1/chat/completions -> bedrock-fable

/v1/chat/completions -> anthropic-fable

/v1/responses -> anthropic-fable

Vertex AI

Before (merge base 15bd8b0e4a, port 40307): /v1/messages -> vertex-sonnet5-global

After (c19a19999f, port 25461): same request

Claude Code

Blast radius

Before: /v1/messages -> bedrock-sonnet45 with the beta header (dropped by the filter)

After: same request (header forwarded, Bedrock answers 400)

Before: /v1/chat/completions -> bedrock-sonnet45 with extra_headers

After: same request

After: other listed betas the model lacks stay 200 on Bedrock

After: the CI e2e suite's request shape (every non-null anthropic header of this config on Sonnet 4.5, first-party)

Observations

Type

Caveats (if any)

Medium

Low

Final Attestation

Uh oh!

CLAassistant commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

codspeed Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

codecov Bot commented Sep 15, 2026

Codecov Report

Uh oh!

mateo-berri commented Sep 18, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented Sep 18, 2026

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

clonylu commented Sep 15, 2026 •

edited by mateo-berri

Loading

Before (merge base `15bd8b0e4a`, port 40307)

After (`c19a19999f`, port 25461)

Before (merge base `15bd8b0e4a`, port 40307): `/v1/messages` -> vertex-sonnet5-global

After (`c19a19999f`, port 25461): same request

Before: `/v1/messages` -> bedrock-sonnet45 with the beta header (dropped by the filter)

Before: `/v1/chat/completions` -> bedrock-sonnet45 with `extra_headers`

After: the CI e2e suite's request shape (every non-null `anthropic` header of this config on Sonnet 4.5, first-party)

CLAassistant commented Sep 15, 2026 •

edited

Loading

greptile-apps Bot commented Sep 15, 2026 •

edited

Loading

codspeed Bot commented Sep 15, 2026 •

edited

Loading