fix(bedrock): sign rerank requests with the shared header-filtered SigV4 helper (internal copy of #36462)

[mateo-berri]

Internal copy of #36462 by @noahnistler (fork head d80608eca6), whose organization-owned fork maintainers cannot push to. It carries both original commits with authorship preserved, plus one commit fixing a bearer-token regression review found there: full credit for the diagnosis and the fix approach goes to the original author

TLDR

Problem this solves:

• Bedrock rerank calls fail with a SigV4 signature mismatch when the proxy forwards client headers
• The rerank handler signed every header instead of only the ones AWS verifies

How it solves it:

• Routes rerank signing through the shared get_request_headers helper other Bedrock handlers use
• The helper filters headers before signing and re-attaches the rest unsigned
• Rerank opts out of Bedrock API-key bearer auth, which its AWS endpoint rejects, staying on SigV4

User Flow

Before: an admin whose proxy forwards client headers to backend LLM APIs gets every Bedrock rerank call rejected by AWS

1. The proxy admin enables forward_client_headers_to_llm_api: true and adds a Bedrock rerank deployment, e.g. bedrock/cohere.rerank-v3-5:0
2. A user's app sends POST https://litellm-domain/v1/rerank with a query, a list of documents, and ordinary client headers such as X-Forwarded-For
3. The call comes back as an error carrying AWS's text: "The request signature we calculated does not match the signature you provided"

After: the same call returns ranked results

1. The proxy admin enables forward_client_headers_to_llm_api: true and adds the same Bedrock rerank deployment
2. The user's app sends the same POST https://litellm-domain/v1/rerank with the same headers
3. The response is 200 with a results array of index and relevance-score pairs, and setups that authenticate with a Bedrock API key (AWS_BEARER_TOKEN_BEDROCK) keep working unchanged

Relevant issues

Linear ticket

Resolves LIT-5458

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live before/after against real AWS Bedrock in us-west-2, no mocks. Each side boots a proxy with 2 uvicorn workers from that exact commit, with all egress routed through a header-rewriting hop (mitmdump with an addon that appends its own address to any X-Forwarded-For it forwards, the standard behavior of a real corporate egress proxy, wired in via HTTPS_PROXY), which is the exact condition that breaks the old signing. Config on both sides:

model_list:
  - model_name: bedrock-rerank
    litellm_params:
      model: bedrock/arn:aws:bedrock:us-west-2::foundation-model/cohere.rerank-v3-5:0
      aws_region_name: us-west-2
      aws_profile_name: default
  - model_name: bedrock-chat
    litellm_params:
      model: bedrock/us.amazon.nova-micro-v1:0
      aws_region_name: us-west-2
      aws_profile_name: default
  - model_name: bedrock-embed
    litellm_params:
      model: bedrock/amazon.titan-embed-text-v2:0
      aws_region_name: us-west-2
      aws_profile_name: default

general_settings:
  master_key: sk-lit5458-qa
  forward_client_headers_to_llm_api: true

Shared request bodies:

RERANK='{"model":"bedrock-rerank","query":"capital of the United States","documents":["Carson City is the capital city of Nevada.","Washington, D.C. is the capital of the United States."],"top_n":1}'
CHAT='{"model":"bedrock-chat","messages":[{"role":"user","content":"Reply with the single word pong"}],"max_tokens":5}'
EMBED='{"model":"bedrock-embed","input":"forwarded header signing check"}'

Before (bb27bfd)

rerank with X-Forwarded-For through the egress hop

1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:45600/v1/rerank -H 'Authorization: Bearer sk-lit5458-qa' -H 'Content-Type: application/json' -H 'X-Forwarded-For: 10.1.2.3' -d "$RERANK"
2. {"error": {"message": "{\"message\":\"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. ...\"}. Received Model Group=bedrock-rerank ...", "code": "403"}} with HTTP 403

unversioned /rerank with X-Forwarded-For through the egress hop

1. Same request against http://127.0.0.1:45600/rerank
2. The same signature-mismatch error, HTTP 403

rerank with AWS_BEARER_TOKEN_BEDROCK set in the proxy env

1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:45600/v1/rerank -H 'Authorization: Bearer sk-lit5458-qa' -H 'Content-Type: application/json' -d "$RERANK" with a valid Bedrock API key exported as AWS_BEARER_TOKEN_BEDROCK
2. {"results": [{"index": 1, "relevance_score": 0.8750309944152832}], "usage": null} with HTTP 200: the old inline signing ignored the bearer
3. With AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-deliberately-invalid instead, rerank still returns the same 200, confirming the bearer never reaches rerank's signing

chat completions and embeddings ride the same shared helper

1. curl ... -X POST http://127.0.0.1:45600/v1/chat/completions -H 'X-Forwarded-For: 10.1.2.3' -d "$CHAT" returns {"content": "Pong"} HTTP 200, and curl ... -X POST http://127.0.0.1:45600/v1/embeddings -H 'X-Forwarded-For: 10.1.2.3' -d "$EMBED" returns a 1024-dim embedding, HTTP 200
2. With the valid bearer exported both still return 200; with the invalid bearer both fail with AWS's {"Message":"Invalid API Key format: Base64 decoding failed"}, proving they authenticate with the bearer when it is set

After (e47e989)

rerank with X-Forwarded-For through the egress hop

1. curl -sS -w '\nHTTP %{http_code}\n' -X POST http://127.0.0.1:47755/v1/rerank -H 'Authorization: Bearer sk-lit5458-qa' -H 'Content-Type: application/json' -H 'X-Forwarded-For: 10.1.2.3' -d "$RERANK"
2. {"results": [{"index": 1, "relevance_score": 0.8750309944152832}], "usage": null} with HTTP 200

unversioned /rerank with X-Forwarded-For through the egress hop

1. Same request against http://127.0.0.1:47755/rerank
2. The same 200 with real relevance scores

rerank with AWS_BEARER_TOKEN_BEDROCK set in the proxy env

1. The same bearer-in-env rerank call returns the same 200 with real scores, and the deliberately invalid bearer changes nothing: rerank stays on SigV4
2. At the intermediate commit d80608eca6 (the original fork commits without the follow-up bearer commit), this exact request instead fails HTTP 403 with AWS's {"message":"Authorization header requires 'Credential' parameter. ..."}, because the shared helper sent Authorization: Bearer ... to bedrock-agent-runtime, which rejects Bedrock API keys. The follow-up commit's supports_bearer_token=False is what keeps bearer-token setups working

chat completions and embeddings ride the same shared helper

1. The same chat and embeddings calls behave identically to Before: 200 through the forwarded header, 200 with the valid bearer, and the same Invalid API Key format failure with the invalid bearer, so bearer precedence on every other Bedrock path is unchanged

Type

🐛 Bug Fix

Caveats (if any)

• Same bug class was fixed for the Bedrock invoke path in [Fix] Claude Code (/messages) - Litellm fix claude code Bedrock Invoke usage, request signing  #19111, never ported to rerank
• The shared helper honors a deliberate extra_headers Authorization over SigV4, now for rerank too (parity with converse and embeddings; header forwarding never reaches it, as the proof shows)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• e47e989341 passes /live-pr-risk: the only contract change is a new defaulted supports_bearer_token param on get_request_headers, whose only override lives in base_aws_llm.py itself; converse and embedding callers were driven live on both sides above, and image generation and image edit keep the default, leaving their behavior byte-identical

---

Note

Medium Risk
Changes AWS auth and SigV4 signing for rerank only; incorrect header filtering or bearer handling could break proxy rerank deployments, but scope is isolated and covered by new tests.

Overview
Fixes Bedrock rerank failing with SigV4 signature mismatches when the proxy forwards client headers (e.g. X-Forwarded-For), by routing request prep through the shared get_request_headers helper used elsewhere on Bedrock instead of signing every header inline.

The helper only includes AWS-relevant headers in the signature and re-attaches forwarded headers unsigned, matching the invoke-path fix. Rerank calls get_request_headers(..., supports_bearer_token=False) because rerank hits bedrock-agent-runtime, which rejects Bedrock API-key bearer auth—so AWS_BEARER_TOKEN_BEDROCK no longer overrides SigV4 for rerank only; chat/embed/converse behavior stays on the default bearer path.

Adds unit tests that x-forwarded-for is present on the request but excluded from SignedHeaders, and that rerank still uses AWS4-HMAC-SHA256 when the bearer env var is set.

^{Reviewed by Cursor Bugbot for commit e47e989. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

The PR routes Bedrock rerank request signing through the shared header-filtered SigV4 helper while explicitly retaining SigV4 authentication for the agent-runtime endpoint.

• Adds a defaulted switch controlling whether the shared helper may use Bedrock bearer tokens.
• Excludes mutable forwarded headers from the rerank signature while transmitting them unchanged.
• Adds regression coverage for forwarded headers and bearer-token configuration.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/base_aws_llm.py	Adds an opt-out for bearer-token authentication while preserving the existing default for all other callers.
litellm/llms/bedrock/rerank/handler.py	Replaces inline rerank signing with the shared filtered-header SigV4 preparation path and disables unsupported bearer authentication.
tests/test_litellm/llms/bedrock/rerank/test_bedrock_rerank_header_forwarding.py	Adds focused tests confirming forwarded headers remain unsigned and rerank continues using SigV4 when a Bedrock bearer token is configured.

_{Reviews (3): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ed8480a. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit5458_rerank_sigv4_bearer_fix (e47e989) with litellm_internal_staging (bb27bfd)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (bb22742) during the generation of this report, so bb27bfd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ec03baa. Configure here.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e47e989. Configure here.}

[shivamrawat1]

this will be in 1.100.0.rc1 this sat