fix(bedrock-passthrough)!: return AWS-shaped errors so AWS SDKs read the code and message

[yucheng-berri]

TLDR

Problem this solves:

• AWS SDK apps on the Bedrock passthrough get errors with no code and no message
• A guardrail block reads ClientError (400) with an empty message
• A Bedrock 429 reads (429) with an empty message, and the SDK never retries it

How it solves it:

• A guardrail block adds x-amzn-errortype: ValidationException and a top-level message
• An error Bedrock returned is passed through with its own status, error type and body
• Other LiteLLM errors (auth, budget, rate limits, unknown model) are unchanged

Intentional product change: AWS SDKs in standard or legacy retry mode now retry Bedrock throttling through the proxy, as they do against Bedrock directly

User Flow

Before: an app using the AWS SDK through the proxy gets errors with no code and no message

1. The app calls converse or converse_stream with endpoint_url=https://litellm-domain/bedrock and a LiteLLM key
2. A guardrail on the key blocks the prompt, or the reply before the first frame
3. The SDK raises ClientError: An error occurred (400) when calling the Converse operation: with an empty message
4. When Bedrock rejects or throttles the request, the SDK raises (400) or (429) with an empty message and does not retry the 429

After: the same app sees the errors it would get from Bedrock

1. The app makes the same call
2. A guardrail on the key blocks the prompt, or the reply before the first frame
3. The SDK raises ValidationException with the guardrail's message, e.g. Violated guardrail policy
4. When Bedrock rejects or throttles the request, the SDK raises Bedrock's own ValidationException or ThrottlingException with Bedrock's message, after its normal retries

Linear ticket

Resolves LIT-9617

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• Every automated review on my PR's latest commit has finished, and I have addressed or replied to each of its findings before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup: proxy with 4 workers and Postgres, every edge real, no stand-ins. Model claude-haiku = bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0 and model pixtral = bedrock/us.mistral.pixtral-large-2502-v1:0, both on real Bedrock (us-east-1). Throttling is real Bedrock throttling: a script sends large requests straight to AWS until this account's Pixtral Large token quota is used up, then the client call goes through the proxy. Keys: one with a real AWS Bedrock guardrail as pre_call (word zzzforbidden blocked), one with a real Akto guardrail as post_call (its policy blocks emails in replies), one with no guardrail. Clients: AWS CLI 1.45.41 with AWS_BEARER_TOKEN_BEDROCK=<LiteLLM key> and --endpoint-url http://<proxy>/bedrock, and boto3 1.43.1 converse_stream

The same cases on the direct model id route (--model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 with a us.anthropic.* deployment), and with a real Noma guardrail as pre_call or during_call, give the same Before and After

Before (2d4472e)

Guardrail blocks the prompt, AWS CLI converse

1. aws bedrock-runtime converse --endpoint-url http://127.0.0.1:4982/bedrock --model-id claude-haiku --messages '[{"role":"user","content":[{"text":"tell me about zzzforbidden"}]}]' with the Bedrock guardrail key
2. An error occurred (400) when calling the Converse operation: with no message

Bedrock rejects the request, AWS CLI converse

1. The same command with the no-guardrail key, "text":"hi" and --inference-config temperature=5
2. An error occurred (400) when calling the Converse operation: with no message

Bedrock throttles, AWS CLI converse

1. Use up the Pixtral Large token quota, then AWS_RETRY_MODE=standard AWS_MAX_ATTEMPTS=3 aws bedrock-runtime converse ... --model-id pixtral --messages file://big-prompt.json with the no-guardrail key
2. An error occurred (429) when calling the Converse operation: with no message
3. The key has 1 spend log row for the call, so the CLI did not retry

Guardrail blocks the reply before the first frame, boto3 converse_stream

1. client.converse_stream(modelId="claude-haiku", ...) asking for a reply with an email address, with the Akto key
2. ClientError: An error occurred (403) when calling the ConverseStream operation:, Code='403' Message=''

The proxy admin opens the blocked request in the Logs page

1. http://127.0.0.1:4982/ui/?page=logs, search the blocked request's id, open the failure row
2. Failure, error code 403, message [Block] Sorry, the model cannot answer this question. This has been blocked by PII-Data policy., guardrail akto-post

After (49056bd)

Guardrail blocks the prompt, AWS CLI converse

1. The same command
2. An error occurred (ValidationException) when calling the Converse operation: Violated guardrail policy

Bedrock rejects the request, AWS CLI converse

1. The same command
2. An error occurred (ValidationException) when calling the Converse operation: 1 validation error detected: Value '5.0' at 'inferenceConfig.temperature' failed to satisfy constraint: Member must have value less than or equal to 1
3. The same request sent straight to AWS prints the identical line

Bedrock throttles, AWS CLI converse

1. The same steps
2. An error occurred (ThrottlingException) when calling the Converse operation (reached max retries: 2): Too many tokens, please wait before trying again.
3. The key has 3 spend log rows for the call, so the CLI retried twice
4. The same call sent straight to AWS prints the identical line

Guardrail blocks the reply before the first frame, boto3 converse_stream

1. The same call
2. ClientError: An error occurred (ValidationException) when calling the ConverseStream operation: [Block] Sorry, the model cannot answer this question. This has been blocked by PII-Data policy., Code='ValidationException' with the same message

The proxy admin opens the blocked request in the Logs page

1. http://127.0.0.1:4981/ui/?page=logs, search the blocked request's id, open the failure row
2. The same Failure, error code 403, message and guardrail akto-post: logging is unchanged

Type

🐛 Bug Fix

Behavior changes

Removed / renamed

• None

Changed on /bedrock/model/<model>/... (converse, converse-stream, invoke, invoke-with-response-stream)

• Guardrail block before any frame: same status and error object, plus a top-level message and x-amzn-errortype: ValidationException
• Error a Bedrock model call returned with an x-amzn-errortype: Bedrock's own status, body, x-amzn-errortype and content-type, replacing {"detail": {"error": "<Bedrock body as a string>"}}
• AWS SDKs in standard or legacy retry mode read ThrottlingException and retry, so one client call can reach Bedrock up to the SDK's max attempts

Unchanged

• LiteLLM's own errors: auth, budget, rate limits, unknown model, guardrail vendor outages, timeouts
• A guardrail service's own HTTP error, even one carrying x-amzn-errortype
• Other providers' passthrough routes, the /bedrock agent-runtime routes, count-tokens
• Successful requests and streams

Caveats (if any)

Medium

• Guardrails that block with a plain-string error or a ValueError keep LiteLLM's body
  • Onyx, Prompt Security, Cato, DynamoAI, Enkrypt, IBM, Aim, Javelin, legacy custom-logger hooks
  • nothing marks those errors as a guardrail block today
• Guardrail vendor failures labeled like a block (XecGuard unreachable, Purview 400s) come back as ValidationException

Low

• Bedrock's request id reaches the client as llm_provider-x-amzn-requestid, so boto3's RequestId stays empty
• count-tokens and malformed /bedrock paths keep {"detail": ...}
• Touches the same file as fix(bedrock)!: stream Converse passthrough frames through post_call guardrails instead of buffering #46307, so whichever merges second rebases

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Ran /live-pr-risk at 49056bd with real edges only (real Bedrock and real Bedrock throttling, real AWS Bedrock guardrail, Akto and Noma, 4 workers, base 2d4472e vs head): no regressions. The backward-incompatible changes it found are exactly the ones listed in Behavior changes. A 40-request burst with a worker killed mid-burst behaved as on base apart from the intended error shapes, with the same failure-row counts per key, over eight rounds per leg at b6c0e20 and two more on 49056bd. The paused-Postgres cell ran at bd884cb, before later commits that do not touch that path

REVIEWER MUST KNOW BEFORE APPROVING

• A guardrail block on a Bedrock passthrough route keeps its status and error object and gains a top-level message plus x-amzn-errortype: ValidationException
• An error Bedrock returned is now Bedrock's raw body with its x-amzn-errortype and content-type, replacing {"detail": {"error": "<string>"}}, so a client that parsed detail breaks
• AWS SDK retries now fire on Bedrock throttling through the proxy, so one client call can hit Bedrock up to the SDK's max attempts, as it does against Bedrock directly

---

Note

Medium Risk
Changes Bedrock passthrough error response contracts (raw Bedrock bodies vs detail wrapping) and enables AWS SDK retries on throttling; guardrail vendor failures may be surfaced as ValidationException in edge cases.

Overview
Bedrock passthrough errors are reshaped so boto3/AWS CLI can read x-amzn-errortype and a top-level message, instead of empty ClientError codes/messages.

Passthrough HTTP failures from the upstream provider are tagged via raise_for_provider_status / is_provider_error. On /bedrock/model/..., real Bedrock errors (with x-amzn-errortype) are returned with Bedrock’s status, body, and headers; guardrail blocks (pre/post-call, PII, stamped HTTPException details) become ValidationException through AwsErrorTypeProxyException and a dedicated FastAPI handler that merges LiteLLM’s error payload with AWS fields. Auth, budget, proxy rate limits, unknown models, and non-Bedrock passthrough routes stay on the existing LiteLLM error shape.

Intentional behavior: SDK retry logic now sees ThrottlingException on proxied Bedrock 429s, so one client call may hit upstream multiple times (same as calling Bedrock directly).

^{Reviewed by Cursor Bugbot for commit 49056bd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[yucheng-berri]

@litereview code-review

[devin-ai-integration]

Devin Review found 1 potential issue.

[litereview]

✅ code-review

5/5 · Ready for human review

Scope: full · Full PR review.

Completed · Some checks not run.

No correctness findings.

The change adds x-amzn-errortype and a top-level message only for stamped guardrail blocks, GuardrailRaisedException(blocked_content=True) and BlockedPiiEntityError when the handled code matches. It returns Bedrock's raw body only when the provider recorded the httpx.HTTPStatusError through raise_for_provider_status and Bedrock sent x-amzn-errortype. Other errors still re-raise through handle_llm_api_exception.

A post_call_failure_hook can replace the error with an HTTPException. That replacement becomes a ProxyException, so the raw-body branch never overrides the callback's response. The title now has fix(bedrock-passthrough)!:, and the description explains the impact and migration.

Checks and execution details

• Passed · Real testing evidence (required): Not applicable: no LLM translation changes in the scoped paths. The PR changes litellm/constants.py, litellm/passthrough/main.py, litellm/proxy/_types.py, litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py, litellm/proxy/proxy_server.py and tests. It touches nothing under litellm/llms/** or litellm/litellm_core_utils/prompt_templates/**.
• Passed · New and changed unit tests (optional): uv run pytest tests/unit/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py tests/unit/proxy/test_proxy_server_endpoints_and_startup.py -k "bedrock or aws" ran 73 tests, all passed. The 10 test functions this PR adds or names (7 matched) also passed.
• Passed · WeakSet provider-error tagging (optional): Confirmed that httpx.HTTPStatusError supports weak references, so weakref.WeakSet.add in raise_for_provider_status does not raise. A plain Exception would raise TypeError there.
• Not run · Integration suite tests/integration/providers/test_bedrock_passthrough_aws_errors.py (optional): Not run locally. The test needs the CircleCI integration harness: a running proxy, Postgres and the INTEGRATION_PROXY_URL and INTEGRATION_UPSTREAM_URL environment. I reviewed the test source. It covers the router and direct routes, boto3 retries on ThrottlingException, and keeps the LiteLLM error shape for auth, budget, rate-limit, unknown-model, vendor-down and Cato errors.

---

View review · Reviewed 49056bd12811

[codspeed]

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• test_completion_streaming

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_passthrough_aws_errors (49056bd) with main (5570d42)¹}

Footnotes

1. No successful run was found on main (cb7e6a7) during the generation of this report, so 5570d42 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

@litereview code-review

[yucheng-berri]

bugbot run

[codecov]

Codecov Report

❌ Patch coverage is 98.18182% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ass_through_endpoints/llm_passthrough_endpoints.py	96.66%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[cursor]

Stale Bugbot comment from a previous run.

[yucheng-berri]

@litereview full

[litereview]

✅ full

Scope: full · Full PR review.

Completed · Some checks not run.

Verdict: no actionable defects found

I didn't find a failure I could pin to the changed code. The change is limited to the two Bedrock model-operation handlers, which both now catch errors through bedrock_passthrough_error_response (llm_passthrough_endpoints.py:1216 and :1416). The registered exception handler only fires for an exception type that is new in this PR.

This verdict comes from reading the code and running the unit tests. I didn't run a proxy, so none of it was observed live.

Who uses what changed

• bedrock_passthrough_error_response is called only from handle_bedrock_passthrough_router_model and the direct-model branch of bedrock_llm_proxy_route. Those are reached through /bedrock/{endpoint} when the path is not an agent-runtime route. The agent-runtime route and count-tokens are untouched.

• The Bedrock pass-through branch (_bedrock_error_response) only acts on an httpx.HTTPStatusError when all three hold:

  • the request URL matches /model/<id>/<op>;
  • Bedrock sent x-amzn-errortype;
  • the shared handler already turned the error into an HTTPException.

  The error reaches it unchanged through allm_passthrough_route (passthrough/main.py:351). Streaming responses are read in full before they raise (AsyncPassthroughStreamingResponse.__await__), so upstream.content is safe to read there.

• The guardrail branch (_guardrail_refusal_status) only adds the AWS error type when the guardrail's own status matches the ProxyException.code the shared handler produced. Fail-closed vendors, 429s from vendors, ModifyResponseException and request-check HTTPExceptions still go through openai_exception_handler unchanged. The new tests cover each of these cases.

• aws_error_type_exception_handler builds on openai_exception_handler and adds only a top-level message and the x-amzn-errortype header. The error object and the call-id headers are preserved.

Contract note (deliberate, not a defect)

When Bedrock itself rejects a request on /bedrock/model/..., the response body is now Bedrock's raw JSON (for example {"message": ...}). Before, it was {"detail": {"error": "<bedrock text>"}}, which came from the HTTPException built at common_request_processing.py:3879.

• A non-AWS client that parsed detail.error would notice the change.
• The PR description says this pass-through is intentional, and AWS SDK clients are the intended consumers.
• AWS SDKs in standard or legacy retry mode will now retry throttling, which the PR also calls intentional.
• I'm not raising this as a defect. If maintainers count it as a breaking change, the title has no ! marker.

Verification

• I installed dependencies with uv sync --frozen --all-extras --all-groups.
• I ran the Bedrock and guardrail pass-through unit tests: 53 passed.

Not verified

• Live base-vs-head proxy comparison: I didn't run it. There was no Postgres, no AWS Bedrock credentials and no AWS SDK client rig in this sandbox. Real Bedrock throttling and real guardrail blocks seen through boto3 are covered only by the PR's own evidence.
• Router retries and fallbacks: I didn't check what happens when a router retry or fallback wraps the HTTPStatusError in another exception. In that case the code falls back to the old behaviour rather than breaking, which I worked out from the code.
• Merged-tree check: I didn't check how the PR behaves merged onto current main; the checkout has no git history.

Checks and execution details

• Passed · Dependency graph of changed symbols (required): I grepped litellm/ and enterprise/ for bedrock_passthrough_error_response, AwsErrorTypeProxyException and aws_error_type_exception_handler. They are reached only from the two Bedrock model-route handlers and the FastAPI exception registry. I traced how HTTPStatusError gets there through litellm/passthrough/main.py and handle_llm_api_exception.
• Passed · Bedrock/guardrail passthrough unit tests (optional): uv run --no-sync pytest tests/unit/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py -k "Bedrock or bedrock or guardrail": 53 passed.
• Not run · Live base vs head proxy A/B (real Postgres, real Bedrock, AWS SDK client) (optional): Not run: the sandbox has no AWS Bedrock credentials, no Postgres and no AWS SDK client rig. The live behaviour rests on the PR's own before/after evidence.
• Not run · Merge-ref drift re-run (optional): Not run: the checkout is a source archive with no git history, so I couldn't build a tree merged onto current main.

---

View review · Reviewed b6c0e2040248

[ryan-crabbe-berri]

lgtm

[ryan-crabbe-berri]

non blocking nit: guessing bedrock vs guardrail from the url feels fragile, could the passthrough call tag its own errors as bedrock instead

[greptile-apps]

Automatic reviews are disabled for this repository.

[yucheng-berri]

@litereview code-review

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 49056bd. Configure here.}

[yucheng-berri]

@litereview code-review

[yucheng-berri]

@litellm-stampbot

[litellm-stampbot]

litellm-stampbot: risk low

Approved at 49056bd.

Rules (stampbot/risk_rules.yml):

• low: no rule flagged it

Judge (gpt-6-luna): low, probability of high 0.28 (high at or above 0.75)

• risk: low (probability of high 0.28)

Pushes do not re-run this check. Comment @litellm-stampbot again to re-check the new head.

[litellm-stampbot]

Low risk by rule and by judgement at 49056bd