Skip to content

fix(exceptions): map upstream status codes for providers with no exception_type branch - #38318

Merged
mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_fix_branchless_provider_status_mapping
Aug 26, 2026
Merged

fix(exceptions): map upstream status codes for providers with no exception_type branch#38318
mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_fix_branchless_provider_status_mapping

Conversation

@mateo-berri

@mateo-berri mateo-berri commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Providers with no exception_type branch answered every upstream error as a 500
  • A bad MiniMax key came back as APIConnectionError, code 500, so clients kept retrying
  • OpenAI-like providers (databricks, watsonx, ...) raised AuthenticationError for an upstream 403

How it solves it:

  • The status-blind fallback now maps by upstream status like the typed providers do
  • 401, 403, 404, 408, 429, 5xx get their typed class; any other 4xx is BadRequestError
  • APIConnectionError is only raised when the exception carries no status at all
  • OpenAI-like 403 now raises PermissionDeniedError
  • The router's single-deployment fail-fast now covers PermissionDeniedError like it does AuthenticationError

User Flow

Before: a developer with an invalid MiniMax key gets a 500 from the gateway, so their client keeps retrying a request that can never succeed

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "minimax-m2.5"
  2. The gateway answers HTTP 500 with litellm.APIConnectionError: MinimaxException - {... "http_code":"401" ...} and "code":"500"
  3. They send POST https://litellm-domain/v1/responses with the same model and get the same HTTP 500
  4. Their SDK treats the 500 as transient and retries, and every retry gets the same 500

After: the same requests come back as 401 authentication errors, so the client stops retrying and the developer fixes the key

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "minimax-m2.5"
  2. The gateway answers HTTP 401 with litellm.AuthenticationError: MinimaxException - {... "http_code":"401" ...} and "code":"401"
  3. They send POST https://litellm-domain/v1/responses with the same model and get the same HTTP 401
  4. Their SDK raises an authentication error right away and does not retry

Relevant issues

Linear ticket

Resolves LIT-6166

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for both legs: a proxy booted with python litellm/proxy/proxy_cli.py --config proxy_config.yaml --port <port> --num_workers 2, no database, against the real MiniMax and Anthropic APIs. minimax-m2.5 carries an invalid key on purpose (MiniMax answers 401 to any invalid key, no account needed). claude-haiku-4-5 is the paid control, claude-haiku-4-5-badkey is the typed-provider control for the same 401

model_list:
  - model_name: minimax-m2.5
    litellm_params:
      model: minimax/MiniMax-M2.5
      api_key: minimax-invalid-key-lit6166
  - model_name: claude-haiku-4-5
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: claude-haiku-4-5-badkey
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: sk-ant-invalid-key-lit6166

general_settings:
  master_key: sk-1234

Before (e52f055)

/v1/chat/completions with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: the upstream 401 comes back as a 500
    {"error":{"message":"litellm.APIConnectionError: MinimaxException - {\"type\":\"error\",\"error\":{\"type\":\"authorized_error\",\"message\":\"login fail: Please carry the API secret key in the 'Authorization' field of the request header (1004)\",\"http_code\":\"401\"},\"request_id\":\"06ddc9ba97ee6340e38f10e09787f547\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}
    HTTP 500
    

/v1/responses with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/responses -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","input":"hi","max_output_tokens":16}'
    
  2. Observed: same 500
    {"error":{"message":"litellm.APIConnectionError: MinimaxException - {\"type\":\"error\",\"error\":{\"type\":\"authorized_error\",\"message\":\"login fail: Please carry the API secret key in the 'Authorization' field of the request header (1004)\",\"http_code\":\"401\"},\"request_id\":\"06ddc9bfe4176d9ac7214a1db123680f\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}
    HTTP 500
    

/v1/messages with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/messages -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: this route already answered 401 (it does not go through the shared mapper), with the raw upstream JSON as the message
    {"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"authentication_error\",\"message\":\"login fail: Please carry the API secret key in the 'X-Api-Key' field of the request header\"},\"request_id\":\"06ddc9c2ea273bfc212258b20f5f8991\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"401"}}
    HTTP 401
    

Control: /v1/chat/completions with an invalid Anthropic key (typed provider)

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5-badkey","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: a typed provider already maps the 401
    {"error":{"message":"litellm.AuthenticationError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"authentication_error\",\"message\":\"API key is invalid.\"},\"request_id\":null}. Received Model Group=claude-haiku-4-5-badkey\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
    HTTP 401
    

Control: valid Anthropic key on all three endpoints

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":16}'
    
  2. Observed
    {"id":"chatcmpl-34327872-1344-49ce-a626-3ba57c88794b","created":1787729603,"model":"claude-haiku-4-5","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"ok","role":"assistant",...}}],"usage":{"completion_tokens":4,"prompt_tokens":13,"total_tokens":17,...}}
    HTTP 200
    
  3. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/responses -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","input":"Reply with the single word ok","max_output_tokens":16}'
    
  4. Observed
    {"id":"resp_lNzEEKpwd752_9UJX0miTE56...","created_at":1787729603,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"claude-haiku-4-5","object":"response","output":[{"type":"message","id":"msg_87ab1023-9543-410c-b2f7-276b19a762ce",...}],...}
    HTTP 200
    
  5. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:40712/v1/messages -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":16}'
    
  6. Observed
    {"model":"claude-haiku-4-5","id":"msg_011CeQyLWvNN576g5jzX3sF3","type":"message","role":"assistant","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":null,"stop_details":null,"usage":{"input_tokens":13,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,...,"output_tokens":4,"service_tier":"standard","inference_geo":"not_available"}}
    HTTP 200
    

After (6386a68)

/v1/chat/completions with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: the upstream 401 is now a 401 AuthenticationError
    {"error":{"message":"litellm.AuthenticationError: MinimaxException - {\"type\":\"error\",\"error\":{\"type\":\"authorized_error\",\"message\":\"login fail: Please carry the API secret key in the 'Authorization' field of the request header (1004)\",\"http_code\":\"401\"},\"request_id\":\"06ddd2520c26c72df2739c8dcad1c6de\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
    HTTP 401
    

/v1/responses with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/responses -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","input":"hi","max_output_tokens":16}'
    
  2. Observed: same 401 AuthenticationError
    {"error":{"message":"litellm.AuthenticationError: MinimaxException - {\"type\":\"error\",\"error\":{\"type\":\"authorized_error\",\"message\":\"login fail: Please carry the API secret key in the 'Authorization' field of the request header (1004)\",\"http_code\":\"401\"},\"request_id\":\"06ddd258ae4bbffd1e6667a5f66006c6\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
    HTTP 401
    

/v1/messages with an invalid MiniMax key

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/messages -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"minimax-m2.5","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: still 401; the litellm.AuthenticationError prefix here comes from fix(otel): map /v1/messages provider errors before failure logging #38310, which landed on the base and is part of this tip
    {"error":{"message":"litellm.AuthenticationError: MinimaxException - {\"type\":\"error\",\"error\":{\"type\":\"authentication_error\",\"message\":\"login fail: Please carry the API secret key in the 'X-Api-Key' field of the request header\"},\"request_id\":\"06ddd25fd2c7c39b870f7702696c2a8f\"}. Received Model Group=minimax-m2.5\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
    HTTP 401
    

Control: /v1/chat/completions with an invalid Anthropic key (typed provider)

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5-badkey","messages":[{"role":"user","content":"hi"}],"max_tokens":16}'
    
  2. Observed: unchanged
    {"error":{"message":"litellm.AuthenticationError: AnthropicException - {\"type\":\"error\",\"error\":{\"type\":\"authentication_error\",\"message\":\"API key is invalid.\"},\"request_id\":null}. Received Model Group=claude-haiku-4-5-badkey\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
    HTTP 401
    

Control: valid Anthropic key on all three endpoints

  1. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":16}'
    
  2. Observed
    {"id":"chatcmpl-60f10bda-5290-408c-8231-1913fb74286c","created":1787731820,"model":"claude-haiku-4-5","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"ok","role":"assistant",...}}],"usage":{"completion_tokens":4,"prompt_tokens":13,"total_tokens":17,...}}
    HTTP 200
    
  3. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/responses -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","input":"Reply with the single word ok","max_output_tokens":16}'
    
  4. Observed
    {"id":"resp__QWfhwXW_eRlC7OmxZ2ejr56RRCwXwHgqQ...","created_at":1787731829,"error":null,"incomplete_details":null,"instructions":null,"metadata":{},"model":"claude-haiku-4-5","object":"response","output":[{"type":"message","id":"msg_f46089ef-8447-4374-a792-be5e8d008c6f",...}],...}
    HTTP 200
    
  5. Run
    curl -s -w '\nHTTP %{http_code}\n' http://localhost:34566/v1/messages -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"claude-haiku-4-5","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":16}'
    
  6. Observed
    {"model":"claude-haiku-4-5","id":"msg_011CeR2Ayuexj7fJf1rDLeSN","type":"message","role":"assistant","content":[{"type":"text","text":"ok"}],"stop_reason":"end_turn","stop_sequence":null,"stop_details":null,"usage":{"input_tokens":13,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,...,"output_tokens":4,"service_tier":"standard","inference_geo":"not_available"}}
    HTTP 200
    

Observations from the run:

  • After leg: the 401 cools the MiniMax deployment down for 5s, so back-to-back calls got a 429 until it expired; cases were spaced 6s apart
  • Before leg never cooled it down: the router skips cooldown on APIConnectionError
  • /v1/messages answered 401 on both legs; its litellm.AuthenticationError prefix on the after leg comes from fix(otel): map /v1/messages provider errors before failure logging #38310 on the base, not this PR

Type

🐛 Bug Fix

Caveats (if any)

Low

  • A branchless provider's 401, 429, or 5xx now cools the deployment down like typed providers do
  • A 403 on a model group with a single deployment now fails immediately instead of being retried against it
  • With multiple deployments a 403 is still retried across them, but it never cools the failing deployment down, same as typed providers today
  • Unlisted 5xx map to APIError with the upstream status, matching the OpenAI branch
  • 422 maps to BadRequestError carrying status 422, matching the OpenAI branch
  • The OpenAI branch's own 403 still raises APIError; left alone, out of scope

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

  • 6386a68 passes /live-pr-risk


Note

Medium Risk
Changes global error mapping and HTTP codes returned to clients for many providers; behavior shifts from generic 500s to typed 4xx/5xx, which affects retries, cooldowns, and SDK handling but is intentional bug-fix scope.

Overview
Providers without a dedicated exception_type branch used to surface most HTTP errors as APIConnectionError with code 500, so invalid keys (e.g. MiniMax 401) looked transient and clients kept retrying.

This PR adds _map_exception_by_status on the generic fallback path so upstream 401, 403, 404, 408, 429, and 5xx map to the same typed LiteLLM exceptions as handled providers; APIConnectionError is only used when the exception has no status. OpenAI-like providers now raise PermissionDeniedError for 403 instead of AuthenticationError.

The router should_retry_this_error logic treats PermissionDeniedError like AuthenticationError (fail fast when only one deployment exists). Tests are updated across exception mapping, router retry, and several provider tests that now expect the correct typed errors.

Reviewed by Cursor Bugbot for commit 6386a68. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR preserves upstream HTTP status semantics for providers without dedicated exception branches and corrects OpenAI-like 403 classification.

  • Maps common upstream status codes to typed LiteLLM exceptions instead of defaulting to APIConnectionError.
  • Maps OpenAI-like 403 responses to PermissionDeniedError.
  • Extends the router’s single-deployment fail-fast guard to PermissionDeniedError.
  • Adds exception-mapping and router retry regression coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
litellm/litellm_core_utils/exception_mapping_utils.py Adds status-based fallback exception mapping and returns typed permission errors for upstream 403 responses.
litellm/router.py Extends the existing authentication fail-fast behavior to permission-denied errors, resolving the previously reported single-deployment retry path.
tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py Expands status-mapping coverage across providers and verifies MiniMax authentication failures retain their upstream 401 semantics.
tests/test_litellm/test_router.py Verifies permission-denied errors fail immediately for one deployment while remaining eligible for advancement when alternatives exist.

Reviews (2): Last reviewed commit: "fix(router): fail fast on PermissionDeni..." | Re-trigger Greptile

Comment on lines +764 to +770
elif original_exception.status_code == 403:
raise PermissionDeniedError(
message=f"{custom_llm_provider.capitalize()}Exception - {original_exception.message}",
llm_provider=custom_llm_provider,
model=model,
response=_response_or_stub(original_exception, status_code=403),
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 403 bypasses fail-fast

If an OpenAI-like provider is configured as a single healthy deployment, mapping its 403 to PermissionDeniedError bypasses the router's AuthenticationError-specific single-deployment guard and retries the permanent authorization failure against the same deployment, adding unnecessary upstream requests and delaying the client response.

Knowledge Base Used: Model request execution

…itellm_fix_branchless_provider_status_mapping

# Conflicts:
#	tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py
@codecov

codecov Bot commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.17949% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ellm/litellm_core_utils/exception_mapping_utils.py 86.84% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6386a68. Configure here.

@mateo-berri
mateo-berri enabled auto-merge August 26, 2026 08:42
@mateo-berri
mateo-berri merged commit 5640e7c into litellm_internal_staging Aug 26, 2026
82 checks passed
@mateo-berri
mateo-berri deleted the litellm_fix_branchless_provider_status_mapping branch August 26, 2026 08:46
@codspeed-hq

codspeed-hq Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_fix_branchless_provider_status_mapping (6386a68) with litellm_internal_staging (137311f)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (6386a68) during the generation of this report, so 137311f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

timothybrush pushed a commit to timothybrush/litellm that referenced this pull request Aug 28, 2026
BerriAI#38318 taught exception_type to map upstream status codes for providers with
no branch of their own. It reads the status code off the exception, but
_handle_error stamps 500 onto every failure that never carried one, so a
refused connection reached the mapper wearing a status code nothing upstream
had sent, and came back as InternalServerError instead of APIConnectionError.

The two are not interchangeable to a caller: a 5xx says the provider answered
and failed, which the router treats as a reason to cool the deployment down,
while a connection error says the request never landed.

BaseLLMException now records whether its status code was received or
synthesized, _handle_error sets that when it invents the 500, and the status
mapper declines to act on a code litellm made up, so those failures fall
through to the APIConnectionError the branch was always meant to produce.

Genuine upstream 5xx responses are untouched, which the second test pins.
The search transformation assertion BerriAI#38318 had loosened to InternalServerError
goes back to APIConnectionError for the same reason.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants