fix(exceptions): map upstream status codes for providers with no exception_type branch - #38318
Conversation
…ption_type branch
Greptile SummaryThe PR preserves upstream HTTP status semantics for providers without dedicated exception branches and corrects OpenAI-like 403 classification.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/litellm_core_utils/exception_mapping_utils.py | Adds status-based fallback exception mapping and returns typed permission errors for upstream 403 responses. |
| litellm/router.py | Extends the existing authentication fail-fast behavior to permission-denied errors, resolving the previously reported single-deployment retry path. |
| tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py | Expands status-mapping coverage across providers and verifies MiniMax authentication failures retain their upstream 401 semantics. |
| tests/test_litellm/test_router.py | Verifies permission-denied errors fail immediately for one deployment while remaining eligible for advancement when alternatives exist. |
Reviews (2): Last reviewed commit: "fix(router): fail fast on PermissionDeni..." | Re-trigger Greptile
| elif original_exception.status_code == 403: | ||
| raise PermissionDeniedError( | ||
| message=f"{custom_llm_provider.capitalize()}Exception - {original_exception.message}", | ||
| llm_provider=custom_llm_provider, | ||
| model=model, | ||
| response=_response_or_stub(original_exception, status_code=403), | ||
| ) |
There was a problem hiding this comment.
If an OpenAI-like provider is configured as a single healthy deployment, mapping its 403 to PermissionDeniedError bypasses the router's AuthenticationError-specific single-deployment guard and retries the permanent authorization failure against the same deployment, adding unnecessary upstream requests and delaying the client response.
Knowledge Base Used: Model request execution
…itellm_fix_branchless_provider_status_mapping # Conflicts: # tests/test_litellm/litellm_core_utils/test_exception_mapping_utils.py
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 6386a68. Configure here.
BerriAI#38318 taught exception_type to map upstream status codes for providers with no branch of their own. It reads the status code off the exception, but _handle_error stamps 500 onto every failure that never carried one, so a refused connection reached the mapper wearing a status code nothing upstream had sent, and came back as InternalServerError instead of APIConnectionError. The two are not interchangeable to a caller: a 5xx says the provider answered and failed, which the router treats as a reason to cool the deployment down, while a connection error says the request never landed. BaseLLMException now records whether its status code was received or synthesized, _handle_error sets that when it invents the 500, and the status mapper declines to act on a code litellm made up, so those failures fall through to the APIConnectionError the branch was always meant to produce. Genuine upstream 5xx responses are untouched, which the second test pins. The search transformation assertion BerriAI#38318 had loosened to InternalServerError goes back to APIConnectionError for the same reason.
TLDR
Problem this solves:
exception_typebranch answered every upstream error as a 500APIConnectionError, code 500, so clients kept retryingAuthenticationErrorfor an upstream 403How it solves it:
BadRequestErrorAPIConnectionErroris only raised when the exception carries no status at allPermissionDeniedErrorPermissionDeniedErrorlike it doesAuthenticationErrorUser Flow
Before: a developer with an invalid MiniMax key gets a 500 from the gateway, so their client keeps retrying a request that can never succeed
"model": "minimax-m2.5"litellm.APIConnectionError: MinimaxException - {... "http_code":"401" ...}and"code":"500"After: the same requests come back as 401 authentication errors, so the client stops retrying and the developer fixes the key
"model": "minimax-m2.5"litellm.AuthenticationError: MinimaxException - {... "http_code":"401" ...}and"code":"401"Relevant issues
Linear ticket
Resolves LIT-6166
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup for both legs: a proxy booted with
python litellm/proxy/proxy_cli.py --config proxy_config.yaml --port <port> --num_workers 2, no database, against the real MiniMax and Anthropic APIs.minimax-m2.5carries an invalid key on purpose (MiniMax answers 401 to any invalid key, no account needed).claude-haiku-4-5is the paid control,claude-haiku-4-5-badkeyis the typed-provider control for the same 401Before (e52f055)
/v1/chat/completions with an invalid MiniMax key
/v1/responses with an invalid MiniMax key
/v1/messages with an invalid MiniMax key
Control: /v1/chat/completions with an invalid Anthropic key (typed provider)
Control: valid Anthropic key on all three endpoints
After (6386a68)
/v1/chat/completions with an invalid MiniMax key
AuthenticationError/v1/responses with an invalid MiniMax key
AuthenticationError/v1/messages with an invalid MiniMax key
litellm.AuthenticationErrorprefix here comes from fix(otel): map /v1/messages provider errors before failure logging #38310, which landed on the base and is part of this tipControl: /v1/chat/completions with an invalid Anthropic key (typed provider)
Control: valid Anthropic key on all three endpoints
Observations from the run:
APIConnectionErrorlitellm.AuthenticationErrorprefix on the after leg comes from fix(otel): map /v1/messages provider errors before failure logging #38310 on the base, not this PRType
🐛 Bug Fix
Caveats (if any)
Low
APIErrorwith the upstream status, matching the OpenAI branchBadRequestErrorcarrying status 422, matching the OpenAI branchAPIError; left alone, out of scopeFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
6386a68 passes /live-pr-risk
Note
Medium Risk
Changes global error mapping and HTTP codes returned to clients for many providers; behavior shifts from generic 500s to typed 4xx/5xx, which affects retries, cooldowns, and SDK handling but is intentional bug-fix scope.
Overview
Providers without a dedicated
exception_typebranch used to surface most HTTP errors asAPIConnectionErrorwith code 500, so invalid keys (e.g. MiniMax 401) looked transient and clients kept retrying.This PR adds
_map_exception_by_statuson the generic fallback path so upstream 401, 403, 404, 408, 429, and 5xx map to the same typed LiteLLM exceptions as handled providers;APIConnectionErroris only used when the exception has no status. OpenAI-like providers now raisePermissionDeniedErrorfor 403 instead ofAuthenticationError.The router
should_retry_this_errorlogic treatsPermissionDeniedErrorlikeAuthenticationError(fail fast when only one deployment exists). Tests are updated across exception mapping, router retry, and several provider tests that now expect the correct typed errors.Reviewed by Cursor Bugbot for commit 6386a68. Bugbot is set up for automated code reviews on this repo. Configure here.