fix: 429s from batch creation being converted to 500

[J-Byron]

Relevant issues

Fixes #23901

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix

Changes

handle_exception_on_proxy in litellm/proxy/utils.py was hardcoding HTTP_500 as the fallback status code for any exception that wasn't an HTTPException or ProxyException. LiteLLM's own exceptions (like RateLimitError) have a status_code attribute — we just weren't using it. One line fix: getattr(e, "status_code", 500) instead of the hardcoded 500.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 27, 2026 10:39pm

[J-Byron]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing J-Byron:fix/batch-creation-429-status-code (0a104af) with main (d949085)}

[greptile-apps]

Greptile Summary

This PR fixes a bug where LiteLLM exceptions carrying non-500 status codes (e.g., RateLimitError with 429, AuthenticationError with 401) were being mapped to HTTP 500 responses by handle_exception_on_proxy in litellm/proxy/utils.py. The fix is a single-line change: getattr(e, \"status_code\", status.HTTP_500_INTERNAL_SERVER_ERROR) replaces the hardcoded status.HTTP_500_INTERNAL_SERVER_ERROR. The misleading \"Internal Server Error, \" message prefix is also removed since it's no longer accurate for non-5xx paths.

Key changes:

• litellm/proxy/utils.py: Reads status_code from the exception via getattr with a safe 500 fallback, correctly forwarding upstream rate-limit (429) and auth (401) codes to the client.
• tests/proxy_unit_tests/test_proxy_utils.py: Three new mock-only unit tests cover the 429 (RateLimitError), 500 (generic Exception), and 401 (AuthenticationError) paths — no real network calls are made.

Minor observation: The type field in the returned ProxyException remains hardcoded to ProxyErrorTypes.internal_server_error regardless of the actual error kind, so the JSON body for a 429 will include \"type\": \"internal_server_error\". The HTTP status code is now correct, but clients inspecting the body's type field may still see a misleading value. A more complete fix would map exception types to appropriate ProxyErrorTypes values, but that is a larger refactor outside this PR's scope."

Confidence Score: 5/5

Safe to merge — minimal, targeted fix with no regressions and solid test coverage.

The only remaining finding is P2: the type field in the error response body stays as internal_server_error even for 429/401 errors. This is a cosmetic/semantic inconsistency in the JSON body; the HTTP status code — the primary signal clients rely on — is now correct. No logic bugs, no security issues, no network calls in tests.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	One-line fix: reads status_code attribute from the exception via getattr instead of hardcoding 500; also drops the misleading "Internal Server Error, " prefix from the message.
tests/proxy_unit_tests/test_proxy_utils.py	Three new mock-only unit tests added covering 429 (RateLimitError), 500 (generic Exception), and 401 (AuthenticationError) cases — no real network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Exception raised in proxy endpoint] --> B{isinstance HTTPException?}
    B -- Yes --> C[ProxyException: code from e.status_code]
    B -- No --> D{isinstance ProxyException?}
    D -- Yes --> E[Return e as-is]
    D -- No --> F[_status_code = getattr e status_code 500 - FIXED: was hardcoded 500]
    F --> G[ProxyException: code from _status_code, type still hardcoded to internal_server_error]
    C --> H[openai_exception_handler uses int of exc.code as HTTP status]
    G --> H
    E --> H
    H --> I[JSONResponse returned to client]

Loading

_{Reviews (1): Last reviewed commit: "fix: 429s from batch creation being conv..." | Re-trigger Greptile}

[greptile-apps]

type field remains internal_server_error for non-5xx errors

The code (HTTP status) is now correctly forwarded (e.g., 429 for rate limits, 401 for auth errors), but type is still hardcoded to ProxyErrorTypes.internal_server_error. This means the JSON response body for a rate-limit error will contain {"error": {"type": "internal_server_error", "code": "429"}}, which is semantically inconsistent.

The OpenAI API returns "type": "requests" for 429s. Clients that inspect the type field to determine error category will get incorrect information.

A follow-up improvement could derive a more appropriate ProxyErrorTypes from the exception class (e.g. a rate_limit_error enum value), though that's a larger refactor than this PR's scope.

[J-Byron]

@shivamrawat1

test_handle_exception_on_proxy_preserves_status_code creates a RateLimitError with status 429 and verifies it passes through handle_exception_on_proxy as 429 instead of 500.

Note: ideally this would be tested against a real OpenAI batch rate limit response, but those are hard to trigger on demand (2000 batches/hour limit). The unit test directly exercises the translation layer where the bug was — which is what matters here.