fix(router): break retry loop on non-retryable errors

[AtharvaJaiswal005]

Fixes #21343

Summary

The retry loop in async_function_with_retries swallows non-retryable errors (400 ContextWindowExceeded, 404 NotFound, etc.) and raises the original retryable error instead. This causes misleading error messages and unnecessary retry attempts.

Root Cause

Inside the for current_attempt in range(num_retries) retry loop in router.py, the except Exception block has two problems:

1. original_exception never updated - stays stuck on the first error, so when retries exhaust, the stale initial error is raised instead of the latest one
2. No retryability check - the pre-loop code calls should_retry_this_error() to break on non-retryable errors, but the retry loop itself never does this check, so it sleeps and retries even on 400/404 errors

Example from the issue: First call hits 429 RateLimitError (retryable), retry hits 400 ContextWindowExceeded (non-retryable). Expected: ContextWindowExceededError raised immediately. Actual: loop continues retrying and eventually raises the stale RateLimitError.

Fix

litellm/router.py - two changes inside the retry loop's except block:

• Added original_exception = e to always track the latest error
• Added should_retry_this_error() check (same function already used in the pre-loop code at line 5106) to break out immediately on non-retryable errors. Guarded by if not _retry_policy_applies to respect retry policy precedence

tests/test_litellm/test_router_retry_non_retryable_errors.py - 5 new tests:

Test	Scenario	Verifies
test_non_retryable_error_in_retry_loop_raises_immediately	429 then 400 ContextWindowExceeded	Raises ContextWindowExceededError, not RateLimitError
test_bad_request_error_in_retry_loop_raises_immediately	429 then 400 BadRequest	Raises BadRequestError immediately
test_original_exception_updated_to_latest_error	All retries hit 429	Last error message is raised, not first
test_retryable_errors_still_retry_normally	All retries hit 429	Still retries full configured count (1 + 3 = 4 calls)
test_not_found_error_in_retry_loop_raises_immediately	429 then 404 NotFound	Raises NotFoundError, only 2 calls made

Type

• Bug fix (non-breaking change which fixes an issue)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 17, 2026 7:02am

[greptile-apps]

Greptile Summary

Fixes the Router retry loop in async_function_with_retries to correctly handle non-retryable errors (400, 404, etc.) that occur during retries. Previously, non-retryable errors inside the retry loop were swallowed, and the stale first exception was always raised. Now: (1) original_exception is updated on each attempt so the most recent error is raised, and (2) should_retry_this_error() is called inside the retry loop (gated by _retry_policy_applies) to break immediately on non-retryable errors.

• Core fix in litellm/router.py: Two additions to the retry loop's except block — tracking latest exception and checking retryability via should_retry_this_error(), consistent with the pre-loop check already at line 5106
• Tests in tests/test_litellm/test_router_retry_non_retryable_errors.py: 5 mock-only tests covering ContextWindowExceeded, BadRequest, NotFound, exception tracking, and normal retry behavior
• Fixes #21343 with verified test coverage

Confidence Score: 4/5

• This PR is safe to merge — it fixes a clear bug with a minimal, well-scoped change that follows existing patterns in the codebase.
• The fix is small and consistent with the pre-loop retry logic already in the codebase. The two changes (tracking latest exception and adding retryability check) are straightforward and well-tested. Minor concern: the broad except Exception catch in the new code could theoretically mask bugs in should_retry_this_error, but this is an acceptable trade-off given the code structure. Score is 4 instead of 5 because the retry logic is complex and any change here warrants careful integration testing.
• litellm/router.py — the retry loop exception handling is critical path code that merits careful review of edge cases.

Important Files Changed

Filename	Overview
litellm/router.py	Adds original_exception = e to track latest error and a should_retry_this_error() check inside the retry loop to break on non-retryable errors. Logic is correct and consistent with pre-loop behavior.
tests/test_litellm/test_router_retry_non_retryable_errors.py	New test file with 5 well-structured mock-only tests covering non-retryable error handling, exception tracking, and normal retry behavior. No real network calls.

Flowchart

flowchart TD
    A[Initial Call] -->|Exception| B[Set original_exception = e]
    B --> C{retry_policy applies?}
    C -->|Yes| D[Get num_retries from policy]
    C -->|No| E[should_retry_this_error]
    E -->|Raises: non-retryable| F[Raise error immediately]
    E -->|Returns True: retryable| G[Enter Retry Loop]
    D --> G
    G --> H[make_call attempt]
    H -->|Success| I[Return response]
    H -->|Exception e| J["original_exception = e (NEW)"]
    J --> K[Log retry & get healthy deployments]
    K --> L{retry_policy applies?}
    L -->|Yes| N[Sleep & continue loop]
    L -->|No| M["should_retry_this_error (NEW)"]
    M -->|Raises: non-retryable| O[Raise e immediately]
    M -->|Returns True: retryable| N
    N --> P{More retries?}
    P -->|Yes| H
    P -->|No| Q[Raise original_exception]

Loading

_{Last reviewed commit: 5992cac}

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Broad except may mask bugs in should_retry_this_error

The except Exception: here catches any exception from should_retry_this_error, including unexpected ones (e.g., AttributeError, TypeError from a bug in the method itself). If should_retry_this_error ever has a bug, it would be silently swallowed and e raised instead, making debugging harder.

Consider catching only the specific exception types that should_retry_this_error raises (which are litellm exception types):

Suggested change

	except Exception:
	raise e
	except litellm.LITELLM_EXCEPTION_TYPES:
	raise e

That said, this is a minor style concern — the current approach works correctly for all expected scenarios.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}