fix(v1.87.0): port WARN/ERROR exception classification family (Wave 6a) - #56
Merged
Merged
Conversation
verbose_proxy_logger.exception() unconditionally logged every auth
failure at ERROR with a full traceback. Routine 401/403 outcomes (no
key, expired key, invalid key, role mismatch, budget exceeded) hit
this path constantly from probes / scanners / fat-fingered keys and
buried genuine errors under noise. The log line itself also lost
signal in two ways:
1. When ProxyException.message was empty, str(e) was empty, so the
formatted message collapsed to a bare exception type with no hint
why the request was rejected.
2. The requester IP was double-encoded (string concat into message +
extra dict), bloating structured-log size.
3. No correlation id — x-litellm-call-id existed but wasn't carried
into the log, making it hard to grep "the log entry for that one
401 the user reported".
Fix:
- Demote known auth-failure exception types (ProxyException,
HTTPException) to WARNING without exc_info.
- Keep ERROR + full traceback for truly unexpected exceptions so real
bugs are still visible.
- Fall back to type(e).__name__ when str(e) is empty so there's
always *something* to grep on.
- Drop the IP from the format string (kept in extra only).
- Add request_id, route, exception_type, http_status to extra for
structured-log aggregation.
Test plan:
- python3 -m pytest tests/test_litellm/proxy/auth/test_auth_exception_handler.py -v
→ 15 passed (3 new + 12 existing)
- New cases:
- test_known_auth_failure_logs_at_warning_without_traceback —
ProxyException → WARN, no traceback, extras populated
- test_unknown_exception_logs_at_error_with_traceback —
ValueError → ERROR + traceback
- test_empty_exception_message_falls_back_to_type_name —
ProxyException(message="") → log message contains "ProxyException"
…h_error (#29) * fix(proxy): classify auth failures into specific types instead of auth_error The wrapper in auth_exception_handler collapsed every wrapped HTTPException into the generic `auth_error` type, leaving UI clients no structured way to tell apart "your session is gone" from "you're logged in but not authorized for this endpoint" — both arrived as 401 with `type=auth_error`. The frontend had to fall back to regex- matching free-text messages, which broke on any backend wording drift. Add three specific ProxyErrorTypes and let the wrapper classify into them based on the HTTPException's status_code + detail content: auth_session_expired - token was once valid, no longer is (expired/revoked/deleted). UI redirect to login. auth_invalid_credentials - credential malformed / never existed / not in DB. UI redirect to login (same recovery: re-auth). auth_permission_denied - authenticated but lacks role/scope/ admin/master-key for this endpoint. UI must NOT redirect — toast only. Mapping rules (in priority order): - HTTP 403 -> auth_permission_denied (semantics of 403) - HTTP 401 + expired/revoked marker -> auth_session_expired - HTTP 401 + invalid/missing-credential marker -> auth_invalid_credentials - HTTP 401 + permission/role marker -> auth_permission_denied (LiteLLM uses 401 for role mismatch too) - otherwise -> auth_error (UI falls back to heuristic) The inner ProxyException pass-through is unchanged — places that already raise specific types (token_not_found_in_db, expired_key, *_access_denied, *_permission_error) keep flowing those types end- to-end. This change only fixes the HTTPException collapse case. Test plan: - python3 -m pytest tests/test_litellm/proxy/auth/test_auth_exception_handler.py -v -> 42 passed (10 new pure-function tests for _classify_auth_failure covering 403, expired markers, invalid markers, permission markers, unknown fallback, empty detail, priority ordering; 2 new integration tests asserting the wrapper's resulting ProxyException carries the classified type end-to-end) Companion PR: fix/ui-401-error-type-routing — frontend reads `error.type` and dispatches per a static map, with the current cookie + marker heuristic preserved as fallback when `type` is missing or is the generic auth_error (graceful degradation if backend is on older build). * fix(proxy): also classify bare-Exception path, not just HTTPException Live e2e revealed that the vast majority of auth failures in LiteLLM flow through the bare-Exception catch-all in the wrapper, NOT HTTPException — e.g.: Exception("No api key passed in.") Exception("LiteLLM Virtual Key expected. Received=..., expected to start with 'sk-'.") Exception("Malformed API Key passed in. Ensure Key has `Bearer` prefix.") The first iteration of _classify_auth_failure only inspected HTTPException.detail, so the catch-all kept emitting plain `auth_error` for these very-common cases, leaving the UI with no useful type to dispatch on. Generalize the classifier to fall back to str(e) when there's no detail attribute, and call it from the catch-all raise. Also expand the invalid-credentials markers list to cover the actual phrasings the auth pipeline emits. Existing tests (42) still pass — the HTTPException path tests continue to work because detail still takes precedence. * fix(proxy): add permission markers found via live e2e (proxy admin / your role) Live e2e against /key/generate with a non-admin key surfaced this real permission-denied message: "Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/key/generate. Your role=unknown. Your user_id=unknown" The original marker list missed "proxy admin" and "your role", so the classifier dropped this into auth_error. Add both. Tests stay green.
#39) * fix(proxy): classify management-endpoint exceptions into WARN vs ERROR Centralizes exception-log routing for proxy management endpoints in a new `log_proxy_exception` helper. Known business errors (HTTPException, ProxyException, BudgetExceededError, RateLimitError, AuthenticationError, NotFoundError, BadRequestError, UnsupportedParamsError, etc.) and any exception carrying a 4xx or upstream-5xx (502/503/504) status_code are logged at WARN as a single structured line with no traceback. Truly unexpected exceptions (RuntimeError, KeyError, real 500s) remain ERROR with a full traceback. Replaces every catch-all `verbose_proxy_logger.exception(...)` across the management endpoints (user, key, team, organization, customer, model, mcp, scim, ui_sso, tag, project, tool, cache_settings, team_callback, config_override, access_group, model_access_group, common_daily_activity). This is the same policy the auth pipeline adopted in commit 168feaf — extending it to the rest of the proxy control plane so client-induced 4xx errors (409 "already exists", 401 "token_not_found_in_db", etc.) stop polluting the ERROR stream with multi-line tracebacks. Tests pin the WARN vs ERROR classification for every business exception type, the 4xx/5xx status-code fallback, and the emission shape (level, exc_info presence, structured extra fields). * fix(proxy): drop traceback from handle_exception_on_proxy and bare-Exception auth path E2E verification of the management-endpoint WARN/ERROR routing surfaced two paths that were still emitting ERROR + traceback for routine 4xx outcomes: 1. ``handle_exception_on_proxy`` (litellm/proxy/utils.py) called ``verbose_proxy_logger.exception("Exception: %s")`` unconditionally. Every endpoint that uses this helper as its raise wrapper therefore re-logged 401/404/409 with a full traceback right after the WARN line the catch-all just produced. Demoted to DEBUG — the caller has already classified and logged. 2. ``_handle_authentication_error`` only treated ``ProxyException`` and ``HTTPException`` as known. The auth pipeline also raises bare ``Exception(...)`` for missing/malformed credentials (``Exception("No api key passed in.")``, ``Exception("Malformed API Key passed in. ...")``, ``Exception("LiteLLM Virtual Key expected. ...")``). These flooded the ERROR stream with tracebacks on every unauthenticated probe. Extended the known-check to consult ``_classify_auth_failure`` — if the classifier confidently routes to a specific ``auth_*`` type (auth_session_expired, auth_invalid_credentials, auth_permission_denied), we treat the exception as known and emit WARN. Only the catch-all ``auth_error`` outcome (genuinely ambiguous) keeps ERROR + traceback. Verified end-to-end against the e2e proxy: - 50 successive 401s with bogus bearer tokens → 50 WARN, 0 ERROR, 0 traceback - /user/new duplicate → single WARN, no traceback (was 2 lines incl. ERROR+stack) - nonexistent user 404 → single WARN - ValueError("totally unexpected") still ERROR + traceback (unit test pins this) Regression tests parametrize the three known bare-Exception messages that the auth pipeline actually raises so a future refactor of ``_KNOWN_AUTH_ERROR_TYPES`` can't silently re-introduce the flood.
PR #39 covered management endpoints; this finishes the job for the LLM request path and shared utilities that wrap it. Same policy: known business exceptions (4xx with stable type, upstream 502/503/504) log at WARN with a single structured line; truly unexpected exceptions (bare RuntimeError, KeyError, real 500) keep ERROR + traceback. Scope: - proxy_server.py: 20 route handlers — completion, moderations, audio_transcriptions, realtime websocket (pre-call / upstream-status / internal-error), token_counter, login_v2/v3/exchange, async_data_generator + async_assistants_data_generator streaming wrappers, plus 9 reload/schedule/cancel/status admin endpoints for model_cost_map and anthropic_beta_headers. Deliberately NOT migrated: 25 startup / background-health-check / config-reload / model-list-filter helpers. Those run outside the request path, and failures there ARE the bug — they should keep ERROR + traceback. - common_request_processing.py: stream first-chunk consumer, deferred-logging hooks, post-call streaming guardrail, orphaned streaming logging, async_data_generator wrapper, and the central _handle_llm_api_exception classifier. - route_llm_request.py: shared aiohttp session recreation fallback — failure is non-fatal (we fall back to None and continue), shouldn't flood ERROR. - litellm_pre_call_utils.py: api-version query-param parse. - utils.py: post_call_failure_hook callback failures (both inner and setup), post_call_response_headers_hook, send_email. - auth/user_api_key_auth.py: websocket auth wrapper (was logging via the noisy code path). Also fixes a misuse of verbose_proxy_logger .exception() outside an except block (custom_litellm_key_header branch) — there's no current exception, so demoted to a single WARNING line that no longer prints "(NoneType: None)". - auth/auth_checks.py: get_team_membership, access_group lookup, team alias lookup, organization alias lookup. All four were ERROR + traceback for routine 404 outcomes. - common_utils/http_parsing_utils.py: invalid-JSON 400 was logged at ERROR (no traceback, but still polluted the ERROR stream on every malformed client payload); demoted to WARN. The unexpected-exception catch-all stays ERROR + traceback — that's the path that catches real server faults. E2E verification on this branch: - 7 distinct 4xx scenarios (unknown model / bad provider key / unauthenticated /schedule/* / bogus bearer / malformed JSON body / bogus login) → all WARN single line, 0 traceback. - 50 successive 401s from rotated bogus bearer tokens → 50 WARN, 0 ERROR, 0 traceback. - Pre-existing 79 unit tests in test_auth_exception_handler + test_exception_logging still pass (RuntimeError/ValueError still route to ERROR + traceback).
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier classification
litellm_extras/only)litellm/corelitellm/coreIf Tier C or D, did you try upstream first?
for auth/management/LLM failures) is opinionated — upstream may want
a configurable hook rather than the hard-coded WARN policy we ship.
Summary
First and highest-risk sub-wave of Wave 6. Re-applies the four-commit
WARN/ERROR exception classification family from
ship/v1.83.10ontov1.87.0. Per the B-report watch list, this is the single most expensive
port to delay further — every other Wave 6 sub-wave touches files in
the same hot region (
proxy_server.py,user_api_key_auth.py,auth_checks.py,common_request_processing.py), so landing thisfirst means subsequent sub-waves patch on top instead of fighting on
the same lines.
Cherry-picks (4 commits, chronological)
16e054067cf3bf9e1a868f5cd566c21eec335b93What changes
litellm/proxy/common_utils/exception_logging.py—log_proxy_exception()decides WARN vs ERROR based on whether theexception is one of
_KNOWN_AUTH_ERROR_TYPES(ProxyException,HTTPException) or a true unexpected failure.
user_api_key_auth.py,auth_checks.py,auth_exception_handler.py): 401/403 outcomes now log at WARN (notraceback). Adds
_classify_auth_failure()so wrapped failurescarry a structured
ProxyErrorTypes(auth_session_expired/auth_invalid_credentials/auth_permission_denied) instead ofthe generic
auth_error— the UI uses this to route 401 →/loginvs 401 → "session expired".verbose_proxy_logger.exceptionforlog_proxy_exception. ProxyException/ HTTPException stay WARN; unexpected failures still log ERROR with
traceback.
common_request_processing.py,proxy_server.py,route_llm_request.py,litellm_pre_call_utils.py,http_parsing_utils.py,utils.py): same policy extended throughthe LLM execution path so chat-completion failures get the same
classification.
Conflict resolutions
Six manual resolutions, all in module-level imports — upstream and our
patch each added new imports to the same line range. Merged both sides
verbatim:
auth_exception_handler.py— bothDB_UNAVAILABLE_FALLBACK_USER_IDand
_KNOWN_AUTH_ERROR_TYPESconstants kept.team_callback_endpoints.py,mcp_management_endpoints.py,team_endpoints.py,key_management_endpoints.py,litellm_pre_call_utils.py,auth_checks.py,user_api_key_auth.py,utils.py— added the newlog_proxy_exceptionimport alongsideupstream's other added imports.
One refactoring conflict in
common_request_processing.py:ProxyBaseLLMRequestProcessing._flush_deferred_async_logging. Ourpatch had touched the OLD inline form to add error logging. Took
upstream's refactored form — the helper handles failures internally
and is the right shape going forward.
Verification
```bash
python3 -m pytest
tests/test_litellm/proxy/auth/test_auth_exception_handler.py
tests/test_litellm/proxy/common_utils/test_exception_logging.py -q
→ 79 passed in 29.57s
```
Smoke-tested all affected module imports — no circular import / missing
symbol regressions.
Phase 2 progress after this PR
Type
🐛 Bug Fix