Litellm oss staging 02 26 2026

[ghost]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 26, 2026 1:10pm

[greptile-apps]

Greptile Summary

This staging PR bundles three independent fixes:

• Anthropic adapter cache token fix: The adapter now populates cache_read_input_tokens from prompt_tokens_details.cached_tokens instead of the private _cache_read_input_tokens attribute. This fixes cache read token reporting for OpenAI/Azure providers which only set prompt_tokens_details and not the Anthropic-specific hidden param. Both sync and streaming code paths are updated.
• Auth exception logging improvements: Expected auth failures (invalid key, budget exceeded, HTTP 4xx) are now logged at WARNING level instead of ERROR, reducing log noise. Unexpected errors (HTTP 5xx, bare exceptions) still log at ERROR with full tracebacks. Structured context (route, HTTP method, masked API key, error type/code, requester IP) is included in all auth error logs.
• Runtime MAX_STRING_LENGTH_PROMPT_IN_DB resolution: The spend tracking sanitizer now reads MAX_STRING_LENGTH_PROMPT_IN_DB from os.getenv() at call time rather than using the module-level constant, so values set later via proxy config environment_variables are properly honored.

All three fixes include corresponding unit tests with no real network calls.

Confidence Score: 4/5

• This PR is safe to merge — all three fixes are well-scoped, backwards-compatible, and covered by new unit tests.
• Score of 4 reflects clean, focused changes with good test coverage across all three fixes. The adapter cache token fix and spend tracking env-var fix are straightforward and well-tested. The auth logging refactor is slightly more involved but follows sound logic. One minor concern: the abbreviate_api_key call could fail on very short (non-empty) API keys, though this is an edge case unlikely in practice.
• Pay attention to litellm/proxy/auth/auth_exception_handler.py — the new log-level branching logic is the most complex change in this PR.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py	Fix: populate cache_read_input_tokens from prompt_tokens_details.cached_tokens instead of _cache_read_input_tokens, fixing OpenAI/Azure cache token mapping in both sync and streaming paths.
litellm/proxy/auth/auth_exception_handler.py	Improved auth error logging: expected auth failures (ProxyException, HTTPException < 500, BudgetExceededError) log at WARNING level; unexpected errors log at ERROR with traceback. Adds structured context (route, method, masked key, error type, code).
litellm/proxy/spend_tracking/spend_tracking_utils.py	Adds runtime resolution of MAX_STRING_LENGTH_PROMPT_IN_DB via os.getenv() to honor values set after module import by proxy config environment_variables.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py	Adds test for cache token mapping from prompt_tokens_details.cached_tokens to cache_read_input_tokens in Anthropic adapter output. Mock-only, no network calls.
tests/test_litellm/proxy/auth/test_auth_exception_handler.py	Adds 4 tests covering: expected errors at WARNING level, unexpected at ERROR, structured log context verification, and None/empty API key handling. All mock-based.
tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py	Adds test for runtime env override of MAX_STRING_LENGTH_PROMPT_IN_DB using monkeypatch. Removes trailing blank line.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Auth Error Raised] --> B{Exception Type?}
    B -->|ProxyException| C[Expected Auth Error]
    B -->|BudgetExceededError| C
    B -->|HTTPException < 500| C
    B -->|HTTPException >= 500| D[Unexpected Error]
    B -->|Other Exception| D
    C --> E["Log at WARNING level\n(no traceback)"]
    D --> F["Log at ERROR level\n(full traceback)"]
    E --> G[Build structured context:\nroute, method, masked key,\nerror_type, error_code, IP]
    F --> G
    G --> H[Post-call failure hook]
    H --> I[Raise ProxyException]

Loading

_{Last reviewed commit: 475bb94}

[greptile-apps]

_{6 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Misleading log field label

The format string route={} {} places http_method as the first {} and route as the second, producing output like route=GET /chat/completions. The route= label is misleading since its value starts with the HTTP method rather than the actual route path. The same issue exists in the exception log on line 130.

Consider giving each value its own label (e.g., route={}, method={}) for clearer log parsing.