Litellm oss staging 04 02 2026 p1

[krrish-berri-2]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 8, 2026 2:07pm

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_oss_staging_04_02_2026_p1 (a076dab) with main (62757ff)}

[greptile-apps]

Greptile Summary

This PR is a staging batch that bundles several independent improvements: SAP GenAI Hub orchestration refactoring (modules, grounding/masking/filtering, credential resolution), Gemini service_tier request/response support, Vertex AI pluggable credential type, Anthropic web-search cost tracking, Prometheus org-label enrichment, and various logging/proxy fixes.

Key changes:

• SAP modules fix: _build_prompt_module now correctly places grounding, masking, filtering, and translation at the top level of the modules dict, resolving the previous finding where they were silently discarded.
• Gemini service_tier: Maps OpenAI service_tier ("auto" → "priority", "default" → "standard") in both request transformation and response header extraction for streaming and non-streaming paths.
• Prometheus org labels: org_id and org_alias are conditionally emitted for a curated set of per-request metrics; org_alias is now properly guarded with is not None, addressing the asymmetry flagged in the previous review round.
• handle_exception_on_proxy: Now preserves the original status_code from the exception instead of hardcoding 500.
• Anthropic handler: logging_obj is now forwarded to client.post() so track_llm_api_timing can record llm_api_duration_ms correctly.
• Vertex AI pluggable credentials: Adds support for the executable-based external credential source (google.auth.pluggable).

Confidence Score: 4/5

Safe to merge for new deployments; existing Prometheus scrapers will see a label cardinality break on the 11 metrics in _org_label_metrics (raised in prior review, unresolved).

All previously flagged issues are either fixed (org_alias guard, SAP modules discard) or confirmed intentional (SAP first-source-wins credentials). No new P0/P1 issues found. Score is 4 rather than 5 solely because the backwards-incompatible Prometheus label cardinality change from the prior review round remains unaddressed.

litellm/types/integrations/prometheus.py — backwards-incompatible label expansion for 11 existing metrics.

Vulnerabilities

No security concerns identified. The credential resolution changes for SAP are intentional (first-source-wins by design per SAP). The organization_alias field is fetched from the database and only stored in request metadata — no exposure to callers. The handle_exception_on_proxy change does not expose internal stack traces.

Important Files Changed

Filename	Overview
litellm/llms/sap/chat/transformation.py	Major refactor of SAP request transformation: _build_prompt_module now correctly spreads grounding/masking/filtering/translation into the top-level modules dict, fixing the silent-discard bug flagged in prior rounds.
litellm/llms/sap/credentials.py	New first-source-wins credential resolution (confirmed intentional per SAP design). Breaking change for users mixing credential sources, but developer has accepted this trade-off.
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Adds service_tier request/response support and refactors chunk_parser into helper methods; response_headers correctly threaded through streaming and non-streaming paths.
litellm/llms/vertex_ai/vertex_llm_base.py	Adds support for pluggable (executable-based) external credential source in Vertex AI authentication.
litellm/llms/anthropic/chat/handler.py	Forwards logging_obj to client.post() across all call paths; consistent with the pattern used throughout the codebase.
litellm/proxy/proxy_server.py	Adds user_api_key_org_alias to request metadata; now consistently guarded with is not None, mirroring the existing org_id guard.
litellm/proxy/utils.py	Fixes handle_exception_on_proxy to preserve the exception's own status_code; joins organization_alias column from SQL query.
litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py	Adds Anthropic server_tool_use.web_search_requests check; is not None guard passes when value is 0 (pre-existing pattern).
litellm/types/integrations/prometheus.py	Introduces _org_label_metrics frozenset and injects org labels; backwards-incompatible label cardinality change for existing deployments.
tests/test_litellm/llms/sap/chat/test_sap_transformation.py	Comprehensive new test file covering SAP transform_request, grounding, masking, filtering, and streaming; mock-only with no live network calls.

_{Reviews (12): Last reviewed commit: "Fix greptile review" | Re-trigger Greptile}

[CLAassistant]

All committers have signed the CLA.