Litellm oss staging 04 01 2026

[krrish-berri-2]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 16, 2026 1:11pm

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
6 out of 8 committers have signed the CLA.

✅ michelligabriele
✅ mats852
✅ Vedanshu7
✅ yuneng-berri
✅ milan-berri
✅ Sameerlite
❌ krrish-berri-2
❌ voidborne-d
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing litellm_oss_staging_04_01_2026 (012f470) with main (d1df4e8)}

In general, to fix this type of issue, any place where two adjacent string literals are meant to be concatenated should use an explicit + operator (or other explicit joining, like ''.join([...])). This preserves the runtime string value but makes the intention clear and avoids confusion with missing commas in lists or tuples.

For this specific case in litellm/litellm_core_utils/secret_redaction.py, we want to keep the regex pattern as a single element in the patterns list, but make its multi-line nature explicit. On lines 20–21, we should add a trailing + at the end of the first line so it reads like:

r"(?:aws_secret_access_key|aws_session_token|aws_access_key_id)" +
r"\s*[:=]\s*[A-Za-z0-9/+=]{20,}",

This leaves the compiled regex unchanged, avoids any missing-comma confusion, and satisfies the CodeQL recommendation. No additional imports, methods, or definitions are required, and no other parts of the file need modification.

Use explicit concatenation (+) for intentionally split string literals inside the patterns list.

Best fix here: in litellm/litellm_core_utils/secret_redaction.py, update the pattern at lines 40–41 so the two raw string literals are joined with +. This preserves exact functionality while making intent explicit and removing the CodeQL finding. No imports, methods, or new definitions are needed.

In general, to fix “implicit string concatenation in a list” when the concatenation is intentional, explicitly join the string literals with + so it’s clear that they form one element, rather than being separate elements accidentally merged by omission of a comma. If the concatenation is not intended, instead insert the missing comma to create distinct list elements.

For this specific case in litellm/litellm_core_utils/secret_redaction.py, the long regex for key-name-based redaction (lines 50–56) is meant to be a single pattern. To retain behavior while satisfying the recommendation and avoiding ambiguity, we should convert the multiple adjacent raw string literals into a single concatenated expression using +, optionally keeping line breaks for readability. We must preserve the exact pattern text, including the closing parenthesis and subsequent character-class portion starting at line 56. The change is confined to the patterns list in _build_secret_patterns; no new imports, methods, or external dependencies are needed.

Concretely, replace the current block:

• from line 50: r"(?:master_key|database_url|db_url|connection_string|"
• through line 56: r"""['\"]?\s*[:=]\s*['\"]?[^\s,'\"})\]{}>]+""",

with a single expression where each part is joined by +, keeping the exact same regex content, e.g.:

r"(?:master_key|database_url|db_url|connection_string|"
+ r"private_key|signing_key|encryption_key|"
+ r"auth_token|access_token|refresh_token|"
+ r"slack_webhook_url|webhook_url|"
+ r"database_connection_string|"
+ r"huggingface_token|jwt_secret)"
+ r"""['\"]?\s*[:=]\s*['\"]?[^\s,'\"})\]{}>]+""",

No other lines in the file need adjustment.

[greptile-apps]

Greptile Summary

This PR bundles several fixes and a new Qostodian Nexus guardrail integration: Gemini Files API 403 fix, Azure AI Foundry embedding api_version forwarding, service_tier cost propagation, team-member rate limit data in the key-lookup SQL join, and a new standalone secret-redaction module that scrubs API keys from exception messages.

• P0: litellm/types/guardrails.py line 35 — missing ) in the qohash import block causes SyntaxError: '(' was never closed, preventing the module from loading.
• P0: litellm/_logging.py line 27 — import re was removed but _build_secret_patterns() still references re.Pattern in its annotation and calls re.compile(), causing a NameError that crashes LiteLLM at startup.

Confidence Score: 4/5

Not safe to merge — two P0 bugs prevent LiteLLM from starting up.

Two P0 findings: a SyntaxError in guardrails.py (unclosed parenthesis in import) and a NameError in _logging.py (import re removed but still used at definition time). Both are confirmed to crash module loading. The fixes are one-liners each, but must be applied before this can ship.

litellm/types/guardrails.py (line 35, SyntaxError) and litellm/_logging.py (line 27, NameError on missing import re)

Important Files Changed

Filename	Overview
litellm/types/guardrails.py	P0 SyntaxError: missing ) in qohash import on line 35-37 prevents the module from loading entirely.
litellm/_logging.py	P0: import re removed but _build_secret_patterns() still references re.Pattern (return annotation) and re.compile() (body), causing NameError at module load; redact_string imported from secret_redaction but unused.
litellm/litellm_core_utils/secret_redaction.py	New standalone redaction utility module; adds URL query-param key pattern to fix Gemini API key exposure in httpx error messages.
litellm/proxy/guardrails/guardrail_hooks/qohash/qohash.py	New Qostodian Nexus guardrail wrapping GenericGuardrailAPI; defaults to HTTP for the API base; pass-through apply_guardrail override needed for proxy/utils.py detection is well-commented.
litellm/llms/vertex_ai/gemini/transformation.py	Adds early-exit branch for Gemini Files API URIs to avoid 403 HTTP fetch; correctly passes file_uri with optional mime_type through to the Gemini API.
litellm/proxy/utils.py	Extends existing raw SQL to LEFT JOIN LiteLLM_BudgetTable for team-member TPM/RPM limits; JOIN condition matches schema (budget_id exists on LiteLLM_TeamMembership).
litellm/llms/azure/cost_calculation.py	Adds service_tier parameter to enable priority/flex pricing tiers; threaded through from cost_calculator.py correctly.
litellm/llms/azure/azure.py	Fixes missing api_version forwarding to aembedding() call for Azure AI Foundry v1 endpoints.
litellm/litellm_core_utils/exception_mapping_utils.py	Applies redact_string() to exception messages when secret redaction is enabled, fixing the Gemini API key leak via raised exceptions.

Class Diagram

%%{init: {'theme': 'neutral'}}%%
classDiagram
    class GenericGuardrailAPI {
        +api_base: str
        +extra_headers: list
        +apply_guardrail(inputs, request_data, input_type, logging_obj)
        +get_config_model() Type
    }

    class QostodianNexus {
        +GUARDRAIL_NAME: str
        +__init__(api_base, **kwargs)
        +apply_guardrail(inputs, request_data, input_type, logging_obj)
        +get_config_model() QostodianNexusConfigModel
    }

    class QostodianNexusConfigModel {
        +api_base: Optional[str]
        +ui_friendly_name() str
    }

    class GuardrailConfigModel {
        <<base>>
    }

    class SupportedGuardrailIntegrations {
        <<enum>>
        QOSTODIAN_NEXUS
    }

    GenericGuardrailAPI <|-- QostodianNexus
    GuardrailConfigModel <|-- QostodianNexusConfigModel
    QostodianNexus ..> QostodianNexusConfigModel : get_config_model()
    QostodianNexus ..> SupportedGuardrailIntegrations : registered as

Loading

Comments Outside Diff (1)

1. litellm/_logging.py, line 27 (link)

   NameError at module load — import re removed but still used

   This PR removes import re from _logging.py, but _build_secret_patterns() still uses re.Pattern in the return-type annotation (line 27) and re.compile / re.IGNORECASE in the body (line 82). In Python 3.11, function annotations are evaluated eagerly at definition time, so the module will crash with NameError: name 're' is not defined the moment Python processes line 27 — before any LiteLLM code can run.

   Verified with exec:

   def foo() -> re.Pattern:
       pass
   # → NameError: name 're' is not defined

   The fix is simply to restore the removed import, or quote the annotation:

   Or add import re back to the imports section. The redact_string import from secret_redaction (added on line 11) is currently unused in this module — if the intent was to delegate to that function, the existing _redact_string / _SECRET_RE must also be removed or kept alongside the restored import re.

_{Reviews (3): Last reviewed commit: "Merge branch 'main' into litellm_oss_sta..." | Re-trigger Greptile}

[greptile-apps]

Plaintext HTTP as default API base

The default URL http://nexus:8800 uses unencrypted HTTP. Even for an internal service, plaintext HTTP exposes the full request body (which may contain user prompts and LLM responses) to network observers on the path between LiteLLM and the Nexus container.

Consider defaulting to https:// and documenting that HTTP may be used for local/container deployments where TLS is terminated at a higher layer.

Suggested change

	api_base = api_base or os.environ.get("QOSTODIAN_NEXUS_API_BASE", "http://nexus:8800")
	api_base = api_base or os.environ.get("QOSTODIAN_NEXUS_API_BASE", "https://nexus:8800")

[greptile-apps]

Pass-through override tightly couples to internal detection logic

The apply_guardrail override is a no-op pass-through that exists solely to ensure the method appears in type(callback).__dict__, satisfying the detection check at litellm/proxy/utils.py:868:

use_unified = "apply_guardrail" in type(callback).__dict__

The comment documents this correctly, but the class now has a hidden dependency on an implementation detail deep in proxy/utils.py. If that detection logic ever changes (e.g. using hasattr or getattr instead), this override becomes dead weight silently, with no compiler error.

Consider converting the detection check to use hasattr() or a registry-based approach to remove this coupling entirely. Alternatively, a class-level marker attribute (e.g. _uses_unified_guardrail = True) would be a more explicit and fragile-proof contract.

[mats852]

Hi @krrish-berri-2, just saw more recent staging branches merged recently. Just wanted to make sure this one was not forgotten 😁 Thanks!

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
29203053	Triggered	Generic Password	e6fb699	.circleci/config.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[veria-ai]

High: Syntax error disables all guardrail enforcement

This PR adds a Qostodian Nexus guardrail integration, secret redaction improvements, cost calculation fixes, and SQL query enhancements. However, a missing closing parenthesis in litellm/types/guardrails.py introduces a SyntaxError that prevents the guardrails type module from loading, which would silently disable every configured guardrail (content filtering, PII detection, prompt injection blocking, etc.) at proxy startup.

[veria-ai]

High: Guardrail bypass via import failure

The import of QostodianNexusConfigModel is missing its closing ). This causes a SyntaxError on this module, which is imported by every guardrail hook in the proxy. At startup, all guardrails (PII masking, prompt injection detection, content filtering, etc.) will fail to load. An attacker who knows the proxy is running this version can send requests that bypass all configured guardrails.

Suggested change

	QostodianNexusConfigModel,
	QostodianNexusConfigModel,
	)
	from litellm.types.proxy.guardrails.guardrail_hooks.hiddenlayer import (

The best fix is to remove the unused import line from litellm/_logging.py and leave behavior unchanged.

• General approach: delete imports that are not referenced.
• Specific change: in the import block at the top of litellm/_logging.py, remove:
  • from litellm.litellm_core_utils.secret_redaction import redact_string
• No additional methods, definitions, or dependency changes are required.

[greptile-apps]

SyntaxError — ( was never closed

The opening parenthesis on line 35 is never closed before the next from statement starts on line 37. Python will raise SyntaxError: '(' was never closed when loading the module, breaking every import of litellm.types.guardrails (and therefore the entire proxy).

Confirmed with ast.parse:

SyntaxError: '(' was never closed
  File "litellm/types/guardrails.py", line 35
    from litellm.types.proxy.guardrails.guardrail_hooks.qohash import (

Fix — add the missing ) and reorder the HiddenlayerGuardrailConfigModel import:

Suggested change

	from litellm.types.proxy.guardrails.guardrail_hooks.qohash import (
	QostodianNexusConfigModel,
	from litellm.types.proxy.guardrails.guardrail_hooks.hiddenlayer import (
	HiddenlayerGuardrailConfigModel
	)
	from litellm.types.proxy.guardrails.guardrail_hooks.qohash import (
	QostodianNexusConfigModel,
	)
	from litellm.types.proxy.guardrails.guardrail_hooks.hiddenlayer import (
	HiddenlayerGuardrailConfigModel,
	)

[greptile-apps]

Too many files changed for review. (1328 files found, 100 file limit)

[Sameerlite]

clean PR: #25856