fix(guardrails): run apply_guardrail-style model-level pre_call guardrails at deployment hook

[yucheng-berri]

Relevant issues

Linear ticket

Resolves LIT-4217

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live proxy plus Postgres, real OpenAI traffic. Config attaches the guardrail at the model level only, with default_on: false:

model_list:
  - model_name: openai-gpt-4.1-mini-2025-04-14
    litellm_params:
      model: gpt-4.1-mini
      api_key: os.environ/OPENAI_API_KEY
      guardrails: ["openai-moderation-pre"]

guardrails:
  - guardrail_name: openai-moderation-pre
    litellm_params:
      guardrail: openai_moderation
      mode: pre_call
      api_key: os.environ/OPENAI_API_KEY
      default_on: false

general_settings:
  master_key: sk-1234

Before the fix (captured at 539bc30e04, current litellm_internal_staging HEAD): flagged content sails through to OpenAI and returns 200 even though the model has a pre_call moderation guardrail attached

curl -s -w '\nHTTP %{http_code}\n' -X POST http://localhost:4217/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"openai-gpt-4.1-mini-2025-04-14","messages":[{"role":"user","content":"<flagged content>"}]}'

{"choices":[{"message":{"content":"I'm sorry you're feeling this way. It might help to talk to someone you trust about what's going on. ...","role":"assistant"}}]}
HTTP 200

After the fix (captured at 962c49553e): the identical request is blocked before reaching the provider

{"error":{"message":"Violated OpenAI moderation policy","type":"None","param":"None","code":"400","provider_specific_fields":{"error":"Violated OpenAI moderation policy","moderation_result":{"violated_categories":["harassment","harassment/threatening","violence"],"category_scores":{"harassment":0.649,"harassment/threatening":0.794,"violence":0.953, ...}}}}}
HTTP 400

Benign content on the same model still completes (HTTP 200), and the pre-existing request-body path ("guardrails": ["openai-moderation-pre"] in the body) still blocks with HTTP 400 including the guardrail_name/guardrail_mode fields, both also captured at 962c49553e. The proxy debug log shows exactly one moderation API call per blocked request, so the request-body and model-level paths do not double-run

Dual-override guardrail parity (presidio, bedrock, enkryptai, live)

To validate the behavior change for guardrails that override both async_pre_call_hook and apply_guardrail, presidio was exercised live against real analyzer and anonymizer containers (mcr.microsoft.com/presidio-analyzer, mcr.microsoft.com/presidio-anonymizer) with the guardrail attached at the model level (mode: pre_call, default_on: false) and real OpenAI traffic. The prompt embeds a name, phone number, and email and asks the model to repeat them verbatim, so the assistant reply reveals exactly what the provider received

curl -s -X POST http://localhost:4217/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"openai-gpt-4.1-mini-2025-04-14","messages":[{"role":"user","content":"My name is Jane Doe, my phone number is 212-555-0123, and my email is jane.doe@example.com. Repeat back exactly the name, phone number, and email I just gave you, verbatim, with no extra words."}]}'

All three variants returned a byte-identical assistant reply, HTTP 200:

My name is <PERSON>, my phone number is <PHONE_NUMBER>, and my email is <EMAIL_ADDRESS>.

1. Old model-level path (a89f9af353 with litellm/integrations/custom_guardrail.py reverted to base): masked via presidio's own async_pre_call_hook
2. New model-level path (a89f9af353 as-is): masked via unified_guardrail -> apply_guardrail; the debug log shows exactly one Running UnifiedLLMGuardrails pre-call hook for the request, proving the new dispatch executed
3. Request-body path (a89f9af353, "guardrails": ["presidio-pre"] in the body): masked via the unchanged canonical path

So for the most divergent-looking dual-override guardrail, the new model-level dispatch produces output identical to both the previous behavior and the request-body path

The same base-vs-fix comparison was run live for bedrock (temporary guardrail with a VIOLENCE HIGH content filter, real AWS ApplyGuardrail calls) and enkryptai (real detect API with a prompt-injection policy), on both the model-level and request-body paths. Every cell is identical before and after this PR: bedrock request-body blocks with 400 and the contentPolicy assessment on both; enkryptai blocks the injection prompt with the same status and detection body on all four cells. Two pre-existing issues surfaced by this exercise reproduce identically on the base branch and are unaffected by this PR: model-level bedrock guardrail calls fail 403 because the deployment's provider api_key is picked up as an AWS bearer token (tracked in LIT-4425), and enkryptai blocks surface as a ValueError mapped to 500 rather than a 400. panw_prisma_airs is the one dual-override guardrail not exercised live; no vendor credentials were available

The legacy-only style (a custom guardrail overriding async_pre_call_hook and not apply_guardrail) was also validated live base-vs-fix with a keyword-blocking guardrail attached at the model level: blocked content returns the same 400 with the guardrail's own error body on both paths and both code versions, and benign content completes with 200 on all cells. The block message originates inside the guardrail's async_pre_call_hook, confirming the new dispatch takes the target = self branch for legacy guardrails and leaves them untouched

Type

🐛 Bug Fix

Changes

Root cause: ProxyLogging.pre_call_hook runs before routing, so it cannot see a deployment's litellm_params.guardrails (no deployment is selected yet and metadata.model_info.id does not exist). The intended enforcement point for model-level guardrails is CustomGuardrail.async_pre_call_deployment_hook (#12968, #30543), which runs after the router spreads the chosen deployment's litellm_params into the request kwargs. That hook correctly decided the guardrail should run, then dispatched only to self.async_pre_call_hook, which is the inherited CustomLogger no-op for guardrails that implement the unified apply_guardrail interface (openai_moderation and most current guardrails). The guardrail was selected and then silently did nothing, so flagged requests returned 200 unless the client also passed "guardrails" in the request body

Fix: the deployment hook now dispatches through the same unified_guardrail path that ProxyLogging._execute_guardrail_hook already uses on the request-body path whenever the guardrail class overrides apply_guardrail. The override check lives in a new CustomGuardrail.uses_apply_guardrail_interface() helper, which is MRO-aware (it also catches an override inherited from an intermediate class). The request-body dispatch in _execute_guardrail_hook is left byte-identical to the current behavior; consolidating the remaining dispatch checks onto the shared helper is deferred to LIT-4417 because the existing pipeline tests drive that path with spec'd mocks that need reworking first. On a base SDK install without proxy extras the hook fails closed: instead of silently skipping the guardrail (the pre-fix behavior, which is a bypass of a configured security control) or crashing with a bare ModuleNotFoundError, it raises an ImportError naming the guardrail and telling the user to install litellm[proxy]. The change also replaces a banned # type: ignore on the call_type argument with a typed conditional literal

Regression tests live in tests/test_litellm/integrations/test_custom_guardrail.py (the mapped file for the changed module): an apply_guardrail-only guardrail with default_on: false must block through the deployment hook for both completion and acompletion, must strip the internal dispatch key on the pass path, and must not run when a different guardrail is requested. The blocking and pass-path tests fail on the unfixed code

Behavior changes

Guardrails that override both async_pre_call_hook and apply_guardrail (presidio, bedrock, panw_prisma_airs, enkryptai) and are attached at the model level now route through the unified apply_guardrail path at the deployment hook. This matches how the request-body path already dispatches them (_execute_guardrail_hook prefers unified for any guardrail defining apply_guardrail), so the two attachment styles now behave identically for these guardrails. No opt-out flag is added on purpose; the previous model-level behavior for them diverged from the request-body behavior for the same guardrail, which had no value worth preserving

On base SDK installs without proxy extras, an apply_guardrail-style guardrail requested at the deployment level now raises an actionable ImportError instead of silently doing nothing. Silent skip of a configured guardrail is the same failure class this PR fixes, so fail-closed is deliberate

A block raised from the model-level path does not include the guardrail_name/guardrail_mode enrichment fields the request-body path adds. That parity gap, model-level enforcement for non-completion call types (embeddings, Responses API, anthropic_messages), model-level during_call guardrails, and consolidating the remaining "apply_guardrail" in type(...).__dict__ dispatch checks are tracked in LIT-4417

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes a silent bypass in model-level pre-call guardrails: the deployment hook previously dispatched to CustomLogger's no-op async_pre_call_hook for guardrails that implement the unified apply_guardrail interface (e.g. openai_moderation), so flagged content reached the provider without being checked.

• Adds uses_apply_guardrail_interface() (MRO-aware identity check) and _deployment_pre_call_target() to route apply_guardrail-style guardrails through the existing unified_guardrail path, matching how the request-body dispatch already works.
• Fails closed on bare-SDK installs: raises an actionable ImportError naming the guardrail and pointing to pip install 'litellm[proxy]' instead of silently skipping a configured security control.
• Replaces a banned # type: ignore on the call_type argument with a typed conditional, and adds four regression tests covering the block, pass, skip, and fail-closed paths.

Confidence Score: 5/5

Safe to merge — the core dispatch logic is correct, deduplication with the request-body path is preserved, and the fail-closed behavior for missing proxy extras is intentional and well-tested.

The fix is narrowly scoped to async_pre_call_deployment_hook: it adds a new dispatch helper and wires it in, leaving the request-body path in _execute_guardrail_hook byte-identical. The live dual-override parity evidence and the four regression tests (including the fail-closed path) give strong confidence the change behaves correctly. The only finding is a test-file style concern about fastapi imports outside of proxy/.

No files require special attention beyond the noted fastapi import in the test helper.

Important Files Changed

Filename	Overview
litellm/integrations/custom_guardrail.py	Adds uses_apply_guardrail_interface() and _deployment_pre_call_target() helpers, then wires the deployment hook to dispatch through unified_guardrail instead of the no-op async_pre_call_hook for apply_guardrail-style guardrails; also replaces the banned # type: ignore with a typed conditional on call_type.
tests/test_litellm/integrations/test_custom_guardrail.py	Adds four regression tests for the new deployment-hook dispatch path (block, pass, skip, and fail-closed); test helper imports fastapi.HTTPException in a non-proxy file, violating the repo fastapi-import rule.

_{Reviews (4): Last reviewed commit: "fix(guardrails): fail closed when proxy ..." | Re-trigger Greptile}

[greptile-apps]

Behaviour change for guardrails that override both hooks, no feature flag

Per the repo's backwards-compatibility rule, behaviour changes should be gated behind a user-controlled flag. Guardrails that override both async_pre_call_hook and apply_guardrail (e.g. presidio, bedrock, panw_prisma_airs, enkryptai) and are attached at the model level previously executed their own async_pre_call_hook at this stage; they now route through unified_guardrail → apply_guardrail instead. The PR acknowledges this under "Behavior changes" but provides no opt-out. If any of those guardrails produce different results across the two paths (e.g. presidio PII scrubbing vs. its apply_guardrail variant), existing model-level attachments will silently change behaviour after upgrade.

Rule Used: What: avoid backwards-incompatible changes without... (source)

[yucheng-berri]

Acknowledged as an intentional alignment rather than gated behind a flag: the request-body path already dispatches these four guardrails through apply_guardrail, so a flag would preserve a divergence with no value. Live parity evidence for presidio, bedrock and enkryptai (identical output on every path, before and after) is in the PR description under Dual-override guardrail parity

[yucheng-berri]

@greptileai

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a89f9af. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4217_model_guardrail_precall (d2f401a) with litellm_internal_staging (53aaabb)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (001457a) during the generation of this report, so 53aaabb was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

@greptileai

[yucheng-berri]

Added live parity evidence for the dual-override guardrail concern: presidio attached at the model level was exercised against real presidio analyzer/anonymizer containers and real OpenAI traffic on three variants (old model-level dispatch, new model-level dispatch, request-body dispatch). All three produced a byte-identical masked reply, and the debug log confirms the new path executed through UnifiedLLMGuardrails exactly once. Details in the PR description under Dual-override guardrail parity

@greptileai

[greptile-apps]

The parity evidence for presidio is convincing — three independent dispatch paths producing a byte-identical masked reply, with the debug log confirming single-execution, directly addresses the core of the concern.

One narrow gap remains: bedrock, panw_prisma_airs, and enkryptai also override both hooks and are affected by the dispatch change, but weren't empirically exercised. The presidio result is a strong existence proof that async_pre_call_hook and apply_guardrail are functionally equivalent for dual-override guardrails (they represent the same logical check implemented twice), but if any of those three have a live environment available during the review window, a quick smoke test would close the loop entirely.

That said, the central argument stands: for guardrails that implement only apply_guardrail, the previous model-level path was a silent no-op, so the fix is correcting broken behavior rather than changing working behavior. The backwards-compatibility rule targets the latter. Withdrawing my objection on the feature-flag requirement — the presidio evidence is sufficient to accept this as a bug fix rather than a behavior change needing an opt-out.

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[yucheng-berri]

Addressed the open security issue: the deployment hook now fails closed when proxy extras are absent, raising an ImportError that names the guardrail and points to pip install 'litellm[proxy]' instead of silently skipping a configured guardrail. Regression test added (test_fails_closed_when_proxy_extras_missing). Also extended the live dual-override parity evidence in the PR description to bedrock and enkryptai using real vendor APIs; every model-level and request-body cell is byte-identical before and after this PR. A pre-existing bedrock guardrail bug surfaced by that exercise (deployment provider api_key picked up as an AWS bearer token at the deployment hook) is tracked separately in LIT-4425

@greptileai