chore(release): backport #30543, #30542 to stable/1.88.x and cut 1.88.3

[yuneng-berri]

Relevant issues

Backports two already-merged guardrail reliability fixes from litellm_internal_staging onto stable/1.88.x and cuts 1.88.3. Both target the guardrail subsystem and neither has shipped on any release line yet.

#30543 fixes a model-level CustomGuardrail (attached via litellm_params.guardrails) having its async_pre_call_hook invoked twice per request; once by the proxy pre-call loop and again by async_pre_call_deployment_hook after the router spreads the model-level guardrails into the top-level kwargs. The proxy loop now records, on the request data that flows downstream, that it already ran a given guardrail, and the deployment hook skips it when that marker is present. Direct-SDK usage never runs the proxy loop, so the deployment hook stays the sole invocation there and still fires exactly once. The marker is tagged with a per-process token and the marker key is stripped from untrusted caller metadata, so a request body cannot pre-seed the marker to suppress a model-level guardrail.

#30542 fixes InMemoryGuardrailHandler re-initializing DB-backed guardrails on every poll cycle. The change compared an in-memory LitellmParams (whose model_dump() carries every field default and coerces enums) against the raw sparse dict loaded from the DB, so the two shapes never compared equal and the guardrail was rebuilt every poll; each rebuild left the prior callback instance stranded in the success/failure/async callback lists. Both sides are now normalized through LitellmParams(...).model_dump() before diffing, and a deleted guardrail's callback is purged from every callback list rather than only litellm.callbacks.

Linear ticket

N/A

Pre-Submission checklist

• I have added meaningful tests
• My PR's scope is as isolated as possible; it only solves the backport
• I have requested a Greptile review

What is included

In cherry-pick (staging merge) order, each carrying its -x provenance footer:

• fix(guardrails): run pre_call hook once for model-level guardrails #30543 fix(guardrails): run pre_call hook once for model-level guardrails (from 4faeabc)
• fix(guardrails): stop re-initializing DB guardrails on every poll #30542 fix(guardrails): stop re-initializing DB guardrails on every poll (from 9fa74ad)

Then the version bump 1.88.2 -> 1.88.3 and a uv.lock refresh. The lock diff is the litellm self-version and the relative exclude-newer = "3 days" snapshot moving forward; no dependency version changed.

#30274 (populate access_via_team_ids on /v1/model/info) was requested but is already present on this line; it was backported earlier as commit 80f0c38 via #30408, so it is not re-picked here.

Adaptation notes

#30542 is a verbatim cherry-pick (patch-id identical to staging). #30543 is adapted in one file; the fix's own added code is preserved byte-for-byte.

litellm/proxy/utils.py: staging added callback.mark_pre_call_hook_ran(data) inside the guardrail-execution block of ProxyLogging.pre_call_hook. On stable/1.88.x that block lives in a dedicated method, ProxyLogging._process_guardrail_callback, which the pre-call loop routes every CustomGuardrail through, and which has no except SensitiveDataRouteException branch (that branch is pre-existing staging code unrelated to this fix). The marker line is placed at the structurally equivalent location, after the response-processing block and before the method's except Exception handler; staging's unrelated except SensitiveDataRouteException block was deliberately not imported. A second, purely cosmetic divergence: in litellm/integrations/custom_guardrail.py the new per-process token and helper methods are byte-identical to staging, but git anchored the insertion at a different surrounding context because the nearby imports differ on this line.

Known noise on this line

The targeted test set (the four test files these PRs add to or modify) ran 73 passed, 0 failures on the line tip before any pick, so there is no pre-existing red to discount in this set. Separately, tests/.../test_guardrail_coverage.py in the broader suite fails locally on a missing optional detect_secrets dependency in enterprise code untouched by these picks; that is an environment gap, not introduced here.

Screenshots / Proof of Fix

Live proxy on the backport branch (real OpenAI API). The picks edit the pre-call serving path, so the first signal is that normal request flow through _process_guardrail_callback is intact.

$ curl -s http://localhost:4010/v1/chat/completions \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Reply with exactly: POSTPICK_OK"}],"max_tokens":10}'
content: POSTPICK_OK
model: gpt-4o

$ # streaming completion through the same pre-call path
$ curl -s -N http://localhost:4010/v1/chat/completions -H "Authorization: Bearer sk-1234" \
    -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Say hi"}],"max_tokens":10,"stream":true}' | grep -c '^data:'
13

$ # generated (scoped) key, auth path
$ curl -s http://localhost:4010/v1/chat/completions -H "Authorization: Bearer sk-...generated..." \
    -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Reply with exactly: SCOPED_POST_OK"}],"max_tokens":10}'
content: SCOPED_POST_OK

Each pick delivers its claim on this line:

• fix(guardrails): run pre_call hook once for model-level guardrails #30543: tests/test_litellm/proxy/test_model_level_guardrails.py::test_pre_call_hook_runs_once_with_model_level_guardrails drives the real ProxyLogging.pre_call_hook (through the adapted _process_guardrail_callback) and then async_pre_call_deployment_hook, asserting the guardrail's hook ran exactly once. It would fail if the marker were misplaced. test_deployment_hook_ignores_forged_caller_marker confirms a forged marker cannot suppress a guardrail
• fix(guardrails): stop re-initializing DB guardrails on every poll #30542: the registry and callback-manager tests confirm an unchanged sparse-DB guardrail no longer re-initializes on poll and that a deleted guardrail's callback is purged from every list

Targeted test delta: 73 passed before the picks, 86 passed after, 0 new failures. A deep adversarial gauntlet run (universal direction, 7 lenses including two independent live-proxy reproductions) returned SURVIVED with all three sub-claims (symbol resolution, each pick delivers its claim, no broken existing caller) holding and zero verified regressions.

Type

🐛 Bug Fix

Changes

Two guardrail reliability backports onto stable/1.88.x with the 1.88.3 version cut. No schema, dependency, or auth-default changes.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ yuneng-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 20.00000% with 52 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/custom_guardrail.py	17.14%	29 Missing ⚠️
litellm/proxy/guardrails/guardrail_registry.py	15.78%	16 Missing ⚠️
litellm/proxy/policy_engine/pipeline_executor.py	0.00%	4 Missing ⚠️
...llm/litellm_core_utils/logging_callback_manager.py	33.33%	2 Missing ⚠️
litellm/proxy/utils.py	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This backport cherry-picks two guardrail reliability fixes onto stable/1.88.x and cuts 1.88.3. Both changes are isolated to the guardrail subsystem with no schema, dependency, or auth-default changes.

• fix(guardrails): run pre_call hook once for model-level guardrails #30543 (double-execution fix): A per-process secret token and a mark_pre_call_hook_ran / _pre_call_hook_already_ran pair on CustomGuardrail let the proxy pre-call loop stamp the request data after executing a guardrail; async_pre_call_deployment_hook skips the hook when that stamp is present. The key is stripped from caller-supplied metadata (_UNTRUSTED_METADATA_CONTROL_FIELDS) and from backend-forwarded metadata (LITELLM_PROXY_INTERNAL_METADATA_KEYS), so a client cannot forge the marker to bypass a guardrail. Direct-SDK usage never sets the stamp, so the deployment hook remains the sole execution path there.
• fix(guardrails): stop re-initializing DB guardrails on every poll #30542 (poll re-initialization fix): _normalize_litellm_params_for_comparison runs both sides of the change-detection diff through LitellmParams(...).model_dump() before comparing, making the shapes comparable; delete_in_memory_guardrail now uses remove_callback_from_all_lists to purge stale callback instances from all five lists instead of only litellm.callbacks.

Confidence Score: 5/5

Straightforward backport with surgical, well-tested changes to the guardrail subsystem; no regressions to existing behavior expected.

Both fixes are well-isolated: the double-execution fix is gated by isinstance(callback, CustomGuardrail) in the pipeline path, _process_guardrail_callback in utils.py only receives CustomGuardrail objects, and the forge-protection via the per-process token is verified by a dedicated test. The poll-fix normalization handles the ValidationError fallback gracefully. Tests are purely mocked, additive, and cover the main edge cases. No auth, schema, or dependency changes.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/custom_guardrail.py	Adds per-process token, mark_pre_call_hook_ran, and _pre_call_hook_already_ran to CustomGuardrail; early-returns in async_pre_call_deployment_hook when the proxy loop already ran the hook. Logic is sound, token forge protection is present, both metadata buckets are read correctly.
litellm/proxy/utils.py	Adds callback.mark_pre_call_hook_ran(data) to _process_guardrail_callback after the response is merged into data, ensuring the marker travels on the final dict that flows downstream to the deployment hook.
litellm/proxy/policy_engine/pipeline_executor.py	Marks both data and response after the hook in the pipeline path; when the hook returns a fresh dict that becomes modified_data, the marker on response ensures it carries through to the caller.
litellm/proxy/guardrails/guardrail_registry.py	Adds _normalize_litellm_params_for_comparison to canonicalize both sides through LitellmParams.model_dump() before diffing, eliminating the false-change on every poll. delete_in_memory_guardrail now uses remove_callback_from_all_lists to purge stale instances from all five lists.
litellm/litellm_core_utils/logging_callback_manager.py	Adds remove_callback_from_all_lists helper that fans out to all five callback lists. The underlying remove_callback_from_list_by_object handles duplicates correctly.
litellm/proxy/litellm_pre_call_utils.py	Adds PRE_CALL_EXECUTED_GUARDRAILS_KEY to _UNTRUSTED_METADATA_CONTROL_FIELDS, preventing callers from pre-seeding the marker via request metadata.
litellm/proxy/common_utils/callback_utils.py	Adds PRE_CALL_EXECUTED_GUARDRAILS_KEY to LITELLM_PROXY_INTERNAL_METADATA_KEYS so the key is not forwarded to backends.
litellm/constants.py	Adds PRE_CALL_EXECUTED_GUARDRAILS_KEY = '_pre_call_executed_guardrails' constant, correctly placed per the sentinel-in-constants rule.
tests/test_litellm/proxy/test_model_level_guardrails.py	Adds four integration-level tests covering: exact-once execution via proxy->deployment path, fresh-dict response, direct-SDK path, and forge-protection. All use mocks, no real network calls.
tests/test_litellm/integrations/test_custom_guardrail.py	Adds four unit tests for the new marker methods (skip when marked, run when unmarked, litellm_metadata bucket, and forge test). Purely additive; existing tests unchanged.
tests/test_litellm/proxy/guardrails/test_guardrail_registry.py	Adds seven new tests covering: unchanged params no longer register as changed, genuine changes still detected, malformed params fail-safe, delete purges all lists, and end-to-end accumulation regression. All mock-only.
tests/litellm_utils_tests/test_logging_callback_manager.py	Adds test_remove_callback_from_all_lists verifying all five lists are cleared in one call. Purely additive.
pyproject.toml	Version bump 1.88.2 -> 1.88.3 in both [project] and [tool.commitizen] sections.

_{Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.88.3" | Re-trigger Greptile}