Skip to content

feat(guardrails): wire apply_guardrail into proxy logging callbacks - #28970

Merged
mateo-berri merged 11 commits into
litellm_internal_stagingfrom
litellm_apply_guardrail_logging_callbacks
May 28, 2026
Merged

feat(guardrails): wire apply_guardrail into proxy logging callbacks#28970
mateo-berri merged 11 commits into
litellm_internal_stagingfrom
litellm_apply_guardrail_logging_callbacks

Conversation

@Sameerlite

@Sameerlite Sameerlite commented May 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Route /apply_guardrail through proxy pre/post hooks and LiteLLM success/failure logging handlers so configured callbacks (Langfuse, OTEL, etc.) run on guardrail-only requests.
  • Set logging call type to pass_through_endpoint with messages/response payload so Langfuse records input and output on this endpoint.
  • Add regression test covering logging pipeline invocation.

Fixes LIT-3139

Test plan

  • poetry run pytest tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py::test_apply_guardrail_not_found -v
  • poetry run pytest tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py::test_apply_guardrail_execution_error -v
  • poetry run pytest tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py::test_apply_guardrail_invokes_logging_pipeline -v
image image

Note

Low Risk
Observability and hook wiring around an existing guardrail test endpoint; hook failures are isolated so API responses still return.

Overview
The /apply_guardrail endpoint now follows the same proxy request and logging path as LLM routes instead of running guardrails in isolation.

It uses ProxyBaseLLMRequestProcessing.common_processing_pre_call_logic with a new apply_guardrail route type (registered in common_request_processing), accepts the FastAPI Request, and on success runs post_call_success_hook, LiteLLM async_success_handler, and sync success_handler. Failures go through the matching failure hooks. The logging object is patched so call_type is pass_through_endpoint and messages/response are shaped for integrations like Langfuse and OTEL. Post-call guardrails still get the correct input_type via _resolve_guardrail_input_type.

Tests add a shared mock_proxy_logging_ctx fixture and assert the logging pipeline is invoked on success.

Reviewed by Cursor Bugbot for commit 156e0d2. Bugbot is set up for automated code reviews on this repo. Configure here.

Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.80645% with 15 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/proxy/guardrails/guardrail_endpoints.py 75.80% 15 Missing ⚠️

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR wires the /apply_guardrail endpoint into the existing proxy pre/post hook pipeline so that configured callbacks (Langfuse, OTEL, etc.) observe guardrail-only requests the same way they observe LLM calls.

  • Three helpers (_resolve_guardrail_input_type, _patch_logging_obj_for_guardrail, _emit_guardrail_success_logs) encapsulate the new logging plumbing; the success/failure hook separation is correctly placed outside the core try-block so a hook failure cannot flip a successful guardrail call into a failure trace.
  • common_processing_pre_call_logic is registered for a new \"apply_guardrail\" route type, and all existing tests are updated with the required fastapi_request parameter and proxy-global mocks; a new test_apply_guardrail_invokes_logging_pipeline test verifies end-to-end invocation of the logging pipeline.

Confidence Score: 4/5

Safe to merge; the hook failures are fully isolated from guardrail responses and the existing success/failure callback separation is correctly implemented.

The GUARDRAIL_REGISTRY lookup currently happens before common_processing_pre_call_logic, so a 404 for an unknown guardrail produces no LiteLLM-level span in Langfuse/OTEL even though proxy_logging_obj.post_call_failure_hook fires. This is an observable gap between how 404 errors are traced here versus on other proxy endpoints, and whether it is intentional should be confirmed before merging.

litellm/proxy/guardrails/guardrail_endpoints.py — the ordering of the registry lookup relative to common_processing_pre_call_logic determines whether not-found errors produce a full trace

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_endpoints.py Adds pre-call processing, success/failure logging hooks, and a new helper trio to the apply_guardrail endpoint; success/failure hook separation is correct but 404 errors skip pre-call processing and produce no LiteLLM-level trace
litellm/proxy/common_request_processing.py Single-line addition registering 'apply_guardrail' as a valid route_type in common_processing_pre_call_logic; change is minimal and correct
tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py Three existing tests updated with proper mocks for proxy globals, plus new test_apply_guardrail_invokes_logging_pipeline asserting the full logging pipeline
tests/enterprise/litellm_enterprise/proxy/guardrails/conftest.py New conftest providing a shared mock_proxy_logging_ctx fixture that patches all proxy-server globals required by the updated apply_guardrail endpoint
tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py Existing tests updated to add fastapi_request=Mock() parameter and wrap with mock_proxy_logging_ctx; no logic changes to assertions
tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py Bedrock integration test updated to add fastapi_request=Mock() parameter and mock_proxy_logging_ctx wrapper; no assertion changes

Reviews (5): Last reviewed commit: "fix(guardrails): use update_messages to ..." | Re-trigger Greptile

Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
@Sameerlite

Copy link
Copy Markdown
Contributor Author

Fixed both issues:

P1 — Success logging is now outside the except block entirely. post_call_success_hook, async_success_handler, and success_handler only run after the guardrail execution completes cleanly. A failure in any of those hooks no longer triggers the failure path.

P2model_call_details["messages"] now uses request.messages when provided, falling back to a single synthetic user message only when the caller sends text only.

@greptileai

@greptile-apps

greptile-apps Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Both fixes look correct. P1 is properly addressed — the success handlers are now entirely outside the try/except block and only reachable on a clean execution path. P2 is handled with the appropriate fallback.

One remaining edge case worth noting: if post_call_success_hook or async_success_handler raises, the exception will bubble up to the caller as an unhandled error, returning a 500 even though the guardrail itself succeeded. The response was already computed — you may want to wrap the success logging block in a try/except that logs but doesn't re-raise, similar to how other endpoints handle hook failures gracefully. Whether that's worth doing here depends on how critical logging failures should be treated vs. delivering the response to the client.

Otherwise the changes look solid. The structure is clean, the test coverage is good, and the messages fallback logic is sensible.

…d response

- Capture return value from post_call_success_hook so callback-modified
  responses propagate to the caller.
- Wrap success/failure logging calls in defensive try/except so logging
  infrastructure failures don't replace the user-visible response or mask
  the original guardrail exception.

Co-authored-by: Yassin Kortam <yassin@berri.ai>
@CLAassistant

CLAassistant commented May 27, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Sameerlite
✅ mateo-berri
❌ cursoragent
You have signed the CLA already but the status is still pending? Let us recheck it.

@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai update the score

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Failure in async_failure_handler silently skips post_call_failure_hook
    • Split the failure path into two independent try/except blocks so a failure in async_failure_handler no longer prevents post_call_failure_hook from running.
  • ✅ Fixed: Logged response diverges from actually returned response
    • Moved response_for_logging construction into _emit_guardrail_success_logs after post_call_success_hook so logging reflects the (possibly hook-modified) response actually returned to the caller.
Preview (9b2e2675a2)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -839,6 +839,7 @@
             "aget_run",
             "acancel_run",
             "adelete_run",
+            "apply_guardrail",
         ],
         version: Optional[str] = None,
         user_model: Optional[str] = None,

diff --git a/litellm/proxy/guardrails/guardrail_endpoints.py b/litellm/proxy/guardrails/guardrail_endpoints.py
--- a/litellm/proxy/guardrails/guardrail_endpoints.py
+++ b/litellm/proxy/guardrails/guardrail_endpoints.py
@@ -10,7 +10,7 @@
 from typing import Any, Dict, List, Literal, Optional, Type, TypeVar, Union, cast
 from urllib.parse import urlparse
 
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, HTTPException, Request
 from pydantic import BaseModel
 
 from litellm.proxy.common_utils.path_utils import safe_join
@@ -2187,9 +2187,88 @@
         )
 
 
+def _resolve_guardrail_input_type(
+    active_guardrail: CustomGuardrail, input_type: str
+) -> Literal["request", "response"]:
+    """Return the effective input_type, auto-upgrading to 'response' for post_call guardrails."""
+    if input_type == "request":
+        hook = getattr(active_guardrail, "event_hook", None)
+        if hook == GuardrailEventHooks.post_call or hook == "post_call":
+            return "response"
+    return "response" if input_type == "response" else "request"
+
+
+def _patch_logging_obj_for_guardrail(
+    litellm_logging_obj: Any, request: ApplyGuardrailRequest
+) -> None:
+    """Configure the logging object so Langfuse/OTEL extract input and output correctly."""
+    litellm_logging_obj.call_type = "pass_through_endpoint"
+    litellm_logging_obj.model_call_details["call_type"] = "pass_through_endpoint"
+    litellm_logging_obj.model_call_details["messages"] = (
+        request.messages if request.messages else [{"role": "user", "content": request.text}]
+    )
+
+
+async def _emit_guardrail_success_logs(
+    proxy_logging_obj: Any,
+    litellm_logging_obj: Any,
+    data: dict,
+    user_api_key_dict: UserAPIKeyAuth,
+    response: ApplyGuardrailResponse,
+    start_time: datetime,
+) -> ApplyGuardrailResponse:
+    """Fire proxy and LiteLLM success hooks after a successful guardrail run.
+
+    Each hook is wrapped defensively so a callback failure never prevents the
+    caller from receiving the guardrail response.  Returns the (possibly
+    hook-modified) response.
+    """
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+
+    try:
+        modified = await proxy_logging_obj.post_call_success_hook(
+            data=data,
+            user_api_key_dict=user_api_key_dict,
+            response=response,
+        )
+        if isinstance(modified, ApplyGuardrailResponse):
+            response = modified
+    except Exception:
+        verbose_proxy_logger.exception("apply_guardrail: post_call_success_hook failed")
+
+    # Build the logging payload after post_call_success_hook so that logged
+    # data matches what the caller actually receives if the hook modified
+    # the response.
+    response_for_logging = {"response": response.model_dump(exclude_none=True)}
+
+    if litellm_logging_obj is not None:
+        end_time = datetime.now(timezone.utc)
+        try:
+            await litellm_logging_obj.async_success_handler(
+                result=response_for_logging,
+                start_time=start_time,
+                end_time=end_time,
+                cache_hit=False,
+            )
+            thread_pool_executor.submit(
+                litellm_logging_obj.success_handler,
+                response_for_logging,
+                start_time,
+                end_time,
+                False,
+            )
+        except Exception:
+            verbose_proxy_logger.exception("apply_guardrail: success logging failed")
+
+    return response
+
+
 @router.post("/guardrails/apply_guardrail", response_model=ApplyGuardrailResponse)
 @router.post("/apply_guardrail", response_model=ApplyGuardrailResponse)
 async def apply_guardrail(
+    fastapi_request: Request,
     request: ApplyGuardrailRequest,
     user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
 ):
@@ -2198,8 +2277,29 @@
 
     This endpoint allows testing guardrails by applying them to custom text inputs.
     """
+    import traceback
+
+    from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+    from litellm.proxy.proxy_server import (
+        general_settings,
+        proxy_config,
+        proxy_logging_obj,
+        version,
+    )
     from litellm.proxy.utils import handle_exception_on_proxy
 
+    data: dict = {
+        "guardrail_name": request.guardrail_name,
+        "input": [request.text],
+        "messages": request.messages or [],
+        "metadata": {"route": "/apply_guardrail"},
+    }
+    litellm_logging_obj = None
+    start_time = datetime.now(timezone.utc)
+
     try:
         active_guardrail: Optional[CustomGuardrail] = (
             GUARDRAIL_REGISTRY.get_initialized_guardrail_callback(
@@ -2212,37 +2312,70 @@
                 detail=f"Guardrail '{request.guardrail_name}' not found. Please ensure the guardrail is configured in your LiteLLM proxy.",
             )
 
-        request_data: dict = {}
-        if request.messages:
-            request_data["messages"] = request.messages
+        request_processor = ProxyBaseLLMRequestProcessing(data=data)
+        data, litellm_logging_obj = await request_processor.common_processing_pre_call_logic(
+            request=fastapi_request,
+            general_settings=general_settings,
+            user_api_key_dict=user_api_key_dict,
+            version=version,
+            proxy_logging_obj=proxy_logging_obj,
+            proxy_config=proxy_config,
+            route_type="apply_guardrail",
+        )
 
-        # Auto-detect input_type: if the caller didn't specify "response" but the
-        # guardrail only runs post_call (e.g. LLM-as-a-judge), use "response" so
-        # the test actually exercises the guardrail logic.
-        from litellm.types.guardrails import GuardrailEventHooks
+        if litellm_logging_obj is not None:
+            _patch_logging_obj_for_guardrail(litellm_logging_obj, request)
 
-        resolved_input_type = request.input_type
-        if resolved_input_type == "request":
-            hook = getattr(active_guardrail, "event_hook", None)
-            if hook == GuardrailEventHooks.post_call or hook == "post_call":
-                resolved_input_type = "response"
-
-        _input_type: Literal["request", "response"] = (
-            "response" if resolved_input_type == "response" else "request"
-        )
+        request_data: dict = {"messages": request.messages} if request.messages else {}
+        _input_type = _resolve_guardrail_input_type(active_guardrail, request.input_type)
         guardrailed_inputs = await active_guardrail.apply_guardrail(
             inputs={"texts": [request.text]},
             request_data=request_data,
             input_type=_input_type,
         )
         response_text = guardrailed_inputs.get("texts", [])
-
-        return ApplyGuardrailResponse(
+        response = ApplyGuardrailResponse(
             response_text=response_text[0] if response_text else request.text
         )
     except Exception as e:
+        try:
+            if litellm_logging_obj is not None and not isinstance(e, HTTPException):
+                await litellm_logging_obj.async_failure_handler(
+                    exception=e,
+                    traceback_exception=traceback.format_exc(),
+                )
+                thread_pool_executor.submit(
+                    litellm_logging_obj.failure_handler,
+                    e,
+                    traceback.format_exc(),
+                )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: async_failure_handler failed"
+            )
+        try:
+            await proxy_logging_obj.post_call_failure_hook(
+                user_api_key_dict=user_api_key_dict,
+                original_exception=e,
+                request_data=data,
+            )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: post_call_failure_hook failed"
+            )
         raise handle_exception_on_proxy(e)
 
+    # Success logging outside except so a hook error never triggers failure handlers.
+    response = await _emit_guardrail_success_logs(
+        proxy_logging_obj=proxy_logging_obj,
+        litellm_logging_obj=litellm_logging_obj,
+        data=data,
+        user_api_key_dict=user_api_key_dict,
+        response=response,
+        start_time=start_time,
+    )
+    return response
 
+
 # Usage (dashboard) endpoints: overview, detail, logs
 router.include_router(guardrails_usage_router)

diff --git a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
--- a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
+++ b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
@@ -1159,7 +1159,11 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error details
     assert str(exc_info.value.code) == "404"
@@ -1186,6 +1190,25 @@
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
     )
 
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_failure_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_failure_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mocker.patch("litellm.litellm_core_utils.thread_pool_executor.executor")
+
     # Create request
     request = ApplyGuardrailRequest(
         guardrail_name="test-guardrail", text="Test input text with forbidden content"
@@ -1196,13 +1219,71 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error is properly handled
     assert "Bedrock guardrail failed" in str(exc_info.value.message)
 
 
 @pytest.mark.asyncio
+async def test_apply_guardrail_invokes_logging_pipeline(mocker):
+    mock_guardrail = mocker.Mock()
+    mock_guardrail.apply_guardrail = AsyncMock(return_value={"texts": ["masked"]})
+
+    mock_registry = mocker.Mock()
+    mock_registry.get_initialized_guardrail_callback.return_value = mock_guardrail
+    mocker.patch(
+        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
+    )
+
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_success_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mock_executor = mocker.Mock()
+    mocker.patch(
+        "litellm.litellm_core_utils.thread_pool_executor.executor", mock_executor
+    )
+
+    request = ApplyGuardrailRequest(
+        guardrail_name="test-guardrail", text="hello@example.com"
+    )
+    response = await apply_guardrail(
+        fastapi_request=mocker.Mock(),
+        request=request,
+        user_api_key_dict=UserAPIKeyAuth(),
+    )
+
+    assert response.response_text == "masked"
+    mock_processor.common_processing_pre_call_logic.assert_awaited_once()
+    mock_proxy_logging.post_call_success_hook.assert_awaited_once()
+    mock_logging_obj.async_success_handler.assert_awaited_once()
+    assert mock_logging_obj.call_type == "pass_through_endpoint"
+    mock_executor.submit.assert_called_once()
+    assert mock_logging_obj.async_success_handler.await_args.kwargs["result"] == {
+        "response": {"response_text": "masked"}
+    }
+
+
+@pytest.mark.asyncio
 async def test_get_guardrail_info_endpoint_config_guardrail(mocker):
     """
     Test get_guardrail_info endpoint returns proper response when guardrail is found in config.

You can send follow-ups to the cloud agent here.

Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Sameerlite and others added 3 commits May 27, 2026 12:37
…nse for logging

- Split async_failure_handler and post_call_failure_hook into independent
  try/except blocks so a callback bug in one does not silently skip the
  other.
- Build response_for_logging inside _emit_guardrail_success_logs after
  post_call_success_hook runs, so logged data matches the response the
  caller actually receives when the hook modifies the response.

Co-authored-by: Yassin Kortam <yassin@berri.ai>
…pi_request param

- Run black on guardrail_endpoints.py to fix CI formatting check
- Add _mock_proxy_logging() helper to enterprise guardrail tests to patch
  proxy-server globals imported at call time
- Pass fastapi_request=Mock() in all direct apply_guardrail test calls
  to match updated function signature

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread litellm/proxy/guardrails/guardrail_endpoints.py
…k in apply_guardrail

Co-authored-by: Yassin Kortam <yassin@berri.ai>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Sync handler silently skipped when async handler fails
    • Split each sync and async handler call into its own try/except in both _emit_guardrail_success_logs and the failure path of apply_guardrail so a failure in one channel no longer prevents the other from running.
Preview (2a400e80e0)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -839,6 +839,7 @@
             "aget_run",
             "acancel_run",
             "adelete_run",
+            "apply_guardrail",
         ],
         version: Optional[str] = None,
         user_model: Optional[str] = None,

diff --git a/litellm/proxy/guardrails/guardrail_endpoints.py b/litellm/proxy/guardrails/guardrail_endpoints.py
--- a/litellm/proxy/guardrails/guardrail_endpoints.py
+++ b/litellm/proxy/guardrails/guardrail_endpoints.py
@@ -10,7 +10,7 @@
 from typing import Any, Dict, List, Literal, Optional, Type, TypeVar, Union, cast
 from urllib.parse import urlparse
 
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, HTTPException, Request
 from pydantic import BaseModel
 
 from litellm.proxy.common_utils.path_utils import safe_join
@@ -2187,9 +2187,97 @@
         )
 
 
+def _resolve_guardrail_input_type(
+    active_guardrail: CustomGuardrail, input_type: str
+) -> Literal["request", "response"]:
+    """Return the effective input_type, auto-upgrading to 'response' for post_call guardrails."""
+    if input_type == "request":
+        hook = getattr(active_guardrail, "event_hook", None)
+        if hook == GuardrailEventHooks.post_call or hook == "post_call":
+            return "response"
+    return "response" if input_type == "response" else "request"
+
+
+def _patch_logging_obj_for_guardrail(
+    litellm_logging_obj: Any, request: ApplyGuardrailRequest
+) -> None:
+    """Configure the logging object so Langfuse/OTEL extract input and output correctly."""
+    litellm_logging_obj.call_type = "pass_through_endpoint"
+    litellm_logging_obj.model_call_details["call_type"] = "pass_through_endpoint"
+    litellm_logging_obj.model_call_details["messages"] = (
+        request.messages
+        if request.messages
+        else [{"role": "user", "content": request.text}]
+    )
+
+
+async def _emit_guardrail_success_logs(
+    proxy_logging_obj: Any,
+    litellm_logging_obj: Any,
+    data: dict,
+    user_api_key_dict: UserAPIKeyAuth,
+    response: ApplyGuardrailResponse,
+    start_time: datetime,
+) -> ApplyGuardrailResponse:
+    """Fire proxy and LiteLLM success hooks after a successful guardrail run.
+
+    Each hook is wrapped defensively so a callback failure never prevents the
+    caller from receiving the guardrail response.  Returns the (possibly
+    hook-modified) response.
+    """
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+
+    try:
+        modified = await proxy_logging_obj.post_call_success_hook(
+            data=data,
+            user_api_key_dict=user_api_key_dict,
+            response=response,
+        )
+        if isinstance(modified, ApplyGuardrailResponse):
+            response = modified
+    except Exception:
+        verbose_proxy_logger.exception("apply_guardrail: post_call_success_hook failed")
+
+    # Build the logging payload after post_call_success_hook so that logged
+    # data matches what the caller actually receives if the hook modified
+    # the response.
+    response_for_logging = {"response": response.model_dump(exclude_none=True)}
+
+    if litellm_logging_obj is not None:
+        end_time = datetime.now(timezone.utc)
+        try:
+            await litellm_logging_obj.async_success_handler(
+                result=response_for_logging,
+                start_time=start_time,
+                end_time=end_time,
+                cache_hit=False,
+            )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: async_success_handler failed"
+            )
+        try:
+            thread_pool_executor.submit(
+                litellm_logging_obj.success_handler,
+                response_for_logging,
+                start_time,
+                end_time,
+                False,
+            )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: success_handler submit failed"
+            )
+
+    return response
+
+
 @router.post("/guardrails/apply_guardrail", response_model=ApplyGuardrailResponse)
 @router.post("/apply_guardrail", response_model=ApplyGuardrailResponse)
 async def apply_guardrail(
+    fastapi_request: Request,
     request: ApplyGuardrailRequest,
     user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
 ):
@@ -2198,8 +2286,29 @@
 
     This endpoint allows testing guardrails by applying them to custom text inputs.
     """
+    import traceback
+
+    from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+    from litellm.proxy.proxy_server import (
+        general_settings,
+        proxy_config,
+        proxy_logging_obj,
+        version,
+    )
     from litellm.proxy.utils import handle_exception_on_proxy
 
+    data: dict = {
+        "guardrail_name": request.guardrail_name,
+        "input": [request.text],
+        "messages": request.messages or [],
+        "metadata": {"route": "/apply_guardrail"},
+    }
+    litellm_logging_obj = None
+    start_time = datetime.now(timezone.utc)
+
     try:
         active_guardrail: Optional[CustomGuardrail] = (
             GUARDRAIL_REGISTRY.get_initialized_guardrail_callback(
@@ -2212,23 +2321,25 @@
                 detail=f"Guardrail '{request.guardrail_name}' not found. Please ensure the guardrail is configured in your LiteLLM proxy.",
             )
 
-        request_data: dict = {}
-        if request.messages:
-            request_data["messages"] = request.messages
+        request_processor = ProxyBaseLLMRequestProcessing(data=data)
+        data, litellm_logging_obj = (
+            await request_processor.common_processing_pre_call_logic(
+                request=fastapi_request,
+                general_settings=general_settings,
+                user_api_key_dict=user_api_key_dict,
+                version=version,
+                proxy_logging_obj=proxy_logging_obj,
+                proxy_config=proxy_config,
+                route_type="apply_guardrail",
+            )
+        )
 
-        # Auto-detect input_type: if the caller didn't specify "response" but the
-        # guardrail only runs post_call (e.g. LLM-as-a-judge), use "response" so
-        # the test actually exercises the guardrail logic.
-        from litellm.types.guardrails import GuardrailEventHooks
+        if litellm_logging_obj is not None:
+            _patch_logging_obj_for_guardrail(litellm_logging_obj, request)
 
-        resolved_input_type = request.input_type
-        if resolved_input_type == "request":
-            hook = getattr(active_guardrail, "event_hook", None)
-            if hook == GuardrailEventHooks.post_call or hook == "post_call":
-                resolved_input_type = "response"
-
-        _input_type: Literal["request", "response"] = (
-            "response" if resolved_input_type == "response" else "request"
+        request_data: dict = {"messages": request.messages} if request.messages else {}
+        _input_type = _resolve_guardrail_input_type(
+            active_guardrail, request.input_type
         )
         guardrailed_inputs = await active_guardrail.apply_guardrail(
             inputs={"texts": [request.text]},
@@ -2236,13 +2347,55 @@
             input_type=_input_type,
         )
         response_text = guardrailed_inputs.get("texts", [])
-
-        return ApplyGuardrailResponse(
+        response = ApplyGuardrailResponse(
             response_text=response_text[0] if response_text else request.text
         )
     except Exception as e:
+        if litellm_logging_obj is not None and not isinstance(e, HTTPException):
+            try:
+                await litellm_logging_obj.async_failure_handler(
+                    exception=e,
+                    traceback_exception=traceback.format_exc(),
+                )
+            except Exception:
+                verbose_proxy_logger.exception(
+                    "apply_guardrail: async_failure_handler failed"
+                )
+            try:
+                thread_pool_executor.submit(
+                    litellm_logging_obj.failure_handler,
+                    e,
+                    traceback.format_exc(),
+                )
+            except Exception:
+                verbose_proxy_logger.exception(
+                    "apply_guardrail: failure_handler submit failed"
+                )
+        try:
+            transformed_exception = await proxy_logging_obj.post_call_failure_hook(
+                user_api_key_dict=user_api_key_dict,
+                original_exception=e,
+                request_data=data,
+            )
+            if transformed_exception is not None:
+                e = transformed_exception
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: post_call_failure_hook failed"
+            )
         raise handle_exception_on_proxy(e)
 
+    # Success logging outside except so a hook error never triggers failure handlers.
+    response = await _emit_guardrail_success_logs(
+        proxy_logging_obj=proxy_logging_obj,
+        litellm_logging_obj=litellm_logging_obj,
+        data=data,
+        user_api_key_dict=user_api_key_dict,
+        response=response,
+        start_time=start_time,
+    )
+    return response
 
+
 # Usage (dashboard) endpoints: overview, detail, logs
 router.include_router(guardrails_usage_router)

diff --git a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
--- a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
+++ b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
@@ -4,7 +4,8 @@
 
 import os
 import sys
-from unittest.mock import AsyncMock, Mock, patch
+from contextlib import contextmanager
+from unittest.mock import AsyncMock, MagicMock, Mock, patch
 
 import pytest
 
@@ -17,6 +18,38 @@
 from litellm.types.guardrails import ApplyGuardrailRequest, ApplyGuardrailResponse
 
 
+@contextmanager
+def _mock_proxy_logging():
+    """Patch the proxy-server globals that apply_guardrail imports at call time."""
+    mock_proxy_logging = MagicMock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock(return_value=None)
+    mock_proxy_logging.post_call_failure_hook = AsyncMock(return_value=None)
+    mock_logging_obj = MagicMock()
+    mock_logging_obj.async_success_handler = AsyncMock(return_value=None)
+    mock_logging_obj.async_failure_handler = AsyncMock(return_value=None)
+    mock_logging_obj.success_handler = MagicMock(return_value=None)
+    mock_logging_obj.failure_handler = MagicMock(return_value=None)
+    mock_logging_obj.model_call_details = {}
+
+    with patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing"
+    ) as mock_proc_cls, patch(
+        "litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging
+    ), patch(
+        "litellm.proxy.proxy_server.general_settings", {}
+    ), patch(
+        "litellm.proxy.proxy_server.proxy_config", MagicMock()
+    ), patch(
+        "litellm.proxy.proxy_server.version", "0.0.0"
+    ):
+        mock_proc = MagicMock()
+        mock_proc.common_processing_pre_call_logic = AsyncMock(
+            return_value=({}, mock_logging_obj)
+        )
+        mock_proc_cls.return_value = mock_proc
+        yield mock_proxy_logging
+
+
 @pytest.mark.asyncio
 async def test_apply_guardrail_endpoint_returns_correct_response():
     """Test that apply_guardrail endpoint returns ApplyGuardrailResponse object"""
@@ -25,7 +58,7 @@
     # Mock the guardrail registry
     with patch(
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    ) as mock_registry, _mock_proxy_logging():
         # Create a mock guardrail
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Apply guardrail returns GenericGuardrailAPIInputs (dict with texts key)
@@ -49,7 +82,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type
@@ -73,7 +108,7 @@
     # Mock the guardrail registry to return None
     with patch(
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    ) as mock_registry, _mock_proxy_logging():
         mock_registry.get_initialized_guardrail_callback.return_value = None
 
         # Create the request
@@ -86,7 +121,11 @@
 
         # Verify exception is raised
         with pytest.raises(ProxyException) as exc_info:
-            await apply_guardrail(request=request, user_api_key_dict=user_api_key_dict)
+            await apply_guardrail(
+                fastapi_request=Mock(),
+                request=request,
+                user_api_key_dict=user_api_key_dict,
+            )
 
         assert "non-existent-guardrail" in exc_info.value.message
         assert "not found" in exc_info.value.message
@@ -100,7 +139,7 @@
     # Mock the guardrail registry
     with patch(
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    ) as mock_registry, _mock_proxy_logging():
         # Create a mock guardrail that simulates Presidio behavior
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Simulate masking PII entities - returns GenericGuardrailAPIInputs (dict with texts key)
@@ -124,7 +163,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type
@@ -145,7 +186,7 @@
     # Mock the guardrail registry
     with patch(
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    ) as mock_registry, _mock_proxy_logging():
         # Create a mock guardrail
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Returns GenericGuardrailAPIInputs (dict with texts key)
@@ -166,7 +207,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type

diff --git a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
--- a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
+++ b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
@@ -4,7 +4,8 @@
 
 import os
 import sys
-from unittest.mock import AsyncMock, patch
+from contextlib import contextmanager
+from unittest.mock import AsyncMock, MagicMock, Mock, patch
 
 import pytest
 
@@ -16,6 +17,38 @@
 from litellm.types.guardrails import ApplyGuardrailRequest, ApplyGuardrailResponse
 
 
+@contextmanager
+def _mock_proxy_logging():
+    """Patch the proxy-server globals that apply_guardrail imports at call time."""
+    mock_proxy_logging = MagicMock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock(return_value=None)
+    mock_proxy_logging.post_call_failure_hook = AsyncMock(return_value=None)
+    mock_logging_obj = MagicMock()
+    mock_logging_obj.async_success_handler = AsyncMock(return_value=None)
+    mock_logging_obj.async_failure_handler = AsyncMock(return_value=None)
+    mock_logging_obj.success_handler = MagicMock(return_value=None)
+    mock_logging_obj.failure_handler = MagicMock(return_value=None)
+    mock_logging_obj.model_call_details = {}
+
+    with patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing"
+    ) as mock_proc_cls, patch(
+        "litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging
+    ), patch(
+        "litellm.proxy.proxy_server.general_settings", {}
+    ), patch(
+        "litellm.proxy.proxy_server.proxy_config", MagicMock()
+    ), patch(
+        "litellm.proxy.proxy_server.version", "0.0.0"
+    ):
+        mock_proc = MagicMock()
+        mock_proc.common_processing_pre_call_logic = AsyncMock(
+            return_value=({}, mock_logging_obj)
+        )
+        mock_proc_cls.return_value = mock_proc
+        yield mock_proxy_logging
+
+
 @pytest.mark.asyncio
 async def test_bedrock_apply_guardrail_success():
     """Test that Bedrock guardrail apply_guardrail method works correctly"""
@@ -167,7 +200,7 @@
     # Mock the guardrail registry
     with patch(
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    ) as mock_registry, _mock_proxy_logging():
         # Mock the make_bedrock_api_request method
         with patch.object(
             guardrail, "make_bedrock_api_request", new_callable=AsyncMock
@@ -194,7 +227,9 @@
 
             # Call the endpoint
             response = await apply_guardrail(
-                request=request, user_api_key_dict=user_api_key_dict
+                fastapi_request=Mock(),
+                request=request,
+                user_api_key_dict=user_api_key_dict,
             )
 
             # Verify the response

diff --git a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
--- a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
+++ b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
@@ -1159,7 +1159,11 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error details
     assert str(exc_info.value.code) == "404"
@@ -1186,6 +1190,25 @@
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
     )
 
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_failure_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_failure_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mocker.patch("litellm.litellm_core_utils.thread_pool_executor.executor")
+
     # Create request
     request = ApplyGuardrailRequest(
         guardrail_name="test-guardrail", text="Test input text with forbidden content"
@@ -1196,13 +1219,71 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error is properly handled
     assert "Bedrock guardrail failed" in str(exc_info.value.message)
 
 
 @pytest.mark.asyncio
+async def test_apply_guardrail_invokes_logging_pipeline(mocker):
+    mock_guardrail = mocker.Mock()
+    mock_guardrail.apply_guardrail = AsyncMock(return_value={"texts": ["masked"]})
+
+    mock_registry = mocker.Mock()
+    mock_registry.get_initialized_guardrail_callback.return_value = mock_guardrail
+    mocker.patch(
+        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
+    )
+
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_success_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mock_executor = mocker.Mock()
+    mocker.patch(
+        "litellm.litellm_core_utils.thread_pool_executor.executor", mock_executor
+    )
+
+    request = ApplyGuardrailRequest(
+        guardrail_name="test-guardrail", text="hello@example.com"
+    )
+    response = await apply_guardrail(
+        fastapi_request=mocker.Mock(),
+        request=request,
+        user_api_key_dict=UserAPIKeyAuth(),
+    )
+
+    assert response.response_text == "masked"
+    mock_processor.common_processing_pre_call_logic.assert_awaited_once()
+    mock_proxy_logging.post_call_success_hook.assert_awaited_once()
+    mock_logging_obj.async_success_handler.assert_awaited_once()
+    assert mock_logging_obj.call_type == "pass_through_endpoint"
+    mock_executor.submit.assert_called_once()
+    assert mock_logging_obj.async_success_handler.await_args.kwargs["result"] == {
+        "response": {"response_text": "masked"}
+    }
+
+
+@pytest.mark.asyncio
 async def test_get_guardrail_info_endpoint_config_guardrail(mocker):
     """
     Test get_guardrail_info endpoint returns proper response when guardrail is found in config.

You can send follow-ups to the cloud agent here.

Comment thread litellm/proxy/guardrails/guardrail_endpoints.py Outdated
Separate each logging handler call into its own try/except so a failure
in the async handler does not silently skip the sync handler submission
(and vice versa). Matches the docstring's defensive intent.

Co-authored-by: Yassin Kortam <yassin@berri.ai>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai

…y_guardrail logging fixture

- Add proxy-server global mocks to test_apply_guardrail_not_found so the
  failure-path post_call_failure_hook call doesn't touch the real proxy
  logging singleton.
- Extract the duplicated _mock_proxy_logging context manager out of the
  two enterprise apply_guardrail test files into a shared conftest fixture
  so the helper stays in one place.
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Logging object messages attribute not updated, causing inconsistency
    • Replaced the direct write to model_call_details['messages'] with a call to the existing update_messages() API, which updates both self.messages and self.model_call_details['messages'] in sync.
Preview (156e0d2c1a)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -839,6 +839,7 @@
             "aget_run",
             "acancel_run",
             "adelete_run",
+            "apply_guardrail",
         ],
         version: Optional[str] = None,
         user_model: Optional[str] = None,

diff --git a/litellm/proxy/guardrails/guardrail_endpoints.py b/litellm/proxy/guardrails/guardrail_endpoints.py
--- a/litellm/proxy/guardrails/guardrail_endpoints.py
+++ b/litellm/proxy/guardrails/guardrail_endpoints.py
@@ -10,7 +10,7 @@
 from typing import Any, Dict, List, Literal, Optional, Type, TypeVar, Union, cast
 from urllib.parse import urlparse
 
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, HTTPException, Request
 from pydantic import BaseModel
 
 from litellm.proxy.common_utils.path_utils import safe_join
@@ -2187,9 +2187,97 @@
         )
 
 
+def _resolve_guardrail_input_type(
+    active_guardrail: CustomGuardrail, input_type: str
+) -> Literal["request", "response"]:
+    """Return the effective input_type, auto-upgrading to 'response' for post_call guardrails."""
+    if input_type == "request":
+        hook = getattr(active_guardrail, "event_hook", None)
+        if hook == GuardrailEventHooks.post_call or hook == "post_call":
+            return "response"
+    return "response" if input_type == "response" else "request"
+
+
+def _patch_logging_obj_for_guardrail(
+    litellm_logging_obj: Any, request: ApplyGuardrailRequest
+) -> None:
+    """Configure the logging object so Langfuse/OTEL extract input and output correctly."""
+    litellm_logging_obj.call_type = "pass_through_endpoint"
+    litellm_logging_obj.model_call_details["call_type"] = "pass_through_endpoint"
+    litellm_logging_obj.update_messages(
+        request.messages
+        if request.messages
+        else [{"role": "user", "content": request.text}]
+    )
+
+
+async def _emit_guardrail_success_logs(
+    proxy_logging_obj: Any,
+    litellm_logging_obj: Any,
+    data: dict,
+    user_api_key_dict: UserAPIKeyAuth,
+    response: ApplyGuardrailResponse,
+    start_time: datetime,
+) -> ApplyGuardrailResponse:
+    """Fire proxy and LiteLLM success hooks after a successful guardrail run.
+
+    Each hook is wrapped defensively so a callback failure never prevents the
+    caller from receiving the guardrail response.  Returns the (possibly
+    hook-modified) response.
+    """
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+
+    try:
+        modified = await proxy_logging_obj.post_call_success_hook(
+            data=data,
+            user_api_key_dict=user_api_key_dict,
+            response=response,
+        )
+        if isinstance(modified, ApplyGuardrailResponse):
+            response = modified
+    except Exception:
+        verbose_proxy_logger.exception("apply_guardrail: post_call_success_hook failed")
+
+    # Build the logging payload after post_call_success_hook so that logged
+    # data matches what the caller actually receives if the hook modified
+    # the response.
+    response_for_logging = {"response": response.model_dump(exclude_none=True)}
+
+    if litellm_logging_obj is not None:
+        end_time = datetime.now(timezone.utc)
+        try:
+            await litellm_logging_obj.async_success_handler(
+                result=response_for_logging,
+                start_time=start_time,
+                end_time=end_time,
+                cache_hit=False,
+            )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: async_success_handler failed"
+            )
+        try:
+            thread_pool_executor.submit(
+                litellm_logging_obj.success_handler,
+                response_for_logging,
+                start_time,
+                end_time,
+                False,
+            )
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: success_handler submit failed"
+            )
+
+    return response
+
+
 @router.post("/guardrails/apply_guardrail", response_model=ApplyGuardrailResponse)
 @router.post("/apply_guardrail", response_model=ApplyGuardrailResponse)
 async def apply_guardrail(
+    fastapi_request: Request,
     request: ApplyGuardrailRequest,
     user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
 ):
@@ -2198,8 +2286,29 @@
 
     This endpoint allows testing guardrails by applying them to custom text inputs.
     """
+    import traceback
+
+    from litellm.proxy.common_request_processing import ProxyBaseLLMRequestProcessing
+    from litellm.litellm_core_utils.thread_pool_executor import (
+        executor as thread_pool_executor,
+    )
+    from litellm.proxy.proxy_server import (
+        general_settings,
+        proxy_config,
+        proxy_logging_obj,
+        version,
+    )
     from litellm.proxy.utils import handle_exception_on_proxy
 
+    data: dict = {
+        "guardrail_name": request.guardrail_name,
+        "input": [request.text],
+        "messages": request.messages or [],
+        "metadata": {"route": "/apply_guardrail"},
+    }
+    litellm_logging_obj = None
+    start_time = datetime.now(timezone.utc)
+
     try:
         active_guardrail: Optional[CustomGuardrail] = (
             GUARDRAIL_REGISTRY.get_initialized_guardrail_callback(
@@ -2212,23 +2321,25 @@
                 detail=f"Guardrail '{request.guardrail_name}' not found. Please ensure the guardrail is configured in your LiteLLM proxy.",
             )
 
-        request_data: dict = {}
-        if request.messages:
-            request_data["messages"] = request.messages
+        request_processor = ProxyBaseLLMRequestProcessing(data=data)
+        data, litellm_logging_obj = (
+            await request_processor.common_processing_pre_call_logic(
+                request=fastapi_request,
+                general_settings=general_settings,
+                user_api_key_dict=user_api_key_dict,
+                version=version,
+                proxy_logging_obj=proxy_logging_obj,
+                proxy_config=proxy_config,
+                route_type="apply_guardrail",
+            )
+        )
 
-        # Auto-detect input_type: if the caller didn't specify "response" but the
-        # guardrail only runs post_call (e.g. LLM-as-a-judge), use "response" so
-        # the test actually exercises the guardrail logic.
-        from litellm.types.guardrails import GuardrailEventHooks
+        if litellm_logging_obj is not None:
+            _patch_logging_obj_for_guardrail(litellm_logging_obj, request)
 
-        resolved_input_type = request.input_type
-        if resolved_input_type == "request":
-            hook = getattr(active_guardrail, "event_hook", None)
-            if hook == GuardrailEventHooks.post_call or hook == "post_call":
-                resolved_input_type = "response"
-
-        _input_type: Literal["request", "response"] = (
-            "response" if resolved_input_type == "response" else "request"
+        request_data: dict = {"messages": request.messages} if request.messages else {}
+        _input_type = _resolve_guardrail_input_type(
+            active_guardrail, request.input_type
         )
         guardrailed_inputs = await active_guardrail.apply_guardrail(
             inputs={"texts": [request.text]},
@@ -2236,13 +2347,55 @@
             input_type=_input_type,
         )
         response_text = guardrailed_inputs.get("texts", [])
-
-        return ApplyGuardrailResponse(
+        response = ApplyGuardrailResponse(
             response_text=response_text[0] if response_text else request.text
         )
     except Exception as e:
+        if litellm_logging_obj is not None and not isinstance(e, HTTPException):
+            try:
+                await litellm_logging_obj.async_failure_handler(
+                    exception=e,
+                    traceback_exception=traceback.format_exc(),
+                )
+            except Exception:
+                verbose_proxy_logger.exception(
+                    "apply_guardrail: async_failure_handler failed"
+                )
+            try:
+                thread_pool_executor.submit(
+                    litellm_logging_obj.failure_handler,
+                    e,
+                    traceback.format_exc(),
+                )
+            except Exception:
+                verbose_proxy_logger.exception(
+                    "apply_guardrail: failure_handler submit failed"
+                )
+        try:
+            transformed_exception = await proxy_logging_obj.post_call_failure_hook(
+                user_api_key_dict=user_api_key_dict,
+                original_exception=e,
+                request_data=data,
+            )
+            if isinstance(transformed_exception, Exception):
+                e = transformed_exception
+        except Exception:
+            verbose_proxy_logger.exception(
+                "apply_guardrail: post_call_failure_hook failed"
+            )
         raise handle_exception_on_proxy(e)
 
+    # Success logging outside except so a hook error never triggers failure handlers.
+    response = await _emit_guardrail_success_logs(
+        proxy_logging_obj=proxy_logging_obj,
+        litellm_logging_obj=litellm_logging_obj,
+        data=data,
+        user_api_key_dict=user_api_key_dict,
+        response=response,
+        start_time=start_time,
+    )
+    return response
 
+
 # Usage (dashboard) endpoints: overview, detail, logs
 router.include_router(guardrails_usage_router)

diff --git a/tests/enterprise/litellm_enterprise/proxy/guardrails/conftest.py b/tests/enterprise/litellm_enterprise/proxy/guardrails/conftest.py
new file mode 100644
--- /dev/null
+++ b/tests/enterprise/litellm_enterprise/proxy/guardrails/conftest.py
@@ -1,0 +1,42 @@
+"""Shared fixtures for guardrail apply_guardrail tests."""
+
+from contextlib import contextmanager
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+
+@contextmanager
+def _mock_proxy_logging():
+    """Patch the proxy-server globals that apply_guardrail imports at call time."""
+    mock_proxy_logging = MagicMock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock(return_value=None)
+    mock_proxy_logging.post_call_failure_hook = AsyncMock(return_value=None)
+    mock_logging_obj = MagicMock()
+    mock_logging_obj.async_success_handler = AsyncMock(return_value=None)
+    mock_logging_obj.async_failure_handler = AsyncMock(return_value=None)
+    mock_logging_obj.success_handler = MagicMock(return_value=None)
+    mock_logging_obj.failure_handler = MagicMock(return_value=None)
+    mock_logging_obj.model_call_details = {}
+
+    with (
+        patch(
+            "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing"
+        ) as mock_proc_cls,
+        patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging),
+        patch("litellm.proxy.proxy_server.general_settings", {}),
+        patch("litellm.proxy.proxy_server.proxy_config", MagicMock()),
+        patch("litellm.proxy.proxy_server.version", "0.0.0"),
+    ):
+        mock_proc = MagicMock()
+        mock_proc.common_processing_pre_call_logic = AsyncMock(
+            return_value=({}, mock_logging_obj)
+        )
+        mock_proc_cls.return_value = mock_proc
+        yield mock_proxy_logging
+
+
+@pytest.fixture
+def mock_proxy_logging_ctx():
+    """Return the proxy-logging context manager factory for use as `with ctx():`."""
+    return _mock_proxy_logging

diff --git a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
--- a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
+++ b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_apply_guardrail_endpoint.py
@@ -18,14 +18,19 @@
 
 
 @pytest.mark.asyncio
-async def test_apply_guardrail_endpoint_returns_correct_response():
+async def test_apply_guardrail_endpoint_returns_correct_response(
+    mock_proxy_logging_ctx,
+):
     """Test that apply_guardrail endpoint returns ApplyGuardrailResponse object"""
     from litellm.proxy.guardrails.guardrail_endpoints import apply_guardrail
 
     # Mock the guardrail registry
-    with patch(
-        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    with (
+        patch(
+            "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
+        ) as mock_registry,
+        mock_proxy_logging_ctx(),
+    ):
         # Create a mock guardrail
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Apply guardrail returns GenericGuardrailAPIInputs (dict with texts key)
@@ -49,7 +54,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type
@@ -65,15 +72,18 @@
 
 
 @pytest.mark.asyncio
-async def test_apply_guardrail_endpoint_guardrail_not_found():
+async def test_apply_guardrail_endpoint_guardrail_not_found(mock_proxy_logging_ctx):
     """Test that apply_guardrail endpoint raises exception when guardrail not found"""
     from litellm.proxy._types import ProxyException
     from litellm.proxy.guardrails.guardrail_endpoints import apply_guardrail
 
     # Mock the guardrail registry to return None
-    with patch(
-        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    with (
+        patch(
+            "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
+        ) as mock_registry,
+        mock_proxy_logging_ctx(),
+    ):
         mock_registry.get_initialized_guardrail_callback.return_value = None
 
         # Create the request
@@ -86,26 +96,35 @@
 
         # Verify exception is raised
         with pytest.raises(ProxyException) as exc_info:
-            await apply_guardrail(request=request, user_api_key_dict=user_api_key_dict)
+            await apply_guardrail(
+                fastapi_request=Mock(),
+                request=request,
+                user_api_key_dict=user_api_key_dict,
+            )
 
         assert "non-existent-guardrail" in exc_info.value.message
         assert "not found" in exc_info.value.message
 
 
 @pytest.mark.asyncio
-async def test_apply_guardrail_endpoint_with_presidio_guardrail():
+async def test_apply_guardrail_endpoint_with_presidio_guardrail(mock_proxy_logging_ctx):
     """Test apply_guardrail endpoint with a Presidio-like guardrail"""
     from litellm.proxy.guardrails.guardrail_endpoints import apply_guardrail
 
     # Mock the guardrail registry
-    with patch(
-        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    with (
+        patch(
+            "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
+        ) as mock_registry,
+        mock_proxy_logging_ctx(),
+    ):
         # Create a mock guardrail that simulates Presidio behavior
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Simulate masking PII entities - returns GenericGuardrailAPIInputs (dict with texts key)
         mock_guardrail.apply_guardrail = AsyncMock(
-            return_value={"texts": ["My name is [PERSON] and my email is [EMAIL_ADDRESS]"]}
+            return_value={
+                "texts": ["My name is [PERSON] and my email is [EMAIL_ADDRESS]"]
+            }
         )
 
         # Configure the registry to return our mock guardrail
@@ -124,7 +143,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type
@@ -138,14 +159,17 @@
 
 
 @pytest.mark.asyncio
-async def test_apply_guardrail_endpoint_without_optional_params():
+async def test_apply_guardrail_endpoint_without_optional_params(mock_proxy_logging_ctx):
     """Test apply_guardrail endpoint without optional language and entities parameters"""
     from litellm.proxy.guardrails.guardrail_endpoints import apply_guardrail
 
     # Mock the guardrail registry
-    with patch(
-        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    with (
+        patch(
+            "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
+        ) as mock_registry,
+        mock_proxy_logging_ctx(),
+    ):
         # Create a mock guardrail
         mock_guardrail = Mock(spec=CustomGuardrail)
         # Returns GenericGuardrailAPIInputs (dict with texts key)
@@ -166,7 +190,9 @@
 
         # Call the endpoint
         response = await apply_guardrail(
-            request=request, user_api_key_dict=user_api_key_dict
+            fastapi_request=Mock(),
+            request=request,
+            user_api_key_dict=user_api_key_dict,
         )
 
         # Verify the response is of the correct type

diff --git a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
--- a/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
+++ b/tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py
@@ -4,7 +4,7 @@
 
 import os
 import sys
-from unittest.mock import AsyncMock, patch
+from unittest.mock import AsyncMock, Mock, patch
 
 import pytest
 
@@ -153,7 +153,7 @@
 
 
 @pytest.mark.asyncio
-async def test_bedrock_apply_guardrail_endpoint_integration():
+async def test_bedrock_apply_guardrail_endpoint_integration(mock_proxy_logging_ctx):
     """Test the full endpoint integration with Bedrock guardrail"""
     from litellm.proxy.guardrails.guardrail_endpoints import apply_guardrail
 
@@ -165,9 +165,12 @@
     )
 
     # Mock the guardrail registry
-    with patch(
-        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
-    ) as mock_registry:
+    with (
+        patch(
+            "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY"
+        ) as mock_registry,
+        mock_proxy_logging_ctx(),
+    ):
         # Mock the make_bedrock_api_request method
         with patch.object(
             guardrail, "make_bedrock_api_request", new_callable=AsyncMock
@@ -194,7 +197,9 @@
 
             # Call the endpoint
             response = await apply_guardrail(
-                request=request, user_api_key_dict=user_api_key_dict
+                fastapi_request=Mock(),
+                request=request,
+                user_api_key_dict=user_api_key_dict,
             )
 
             # Verify the response

diff --git a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
--- a/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
+++ b/tests/test_litellm/proxy/guardrails/test_guardrail_endpoints.py
@@ -1149,6 +1149,13 @@
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
     )
 
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_failure_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+
     # Create request
     request = ApplyGuardrailRequest(
         guardrail_name="non-existent-guardrail", text="Test input text"
@@ -1159,7 +1166,11 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error details
     assert str(exc_info.value.code) == "404"
@@ -1186,6 +1197,25 @@
         "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
     )
 
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_failure_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_failure_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mocker.patch("litellm.litellm_core_utils.thread_pool_executor.executor")
+
     # Create request
     request = ApplyGuardrailRequest(
         guardrail_name="test-guardrail", text="Test input text with forbidden content"
@@ -1196,13 +1226,71 @@
 
     # Call endpoint and expect ProxyException
     with pytest.raises(ProxyException) as exc_info:
-        await apply_guardrail(request=request, user_api_key_dict=mock_user_auth)
+        await apply_guardrail(
+            fastapi_request=mocker.Mock(),
+            request=request,
+            user_api_key_dict=mock_user_auth,
+        )
 
     # Verify error is properly handled
     assert "Bedrock guardrail failed" in str(exc_info.value.message)
 
 
 @pytest.mark.asyncio
+async def test_apply_guardrail_invokes_logging_pipeline(mocker):
+    mock_guardrail = mocker.Mock()
+    mock_guardrail.apply_guardrail = AsyncMock(return_value={"texts": ["masked"]})
+
+    mock_registry = mocker.Mock()
+    mock_registry.get_initialized_guardrail_callback.return_value = mock_guardrail
+    mocker.patch(
+        "litellm.proxy.guardrails.guardrail_endpoints.GUARDRAIL_REGISTRY", mock_registry
+    )
+
+    mock_logging_obj = mocker.Mock()
+    mock_logging_obj.async_success_handler = AsyncMock()
+    mock_logging_obj.model_call_details = {}
+    mock_processor = mocker.Mock()
+    mock_processor.common_processing_pre_call_logic = AsyncMock(
+        return_value=({"guardrail_name": "test-guardrail"}, mock_logging_obj)
+    )
+    mocker.patch(
+        "litellm.proxy.common_request_processing.ProxyBaseLLMRequestProcessing",
+        return_value=mock_processor,
+    )
+
+    mock_proxy_logging = mocker.Mock()
+    mock_proxy_logging.post_call_success_hook = AsyncMock()
+    mocker.patch("litellm.proxy.proxy_server.proxy_logging_obj", mock_proxy_logging)
+    mocker.patch("litellm.proxy.proxy_server.general_settings", {})
+    mocker.patch("litellm.proxy.proxy_server.proxy_config", mocker.Mock())
+    mocker.patch("litellm.proxy.proxy_server.version", "test")
+    mock_executor = mocker.Mock()
+    mocker.patch(
+        "litellm.litellm_core_utils.thread_pool_executor.executor", mock_executor
+    )
+
+    request = ApplyGuardrailRequest(
+        guardrail_name="test-guardrail", text="hello@example.com"
+    )
+    response = await apply_guardrail(
+        fastapi_request=mocker.Mock(),
+        request=request,
+        user_api_key_dict=UserAPIKeyAuth(),
+    )
+
+    assert response.response_text == "masked"
+    mock_processor.common_processing_pre_call_logic.assert_awaited_once()
+    mock_proxy_logging.post_call_success_hook.assert_awaited_once()
+    mock_logging_obj.async_success_handler.assert_awaited_once()
+    assert mock_logging_obj.call_type == "pass_through_endpoint"
+    mock_executor.submit.assert_called_once()
+    assert mock_logging_obj.async_success_handler.await_args.kwargs["result"] == {
+        "response": {"response_text": "masked"}
+    }
+
+
+@pytest.mark.asyncio
 async def test_get_guardrail_info_endpoint_config_guardrail(mocker):
     """
     Test get_guardrail_info endpoint returns proper response when guardrail is found in config.

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 8fd6f71. Configure here.

Comment thread litellm/proxy/guardrails/guardrail_endpoints.py
Co-authored-by: Yassin Kortam <yassin@berri.ai>
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

On the registry-lookup ordering observation: the current placement is intentional.

  1. The cheap GUARDRAIL_REGISTRY.get_initialized_guardrail_callback check is deliberately ahead of common_processing_pre_call_logic so a misconfigured guardrail name does not trigger pre-call hooks (which run any other configured guardrails on the input text and consume rate-limit budget). For a 404 misconfiguration we want a fast, side-effect-free response.

  2. The not isinstance(e, HTTPException) guard inside the except block (present since the first commit on this PR) already determines trace behavior independently of ordering: HTTPException-class results are the user-facing 4xx response and are intentionally not emitted as LiteLLM-level failure spans. Reordering the lookup would not change this — litellm_logging_obj.async_failure_handler is still skipped for HTTPException either way.

  3. proxy_logging_obj.post_call_failure_hook continues to fire for the 404 path, so proxy-level observability (alerting, etc.) still sees the failure. That's consistent with how other proxy endpoints distinguish 4xx client errors from upstream LLM failures.

No code change needed.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

@mateo-berri
mateo-berri merged commit 5bd59b3 into litellm_internal_staging May 28, 2026
116 of 119 checks passed
@mateo-berri
mateo-berri deleted the litellm_apply_guardrail_logging_callbacks branch May 28, 2026 16:41
shudonglin added a commit to rayward-external/litellm that referenced this pull request Jun 1, 2026
* feat: add support for claude code goal mode for bedrock opus output config (BerriAI#28898)

* feat: support goal mode for claude on bedrock

* fix failing lint test

* addressing greptile comments

* fixing failed test

* address greptile: copy output_config and warn on dropped converse format

* fix(bedrock): skip redundant output_config normalization on Converse reasoning_effort path

When reasoning_effort is mapped via _handle_reasoning_effort_parameter, the
resulting output_config is already normalized via
normalize_bedrock_opus_output_config_effort. Mark it as normalized so
_prepare_request_params can skip the redundant call (and the associated
get_model_info lookup) on every request.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(reasoning-effort-grid): reflect Bedrock opus-4-6 xhigh→max clamping

* fix(bedrock): stop leaking output_config marker and message-content mutation

* fix(bedrock): guard effort key access in normalize_bedrock_opus_output_config_effort

Defensively check that 'effort' is a valid key in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER
before indexing, to prevent a KeyError if the hardcoded guard tuple ever drifts from
the order dict's keys.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(bedrock): drop dead second clause in effort normalization guard

The 'effort not in _BEDROCK_OUTPUT_CONFIG_EFFORT_ORDER' check is
unreachable once 'effort not in ("xhigh", "max")' has been ruled out,
since both literals are present in the order dict. Keep the literal
membership check and let the dict lookups below speak for themselves.

* fix(bedrock): clamp output_config.effort against ceiling for any known value

The early return when effort was not 'xhigh'/'max' meant a ceiling of
'low' or 'medium' would silently forward an out-of-range value. Gate on
the known effort ordering instead so the ceiling comparison runs for
every recognized effort.

* test(grid_spec): use _CAPS_OPUS_4_7 for non-Bedrock opus-4-6 entries

claude-opus-4-6 now declares supports_xhigh_reasoning_effort in the model
map, so production accepts xhigh on Azure AI and Vertex AI routes. Update
those grid_spec entries to match production capabilities so expected()
predicts 200 for xhigh instead of 400.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* test(grid_spec): revert xhigh caps for non-Bedrock opus-4-6

azure_ai/claude-opus-4-6 and vertex_ai/claude-opus-4-6 do not declare
supports_xhigh_reasoning_effort in model_prices_and_context_window.json.
Azure AI upstream rejects xhigh with HTTP 400 ("Supported levels: high,
low, max, medium"). Restore _CAPS_4_6 so the grid predicts 400 for
xhigh, matching production capabilities.

* fix: stop advertising xhigh effort on Opus 4.5/4.6

Only Opus 4.7 supports the xhigh reasoning effort level. Remove the
supports_xhigh_reasoning_effort flag from every Opus 4.5 and Opus 4.6
entry (direct Anthropic, Bedrock, and regional variants) in both model
catalog files.

On the direct Anthropic path there is no effort clamp, so flagging 4.5/4.6
as xhigh-capable caused litellm to forward xhigh to a model that rejects it
(and made get_model_info misreport the capability). xhigh now correctly
degrades to high / raises on those models.

Bedrock graceful degradation for Claude Code goal mode is unaffected: it
relies solely on the bedrock_output_config_effort_ceiling clamp (4.5->high,
4.6->max, 4.7->xhigh), which runs before validation, so xhigh requests to
older Bedrock Opus models are still silently lowered rather than rejected.

Update effort-gating tests to reflect that 4.5/4.6 no longer accept xhigh.

* fix: clamp xhigh effort on Bedrock Invoke /v1/messages instead of rejecting

Claude Code "goal mode" sends output_config.effort=xhigh over the Anthropic
/v1/messages API, which routes Bedrock models through
AmazonAnthropicClaudeMessagesConfig. That path validated effort against the
model's native capability and raised 400 for xhigh on Opus 4.6, while the
chat-completions paths (Converse + Invoke) already clamp xhigh to the model's
bedrock_output_config_effort_ceiling. That asymmetry broke goal mode on the
exact API surface Claude Code uses.

Apply the same ceiling clamp on the messages path before the shared effort
gate runs, so xhigh degrades to max on Opus 4.6 (and stays xhigh on 4.7).
Scoped to adaptive-thinking models and to models that declare a ceiling, so
Sonnet 4.6 (no ceiling) and Opus 4.5 (budget mode) are unaffected and still
reject xhigh.

* fix(bedrock): preserve user output_config when applying reasoning_effort

- Converse path: merge mapped effort into existing output_config via
  setdefault instead of overwriting it, matching the Anthropic Messages
  path. Prevents user-supplied output_config.format from being silently
  dropped when reasoning_effort is also provided.
- tests: clear _get_local_model_cost_map lru_cache in the autouse
  fixture alongside get_bedrock_response_stream_shape to avoid stale
  cache leakage between tests.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(bedrock): pre-clamp reasoning_effort for chat invoke; correct test caps

- Add _clamp_adaptive_reasoning_effort_for_bedrock to AmazonAnthropicClaudeConfig
  so raw reasoning_effort=xhigh degrades to the model's bedrock effort ceiling
  before AnthropicConfig.map_openai_params converts it to output_config.
  Mirrors converse path (_handle_reasoning_effort_parameter) and messages path
  (_clamp_adaptive_reasoning_effort_for_bedrock) so the three Bedrock paths
  are consistent.

- grid_spec: restore caps=_CAPS_4_6 for Bedrock converse/invoke Opus 4.6 entries
  so the test reflects the model's actual JSON capabilities. Teach expected()
  to bypass the xhigh/max cap check when bedrock_effort_ceiling will clamp
  the wire effort, so the test still passes for Bedrock's graceful degradation
  contract without lying about native model caps.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Dennis Henry <dennis.henry@okta.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>

* feat(guardrails): wire apply_guardrail into proxy logging callbacks (BerriAI#28970)

* feat(guardrails): wire apply_guardrail into proxy logging callbacks

Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(guardrails): fix Greptile review comments on apply_guardrail logging

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(apply_guardrail): preserve original exception and capture modified response

- Capture return value from post_call_success_hook so callback-modified
  responses propagate to the caller.
- Wrap success/failure logging calls in defensive try/except so logging
  infrastructure failures don't replace the user-visible response or mask
  the original guardrail exception.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* Fix mypy

* fix(apply_guardrail): isolate failure logging and use post-hook response for logging

- Split async_failure_handler and post_call_failure_hook into independent
  try/except blocks so a callback bug in one does not silently skip the
  other.
- Build response_for_logging inside _emit_guardrail_success_logs after
  post_call_success_hook runs, so logged data matches the response the
  caller actually receives when the hook modifies the response.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(apply_guardrail): fix black formatting and update tests for fastapi_request param

- Run black on guardrail_endpoints.py to fix CI formatting check
- Add _mock_proxy_logging() helper to enterprise guardrail tests to patch
  proxy-server globals imported at call time
- Pass fastapi_request=Mock() in all direct apply_guardrail test calls
  to match updated function signature

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(guardrails): use transformed exception from post_call_failure_hook in apply_guardrail

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(guardrails): isolate sync/async logging handlers in apply_guardrail

Separate each logging handler call into its own try/except so a failure
in the async handler does not silently skip the sync handler submission
(and vice versa). Matches the docstring's defensive intent.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(apply_guardrail): guard transformed_exception with isinstance check

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(guardrails): mock proxy globals in not_found test and share apply_guardrail logging fixture

- Add proxy-server global mocks to test_apply_guardrail_not_found so the
  failure-path post_call_failure_hook call doesn't touch the real proxy
  logging singleton.
- Extract the duplicated _mock_proxy_logging context manager out of the
  two enterprise apply_guardrail test files into a shared conftest fixture
  so the helper stays in one place.

* fix(guardrails): use update_messages to keep logging obj in sync

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>

* chore(ci): merge dev brach (BerriAI#29192)

* build(deps): bump next from 16.2.4 to 16.2.6 in /ui/litellm-dashboard (BerriAI#27665)

Bumps [next](https://github.com/vercel/next.js) from 16.2.4 to 16.2.6.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](vercel/next.js@v16.2.4...v16.2.6)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 16.2.6
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump protobufjs in /tests/pass_through_tests (BerriAI#28296)

Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 7.5.6 to 7.6.0.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/protobufjs-v7.6.0/CHANGELOG.md)
- [Commits](protobufjs/protobuf.js@protobufjs-v7.5.6...protobufjs-v7.6.0)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-version: 7.6.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump ws from 8.20.0 to 8.20.1 in /tests/pass_through_tests (BerriAI#28303)

Bumps [ws](https://github.com/websockets/ws) from 8.20.0 to 8.20.1.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](websockets/ws@8.20.0...8.20.1)

---
updated-dependencies:
- dependency-name: ws
  dependency-version: 8.20.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: improve bedrock streaming hot path perf (BerriAI#28720)

* fix(proxy): enforce tag budgets for key-level tags (BerriAI#29108)

* fix(proxy): enforce tag budgets for key-level tags

Merge API key metadata.tags into request_data before _tag_max_budget_check
so per-tag budgets apply when tags are set on the key at creation time.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(auth): avoid false reject for key-inherited tags

Run reject_clientside_metadata_tags before key-tag injection, then inject key metadata tags immediately before tag budget checks so key tags still enforce budgets without being treated as client-supplied tags.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit (BerriAI#29098)

* fix(vertex-ai): pass litellm_params to validate_environment in video handlers and implement video edit for Veo

- Pass litellm_params to validate_environment in 11 video handler call sites
  (remix, create_character, get_character, edit, extension, delete) so
  DB-stored Vertex AI credentials are used instead of falling back to ADC
- Implement transform_video_edit_request/response for VertexAI: fetches
  source video via fetchPredictOperation then submits a new
  predictLongRunning request with the video bytes/gcsUri + edit prompt

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(vertex-ai): hoist fetchPredictOperation into handlers to avoid blocking event loop

- Add get_video_edit_prefetch_params() to BaseVideoConfig (returns None)
- VertexAI overrides it to return the fetchPredictOperation URL/body
- Both sync and async video_edit handlers call this and use their shared
  httpx client for the fetch, passing the result as prefetched_source_data
- transform_video_edit_request is now a pure transform with no HTTP calls
- Fix extra_body.pop() mutation by working on a shallow copy

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(vertex-ai): include prefetch call inside _handle_error try/except block

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(videos): add prefetched_source_data param to all transform_video_edit_request overrides

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(video_edit): keep transform/pre_call outside try so validation errors propagate

Move transform_video_edit_request and logging_obj.pre_call outside the
try/except that wraps HTTP calls in (async_)video_edit_handler so that
ValueError validation errors (e.g. 'source video not complete yet') are
not silently wrapped as 500s by _handle_error. The prefetch HTTP call
keeps its own try/except so its errors are still mapped through the
provider's error handler. Matches the pattern used by
video_extension_handler and video_remix_handler.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* refactor(vertex_ai): delegate get_video_edit_prefetch_params to status retrieve

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* Fix varia review

* fix(video_edit): route transform errors through _handle_error

Wrap transform_video_edit_request and pre_call in the same try/except
as the HTTP call in sync and async handlers so validation failures
(e.g. source video not complete) return typed LiteLLM exceptions.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist (BerriAI#28487)

* fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist

* fix(datadog): guard non-dict callback_specific_params + log empty aggregation

* fix(datadog): block user-controlled tags from overwriting reserved cost-attribution dimensions

* fix(datadog): cast metadata to dict[str, Any] to satisfy mypy

* feat(helm): split per-component ServiceAccounts for gateway, backend, and UI (BerriAI#28712)

* feat(helm): split per-component ServiceAccounts for gateway, backend, and UI

Replace the single shared serviceAccount with three separate serviceAccounts
(gateway, backend, ui) so operators can attach different IRSA / Workload
Identity annotations per component without granting data-plane credentials
to the UI pod.

Key changes:
- values.yaml: rename serviceAccount → serviceAccounts with gateway/backend/ui
  sub-keys; UI defaults to automount: false
- _helpers.tpl: replace litellm.serviceAccountName with three component-scoped
  helpers (litellm.gateway/backend/ui.serviceAccountName)
- serviceaccount.yaml: create up to three separate ServiceAccount objects with
  component labels and per-SA automountServiceAccountToken
- gateway/backend deployments: use their respective SA helpers
- ui deployment: use litellm.ui.serviceAccountName + explicit
  automountServiceAccountToken: false on the pod spec so the projected token
  is absent even when the SA itself allows it
- migrations-job: share the backend SA (both need DB write access)

Resolves LIT-3171

https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF

* fix(helm): enforce automountServiceAccountToken on all pod specs; fix leading --- in serviceaccount.yaml

- gateway/backend deployments: add explicit automountServiceAccountToken on
  the pod spec so serviceAccounts.*.automount is honoured regardless of
  whether the SA is chart-created or operator-supplied (previously the flag
  only took effect on the SA object when create: true, creating an asymmetry
  with the UI which already enforced it at pod-spec level)
- serviceaccount.yaml: use a $prev sentinel to emit --- only between
  documents, preventing a leading --- when gateway SA is skipped but
  backend or ui SA is created (avoids lint/GitOps warnings from strict
  YAML parsers and tools like ArgoCD)

https://claude.ai/code/session_01QPy362WnjmEpeNuJaPUqmF

---------

Co-authored-by: Claude <noreply@anthropic.com>

* bump deps (BerriAI#29208) (BerriAI#29226)

* fix(deps): bump vulnerable proxy dependencies (starlette/fastapi, granian, pyarrow, semantic-router)

Resolve known CVEs flagged by osv-scanner/grype against uv.lock. All bumped
versions verified to resolve, install, and pass the proxy auth/route/middleware
unit suites (717 tests) plus an import smoke on the new stack.

- starlette 0.50.0 -> 1.1.0 (CVE-2026-48710 "BadHost", GHSA-86qp-5c8j-p5mr):
  versions <1.0.1 reconstruct request.url from the unvalidated Host header,
  poisoning request.url.path. Required raising fastapi 0.124.4 -> 0.136.3,
  which dropped fastapi's starlette<0.51.0 cap; an explicit starlette>=1.0.1
  floor blocks regression to a vulnerable transitive resolution. The proxy's
  own auth already reads scope["path"] via get_request_route, but the locked
  starlette still flagged in container scanners and left other request.url
  consumers exposed.
- granian 2.5.7 -> 2.7.4 (CVE-2026-42544, unauthenticated DoS via WebSocket
  subprotocol header panic; CVE-2026-42545, WSGI response-header-panic DoS).
  granian is a selectable proxy server (proxy_cli).
- pyarrow 22.0.0 -> 23.0.1 (CVE-2026-25087 / PYSEC-2026-113).
- semantic-router 0.1.12 -> 0.1.15: 0.1.12 was yanked (CVE-2026-42208 — its
  unbounded litellm pin could resolve a credential-exfiltrating litellm==1.82.8
  wheel).

Not fixable by bump: diskcache 5.6.3 (CVE-2025-69872, unsafe pickle
deserialization) has no upstream fix and is left pinned; exploiting it requires
write access to the local cache directory.

Relock side effect: sse-starlette 3.4.2 -> 3.4.4.

* deps: relax exact pins in optional extras to compatible ranges

The proxy/optional extras exact-pinned every dependency, which (1) forces
downstream `pip install litellm[proxy]` consumers into version lockstep and
(2) blocks them from pulling transitive security patches without forking — the
structural cause behind needing a litellm release to clear the starlette CVE in
the previous commit.

Convert the ordinary extras deps to `>=current,<next_major` ranges, mirroring
the core [project].dependencies style. Reproducibility for litellm's own
Docker/CI is unaffected: images install via `uv sync --frozen`, and the lock
re-resolves to the identical versions (no locked version changed).

Kept exact-pinned:
- litellm-proxy-extras, litellm-enterprise — litellm's own sub-packages,
  versioned in lockstep with the release.
- opentelemetry-api/sdk/exporter-otlp — must resolve to matching versions.
- grpcio — supply-chain-pinned to a vetted, aged release.

Also corrects the stale comment claiming the extras are exact-pinned for Docker
reproducibility (the images use the lock, not these pins).

* fix(ci): resolve license-check lookup version from the floor for ranged deps

check_licenses.py derived the PyPI lookup version with
`next(iter(req.specifier))`, which returns an arbitrary specifier clause. For
a range like `>=0.12.1,<1.0` it picked the upper bound (`1.0`) — a version
that doesn't exist on PyPI — so the license lookup 404'd and the package was
flagged as having an unknown license.

The previous commit's switch from exact pins to ranges exposed this for
soundfile, pyroscope-io, redisvl, diskcache, and mlflow (the ranged deps not
already in liccheck.ini's allowlist). Prefer a lower-bound/exact version (a
real released version) for the lookup.

* fix(proxy): set strict_content_type=False on the FastAPI app

Starlette 1.0 / FastAPI 0.13x flipped the default to strict_content_type=True,
which refuses to parse a JSON request body when the client omits the
Content-Type header. The proxy previously accepted those requests, so the
fastapi/starlette bump in this PR would silently break clients that don't send
a Content-Type. Restore the prior lenient behavior explicitly.

Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>

* fix(tests/vcr): mint Google OAuth tokens live to prevent stale-token replay (BerriAI#29229)

The Redis-backed VCR layer was recording and replaying the Google
OAuth2/STS token-mint call. The replayed ya29.* access token is
long-expired, but its recorded expires_in keeps credentials.expired
False, so litellm never refreshes it and sends the stale token to a live
Vertex/Gemini endpoint, which returns 401 ACCESS_TOKEN_EXPIRED. This
broke live partner-model tests whose completion call is not itself
cassette-backed (e.g. test_vertex_ai_llama_tool_calling).

Force credential-exchange hosts to pass through live (never recorded,
never replayed) by returning None from before_record_request, mirroring
the existing telemetry passthrough, so a fresh token is minted each run.

Regression from BerriAI#28826, which added OAuth-token matcher tolerance plus
TTL-refresh-on-read so a stale token episode matched and never expired.

* chore(cookbook): bump Go directive to 1.26.3 in gollem example (BerriAI#29234)

Updates the gollem_go_agent_framework example to the current Go release.
Clears stale Go stdlib advisories reported by osv-scanner against the
older 1.25.1 directive. No source changes; the single pinned dependency
(gollem v0.1.0) is backward compatible.

* chore(ci): bump version (BerriAI#29242)

* bump: version 1.87.0 → 1.88.0

* uv lock

* feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags (BerriAI#29238)

* feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags

Register claude-opus-4-8 across the anthropic/bedrock/vertex/azure cost-map
entries, BEDROCK_CONVERSE_MODELS, and the setup-wizard provider list.

Prune two reasoning-effort fields from the cost map:
- Drop supports_minimal_reasoning_effort from the Claude fleet (58 entries).
  "minimal" is not a real Anthropic effort level (the API accepts only
  low/medium/high/xhigh/max), so LiteLLM degrades it to "low" regardless;
  the flag was inert and misleading on Anthropic.
- Remove tool_use_system_prompt_tokens everywhere (103 entries). It is not in
  the ModelInfo type and is read by no production code.

Update the affected config/schema tests; the reasoning-effort registry tests
now assert the Claude fleet omits supports_minimal.

* fix(anthropic): recognize output_config effort after minimal-flag prune

Pruning supports_minimal_reasoning_effort from the Claude fleet removed the
only "supports effort param" marker from 11 Opus 4.5 / mythos-preview map
entries that lack supports_output_config. _model_supports_effort_param then
returned False for them, so output_config was wrongly dropped under
drop_params=True -- regressing
test_anthropic_model_supports_effort_param_recognizes_supporting_models for
claude-opus-4-5-20251101 and the mythos preview.

- _model_supports_effort_param now treats supports_output_config as a
  sufficient signal, matching the bedrock-invoke call sites that already
  check supports_output_config OR a reasoning-effort flag. Shared map lookup
  extracted into _supports_model_capability.
- Add supports_output_config: true to the 11 Opus 4.5 / mythos entries that
  lost their only marker, restoring prior effort-forwarding behavior without
  re-adding the inert minimal flag.

* fix(ci): restore real Bedrock batch S3 bucket and role in oai_misc_config (BerriAI#29245)

The OSS-staging sync (d52fbfb) overwrote the Bedrock batch model's
s3_bucket_name and aws_batch_role_arn with public-safe placeholders
(account 123456789012 / *_EXAMPLE role). The e2e_openai_endpoints CI job
runs the proxy with AWS account 941277531214 credentials, so on file
upload test_bedrock_batches_api failed with:

    NoSuchBucket: The specified bucket does not exist
    <BucketName>litellm-proxy-123456789012</BucketName>

Restore the real resources that live in account 941277531214 (verified
to exist) — the same values tests/batches_tests/test_bedrock_files_and_batches.py
already references.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(guardrails): persist disable_global_guardrails on keys (BerriAI#29233)

* fix(guardrails): restore disable_global_guardrails persistence for keys

The per-key/team "Disable Global Guardrails" toggle silently stopped
working after BerriAI#17042, which removed `disable_global_guardrails` from the
key/team request models and from the premium metadata allowlist. Without
those, the UI's top-level field was dropped by pydantic and never folded
into key `metadata`, so the runtime gate always read False and global
default_on guardrails kept running.

Restore the request-model fields (KeyRequestBase, NewTeamRequest,
UpdateTeamRequest) and the `LiteLLM_ManagementEndpoint_MetadataFields_Premium`
entry so the flag is promoted into metadata again. Because the key edit
form always submits the flag (false by default), guard the UI so it is
only sent when it actually changed (edit) or is enabled (create) — this
keeps the premium gate on enabling intact while not 403-ing non-premium
users who edit unrelated key fields, mirroring how guardrails/tags are
already stripped.

* test(guardrails): cover disable_global_guardrails toggle-off + clarify premium field comment

Add a prepare_metadata_fields case asserting `disable_global_guardrails: False`
overwrites an existing `True`, and rewrite the PREMIUM_METADATA_FIELDS comment to
explain why boolean premium fields are excluded from the empty-value strip loop.

* test(e2e): cover Team Admin view + member + key flows (BerriAI#29072)

* test(e2e): cover Team Admin view + member + key flows

Adds a new spec exercising the previously-uncovered team-admin manual-QA
items: viewing all team keys (including other members'), adding a member,
removing a member, and creating a team key with All Team Models. Also
seeds a dedicated invitee user so the add-member test can run in parallel
with the proxy-admin invite test without colliding on the team roster.

* test(e2e): harden team-admin member specs per review feedback

Address Greptile feedback on the Team Admin spec:
- locate the delete action via getByTestId("delete-member") instead of
  the fragile svg/img .last() selector
- match the seeded removable member by user_id (members_with_roles stores
  no email, so the roster renders user_id)
- assert exact success-toast strings rather than broad regexes that could
  match unrelated "success" text

* docs: hand-written CLAUDE.md; point GEMINI.md and AGENTS.md at it (BerriAI#29252)

* docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md

Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md.

* fix: further clarify communication guidelines

* docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance

Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth.

* docs: simplify PR template test checklist item

Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item.

* docs: point AGENTS.md at CLAUDE.md instead of deleting it

Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth.

* fix: make AI-generated rules more concise

* fix: spelling

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: make the .env usage more careful

* docs: restore MCP available_on_public_internet note to CLAUDE.md

The PR description states this note was carried over verbatim from the
previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the
file matches the description and the team guidance is not lost.

* docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md

These security-relevant rules were dropped in the rewrite. Restore the
sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain
rules (no curl|bash, pin versions, verify checksums) so agents editing UI
or CI code are still steered away from those pitfalls.

* docs: move area-specific guidance into nested CLAUDE.md files

The MCP, browser-storage, and CI supply-chain notes are scoped to
particular parts of the tree, so move each into a nested CLAUDE.md that
Claude Code loads on demand when those files are touched: the MCP note
under the mcp_server gateway, the browser-storage rule under the UI
dashboard, and the CI supply-chain rules under .circleci. Keeps the root
CLAUDE.md focused on general guidance while the area notes surface where
they are relevant.

* docs: keep CI supply-chain note in root CLAUDE.md

CI guidance applies beyond .circleci (it also covers downloads in GitHub
workflows and any CI script), and CI work does not reliably touch a single
subtree, so a nested file under .circleci would not surface it dependably.
Keep it in the always-loaded root instead. The MCP and browser-storage
notes stay nested where they map cleanly to one area of the tree.

* fix: make it clear we prefer httpOnly

* chore: make ci rule more concise

* chore: make concise

Fix formatting and punctuation in MCP note.

* fix: don't include Claude attribution

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: regenerate uv.lock to sync with pyproject.toml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Dennis Henry <dennis.henry@okta.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…erriAI#28970)

* feat(guardrails): wire apply_guardrail into proxy logging callbacks

Route /apply_guardrail through pre/post proxy hooks and LiteLLM success/failure handlers so Langfuse and OTEL integrations receive input/output on guardrail-only requests.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(guardrails): fix Greptile review comments on apply_guardrail logging

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(apply_guardrail): preserve original exception and capture modified response

- Capture return value from post_call_success_hook so callback-modified
  responses propagate to the caller.
- Wrap success/failure logging calls in defensive try/except so logging
  infrastructure failures don't replace the user-visible response or mask
  the original guardrail exception.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* Fix mypy

* fix(apply_guardrail): isolate failure logging and use post-hook response for logging

- Split async_failure_handler and post_call_failure_hook into independent
  try/except blocks so a callback bug in one does not silently skip the
  other.
- Build response_for_logging inside _emit_guardrail_success_logs after
  post_call_success_hook runs, so logged data matches the response the
  caller actually receives when the hook modifies the response.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(apply_guardrail): fix black formatting and update tests for fastapi_request param

- Run black on guardrail_endpoints.py to fix CI formatting check
- Add _mock_proxy_logging() helper to enterprise guardrail tests to patch
  proxy-server globals imported at call time
- Pass fastapi_request=Mock() in all direct apply_guardrail test calls
  to match updated function signature

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(guardrails): use transformed exception from post_call_failure_hook in apply_guardrail

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(guardrails): isolate sync/async logging handlers in apply_guardrail

Separate each logging handler call into its own try/except so a failure
in the async handler does not silently skip the sync handler submission
(and vice versa). Matches the docstring's defensive intent.

Co-authored-by: Yassin Kortam <yassin@berri.ai>

* fix(apply_guardrail): guard transformed_exception with isinstance check

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(guardrails): mock proxy globals in not_found test and share apply_guardrail logging fixture

- Add proxy-server global mocks to test_apply_guardrail_not_found so the
  failure-path post_call_failure_hook call doesn't touch the real proxy
  logging singleton.
- Extract the duplicated _mock_proxy_logging context manager out of the
  two enterprise apply_guardrail test files into a shared conftest fixture
  so the helper stays in one place.

* fix(guardrails): use update_messages to keep logging obj in sync

Co-authored-by: Yassin Kortam <yassin@berri.ai>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants