feat(guardrails): return structured guardrail_response on /apply_guardrail for success and blocked responses - #29384
Conversation
|
|
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
Greptile SummaryThis PR surfaces structured guardrail results on the
Confidence Score: 5/5Safe to merge — the change is additive, the endpoint returns an enriched response on both paths, and all existing behaviour is preserved when no guardrail info is available. The previously flagged success-path gap (guardrail_response never populated on HTTP 200) is correctly resolved by moving request_data before the try block and calling _collect_guardrail_info_from_data after apply_guardrail returns. The failure path correctly converts HTTPException to ProxyException with provider_specific_fields, which the existing exception handler serialises at the right HTTP status code. The model addition is backwards-compatible. No new regressions are introduced. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_endpoints.py | Adds _collect_guardrail_info_from_data and _enrich_guardrail_block_exception helpers; moves request_data before the try block; populates guardrail_response on both the 200 and 403 paths correctly. |
| litellm/types/guardrails.py | Adds optional guardrail_response field to ApplyGuardrailResponse; backwards-compatible addition with None default. |
| tests/test_litellm/proxy/guardrails/test_apply_guardrail_response.py | New unit tests for both helpers and the model field; all mocked, no real network calls. |
Reviews (2): Last reviewed commit: "feat(guardrails): return structured guar..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| return ProxyException( | ||
| message=message if isinstance(message, str) else str(message), | ||
| type=ProxyErrorTypes.internal_server_error, | ||
| param="None", | ||
| code=e.status_code, | ||
| provider_specific_fields={"guardrail_response": info}, | ||
| ) |
There was a problem hiding this comment.
ProxyErrorTypes.internal_server_error is semantically wrong for a guardrail block. The HTTP status code (e.status_code, typically 403) is preserved correctly in code, but the type field in the serialized error body will read "internal_server_error", which is misleading for clients that decode it. A more descriptive type (e.g. bad_request_error, or ideally a dedicated guardrail_blocked value) should be used here.
| return ProxyException( | |
| message=message if isinstance(message, str) else str(message), | |
| type=ProxyErrorTypes.internal_server_error, | |
| param="None", | |
| code=e.status_code, | |
| provider_specific_fields={"guardrail_response": info}, | |
| ) | |
| return ProxyException( | |
| message=message if isinstance(message, str) else str(message), | |
| type=ProxyErrorTypes.bad_request_error, | |
| param="None", | |
| code=e.status_code, | |
| provider_specific_fields={"guardrail_response": info}, | |
| ) |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| for key in ("metadata", "litellm_metadata"): | ||
| container = data.get(key) or {} | ||
| found = container.get("standard_logging_guardrail_information") | ||
| if found: | ||
| entries.extend(found if isinstance(found, list) else [found]) |
There was a problem hiding this comment.
The loop iterates ("metadata", "litellm_metadata") and appends entries from each independently. If a guardrail (or a pre-call hook) writes standard_logging_guardrail_information into both data["metadata"] and data["litellm_metadata"], the same guardrail run will appear twice in the collected list. A deduplication step or a clear policy on which namespace takes precedence would prevent callers from seeing doubled results.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
a65a5a1 to
3d5dbd5
Compare
…drail (success + block)
The /apply_guardrail endpoint only returned response_text. This surfaces the
full guardrail result on both paths:
- Add guardrail_response: Optional[List[Dict[str, Any]]] to ApplyGuardrailResponse.
- On success (HTTP 200), populate it from the guardrail's own logged result
(standard_logging_guardrail_information, written into request_data in the
guardrail's finally block) via _collect_guardrail_info_from_data.
- On a block (HTTP 403), _enrich_guardrail_block_exception raises a ProxyException
with provider_specific_fields={"guardrail_response": ...} so the failure
response carries the same structured classification as success, instead of a
stringified dict in message.
Provider agnostic: works for any CustomGuardrail (litellm_content_filter,
Bedrock, etc.) because they all write to the shared
standard_logging_guardrail_information container.
Adds mocked unit tests for the helpers and the new model field.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3d5dbd5 to
2b18e2b
Compare
|
@greptileai please re-review. The success-path gap from the previous review is fixed: the endpoint now populates |
2 participants
Relevant issues
Related to #28970 (same endpoint, independent and self-contained change).
Type
New Feature
Changes
The
/apply_guardrailendpoint only returnedresponse_text. This change surfaces the full guardrail result on both the success path and the blocked path.Add a new optional field
guardrail_responsetoApplyGuardrailResponse.On success (HTTP 200), the endpoint populates
guardrail_responsefrom the guardrail's own logged result. Guardrails write their result (standard_logging_guardrail_information) into therequest_datadict in their finally block, so the endpoint reads it back from the same dict via_collect_guardrail_info_from_data.On a block (HTTP 403),
_enrich_guardrail_block_exceptionraises aProxyExceptionwithprovider_specific_fieldsset to the structured classification, so the failure response carries the same guardrail detail as success, instead of a stringified dict inside themessagefield.This is provider agnostic. It works for any
CustomGuardrail(litellm_content_filter, Bedrock, and others) because they all write to the sharedstandard_logging_guardrail_informationcontainer.Pre-Submission checklist
tests/test_litellm/(mocked unit tests for the helpers and the new model field)make test-unit@greptileaireview (will do right after opening)Screenshots / Proof of Fix
Success (HTTP 200):
{ "response_text": "What is the capital of India", "guardrail_response": [ { "guardrail_name": "content-safety-multi", "guardrail_status": "success", "guardrail_provider": "litellm_content_filter", "guardrail_response": [] } ] }Blocked (HTTP 403):
{ "error": { "message": "Content blocked: denied_medical_advice category keyword 'medicine' detected (severity: high)", "code": "403", "provider_specific_fields": { "guardrail_response": [ { "guardrail_name": "content-safety-multi", "guardrail_status": "guardrail_intervened", "guardrail_response": [ { "category": "denied_medical_advice", "keyword": "medicine", "severity": "high", "action": "BLOCK" } ] } ] } } }