feat(guardrails): add Cisco AI Defense integration (#28249)

[yassin-berriai]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[codecov]

Codecov Report

❌ Patch coverage is 82.06708% with 262 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ardrail_hooks/cisco_ai_defense/cisco_ai_defense.py	84.04%	172 Missing ⚠️
...ail_hooks/cisco_ai_defense/cisco_ai_defense_mcp.py	73.37%	86 Missing ⚠️
...rails/guardrail_hooks/cisco_ai_defense/__init__.py	87.50%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds Cisco AI Defense as a new guardrail integration for LiteLLM, supporting both chat conversation inspection (/api/v1/inspect/chat) and MCP tool-call inspection (/api/v1/inspect/mcp). It also registers Cisco in the UI guardrail garden and ships a thorough mock-based test suite.

• Core guardrail (cisco_ai_defense.py, cisco_ai_defense_mcp.py): implements pre/during/post-call hooks, streaming buffering, and async_post_mcp_tool_call_hook for MCP output scanning; supports block/monitor on-flagged actions, redact rewrites across all message shapes (chat, text-completion, Responses API, MCP), and fail-open/fail-closed fallback on API errors.
• HTTP layer bug: timeout=self.timeout is passed to build_request(), which does not accept that parameter in httpx; the send() call has no explicit timeout override, so the configured 1–60 s guardrail timeout is never applied and requests run against the client's 600-second default.
• Unrelated file included: tests/llm_translation/reasoning_effort_grid/grid_spec.py adds a fail_reason for an unrelated Anthropic model and should be removed from this PR.

Confidence Score: 4/5

• The integration is functionally complete and well-tested, but the HTTP timeout is not applied correctly — all Cisco API calls run with a 600-second ceiling rather than the operator's configured 1–60 s limit.
• The guardrail logic, redaction paths, MCP hook dispatch, and streaming buffering are all well-structured and backed by a large mock test suite. The one active defect is that timeout=self.timeout is given to httpx's build_request() (which rejects unknown kwargs) rather than to send(), meaning the per-guardrail timeout config has no runtime effect and requests can hang for up to 600 seconds instead of the intended 10 s default. The unrelated grid_spec.py change is a minor hygiene issue. Neither finding is a blocking security concern, but the timeout bug warrants a fix before the guardrail behaves as documented.
• cisco_ai_defense.py — specifically the _post_inspection method where the httpx send call needs the timeout moved from build_request to send.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py	Core guardrail implementation (~2358 lines) covering chat/MCP inspection, redaction, streaming support, and verdict parsing. Has a P1 bug where timeout=self.timeout is passed to httpx's build_request() (which doesn't accept it) instead of to send(), so the configured 1–60 s timeout is never applied and all scans run against the 600 s client default.
litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense_mcp.py	MCP-specific mixin: post-tool-call hook, JSON-RPC envelope building, MCP response normalization, and in-place redaction of tool output. Logic is well-structured; the HTTPException-to-blocking-response conversion in the post-tool hook correctly prevents the exception from propagating to the non-blocking success-callback dispatcher in litellm_logging.
litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/init.py	Guardrail initializer and registries. Correctly adds the callback to both litellm.callbacks (for pre/during/post_call hooks) and litellm.success_callback (for async_post_mcp_tool_call_hook dispatch). The _get_optional_value helper avoids inheriting sibling defaults from a shared LitellmParams object.
litellm/types/proxy/guardrails/guardrail_hooks/cisco_ai_defense.py	Pydantic config model with typed fields for all Cisco-specific parameters. Clean and well-documented.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_cisco_ai_defense_chat.py	Comprehensive mock-based test suite for chat inspection (2842 lines). Tests cover pre/during/post call hooks, streaming, redaction paths, error fallback, and verdict metadata. The _patch_inspection_post helper patches send but not build_request, so the timeout-parameter bug would manifest as a fallback error in tests that use the mock (but is obscured because except Exception in _inspect_chat catches it silently).
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_cisco_ai_defense_mcp.py	Mock-based tests for MCP inspection (832 lines). Covers request inspection, response scanning, blocking, redaction, and the post-tool-call hook. Inherits the same timeout-bug exposure as the chat tests.
tests/llm_translation/reasoning_effort_grid/grid_spec.py	Marks claude-fable-5 with a fail_reason explaining it's not yet available on the Anthropic API. This change is unrelated to Cisco AI Defense and appears to have been accidentally included.

_{Reviews (11): Last reviewed commit: "fix(guardrails): validate Cisco enabled_..." | Re-trigger Greptile}

[veria-ai]

PR overview

This pull request adds a Cisco AI Defense guardrail integration for the LiteLLM proxy, including inspection and handling of prompts, completions, MCP content, and tool/function definitions. The touched guardrail hook code processes Cisco verdicts and applies allow, block, or redact behavior to proxy traffic.

There are still two open security issues in the Cisco AI Defense handling. Both allow an authenticated caller to bypass intended guardrail behavior in specific cases: unsafe Cisco verdicts without an explicit action may be allowed, and redaction may fail to sanitize tool/function definitions. Four issues have already been addressed, so the PR is improving, but these remaining gaps still weaken blocking and redaction guarantees for deployments using this guardrail.

Open issues (2)

• Medium: Unsafe Cisco verdicts can be allowed — litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py:1120
• Medium: Redaction bypass in tool definitions — litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py:1536

Fixed/addressed: 4 · PR risk: 6/10

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ mateo-berri
✅ munnr
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mateo-berri]

@greptileai

[mateo-berri]

LGTM; thanks!

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[veria-ai]

Medium: Unsafe Cisco verdicts can be allowed

When Cisco returns a verdict without an action, this defaults to allow even if is_safe is False. An authenticated caller can get prompt, completion, or MCP content through a configured blocking guardrail whenever the inspection response reports unsafe content using is_safe/classifications but does not include an action string.

Suggested change

	action = _ACTION_ALLOW
	action = _ACTION_BLOCK if verdict.is_safe is False else _ACTION_ALLOW

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[greptile-apps]

Streaming block emits HTTP 200 instead of HTTP 400

All chunks are buffered before _inspect_chat is called, so no bytes have been sent to the client when HTTPException is raised. Catching it here and yielding a raw SSE error string causes the proxy to return HTTP 200 with an error embedded in the body. Non-streaming blocked requests return HTTP 400, so clients checking the HTTP status code will see an apparent success for a streaming block. Re-raising the exception would let the proxy infrastructure send a proper HTTP 400 before streaming begins — the same approach used by PANW Prisma AIRS and other guardrail hooks in this codebase.

[mateo-berri]

This is a false positive. All chunks are buffered first, then _inspect_chat raises HTTPException(status_code=400) before any chunk is yielded, so the SSE error event is the very first item the generator emits. create_response in litellm/proxy/common_request_processing.py inspects that first chunk via _parse_event_data_for_error; when it finds error.code in the 100-599 range it returns JSONResponse(status_code=code) instead of a StreamingResponse. _http_exception_to_error_obj sets code = exc.status_code (400 for a block), so the client receives an HTTP 400 JSON error, matching the non-streaming path.

This is exactly the PANW Prisma AIRS pattern cited in the comment. PANW also yields an SSE error event from its except HTTPException branch rather than re-raising, and its own inline comment notes that raising from the generator hits create_response's generic except and surfaces as a 500, not a 400. So re-raising here would regress the status code, not fix it

[greptile-apps]

timeout silently ignored — every scan uses the client's 600 s default

build_request() in httpx does not accept a timeout keyword argument (the signature is method, url, *, content, data, files, json, params, headers, extensions). Passing timeout=self.timeout there either raises a TypeError (caught by the broad except Exception two levels up, causing every scan to fall back as an API error) or is silently discarded. Either way, the send() call carries no per-request timeout and will use the AsyncHTTPHandler's client-level default of 600 seconds (COMPLETION_HTTP_FALLBACK_SECONDS), not the operator-configured 1–60 s guardrail timeout.

The fix is to move the timeout to send():

[mateo-berri]

This is a false positive against the pinned httpx version. The project pins httpx==0.28.1 (pyproject.toml), whose AsyncClient.build_request signature is build_request(method, url, *, content, data, files, json, params, headers, cookies, timeout=USE_CLIENT_DEFAULT, extensions). It does accept timeout, and it writes the value into request.extensions['timeout'], which send() honors. Verified against 0.28.1: client.build_request('POST', url, json=..., timeout=10.0).extensions['timeout'] is {'connect': 10.0, 'read': 10.0, 'write': 10.0, 'pool': 10.0}, overriding the client-level default. So the operator-configured 1-60s guardrail timeout is applied per request.

The suggested change is the opposite of correct. AsyncClient.send has signature send(request, *, stream=False, auth=..., follow_redirects=...) and takes no timeout kwarg, so moving timeout onto send() would raise TypeError and make every scan fail

[greptile-apps]

timeout silently ignored — every scan uses the client's 600 s default

build_request() in httpx does not accept a timeout keyword argument. Passing timeout=self.timeout there is either silently discarded or raises a TypeError (caught upstream, causing every scan to fail as an API error). Either way the send() call carries no per-request timeout and falls back to the AsyncHTTPHandler client-level default of 600 seconds, not the operator-configured 1–60 s guardrail timeout. Move the timeout to send().

Suggested change

	request = self.async_handler.client.build_request(
	"POST",
	url,
	headers=headers,
	json=payload,
	timeout=self.timeout,
	)
	response = await self.async_handler.client.send(
	request,
	follow_redirects=False,
	)
	request = self.async_handler.client.build_request(
	"POST",
	url,
	headers=headers,
	json=payload,
	)
	response = await self.async_handler.client.send(
	request,
	follow_redirects=False,
	timeout=self.timeout,
	)

[mateo-berri]

This is a false positive against the pinned httpx version. The project pins httpx==0.28.1 (pyproject.toml), whose AsyncClient.build_request signature is build_request(method, url, *, content, data, files, json, params, headers, cookies, timeout=USE_CLIENT_DEFAULT, extensions). It does accept timeout, and it writes the value into request.extensions['timeout'], which send() honors. Verified against 0.28.1: client.build_request('POST', url, json=..., timeout=10.0).extensions['timeout'] is {'connect': 10.0, 'read': 10.0, 'write': 10.0, 'pool': 10.0}, overriding the client-level default. So the operator-configured 1-60s guardrail timeout is applied per request.

The suggested change is the opposite of correct. AsyncClient.send has signature send(request, *, stream=False, auth=..., follow_redirects=...) and takes no timeout kwarg, so moving timeout onto send() would raise TypeError and make every scan fail

[greptile-apps]

You're right, I was wrong. The httpx source confirms that build_request() accepts timeout and stores it in request.extensions['timeout'], which send() honors automatically. And send() takes no timeout kwarg, so my suggested fix would have broken things. The code is correct as-is — apologies for the noise.

[veria-ai]

Medium: Redaction bypass in tool definitions

_extract_tool_definition_text() appends tool/function definitions for inspection, but this branch drops the corresponding sanitized message and then returns success after rewriting only the conversation. An authenticated caller can place PII or prompt-injection text in tools[].function.description or schema descriptions and have it forwarded unchanged whenever Cisco returns a redact verdict instead of block; either rewrite the original tools/functions fields from the sanitized payload, or return False so the request falls back to the configured block behavior.