Skip to content

fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging #44067

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yucheng-berri merged 5 commits into from
Oct 2, 2026
Merged

fix(guardrails): restore Azure guardrail get_user_prompt dispatch and allow logging #44067

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1ccda8f
fix(guardrails): restore Azure guardrail get_user_prompt dispatch and…
yucheng-berri Oct 1, 2026
ca1f5a0
test(guardrails): use transport-level doubles in Azure dispatch regre…
yucheng-berri Oct 1, 2026
8ca98df
test(guardrails): add Azure dispatch audit matrix integration cells
yucheng-berri Oct 1, 2026
ac0e8bd
test(guardrails): restore global callback lists after Router reset in…
yucheng-berri Oct 1, 2026
53e6e47
test(guardrails): make proxy restart cell independent of shutdown timing
yucheng-berri Oct 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions litellm/proxy/guardrails/guardrail_hooks/azure/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,20 @@ def split_text_by_words(text: str, max_length: int) -> list[str]:

return chunks

def get_user_prompt(self, messages: list[AllMessageValues]) -> str | None:
"""
Get the last consecutive block of messages from the user.

Example:
messages = [
{"role": "user", "content": "Hello, how are you?"},
{"role": "assistant", "content": "I'm good, thank you!"},
{"role": "user", "content": "What is the weather in Tokyo?"},
]
get_user_prompt(messages) -> "What is the weather in Tokyo?"
"""
return get_last_user_message(messages)

def get_user_prompt_from_request(self, data: Mapping[str, object], call_type: CallTypesLiteral) -> str | None:
if call_type in _RESPONSES_API_CALL_TYPES:
responses_input: Final = data.get("input")
Expand All @@ -147,6 +161,6 @@ def get_user_prompt_from_request(self, data: Mapping[str, object], call_type: Ca
return get_last_user_message(ResponsesAPIRequestUtils.responses_input_to_chat_messages(validated_input))

messages: Final = data.get("messages")
if not isinstance(messages, list):
if messages is None:
return None
return get_last_user_message(cast(list[AllMessageValues], messages)) # cast-ok: narrowed to list
return self.get_user_prompt(cast(list[AllMessageValues], messages)) # cast-ok: sequence of request messages

@devin-ai-integration devin-ai-integration Bot Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Tuple messages break list-based prompt overrides

For tuple messages, get_user_prompt_from_request passes a tuple to list-based get_user_prompt overrides. An override calling messages.copy() fails before Azure scans the prompt.

Learn more

The shared Azure base extracts user text from chat messages. SDK callers and earlier pre-call guardrails can supply tuples, but get_user_prompt advertises a list to subclasses. The dispatch passes the tuple unchanged, so an override using a valid list method fails before either Azure guardrail scans it. The built-in implementation happens to work because it only iterates backward.

Example: An earlier hook writes data["messages"] = ({"role": "user", "content": "hello"},). An override uses messages.copy() before choosing which turns to scan. It receives a tuple and raises instead of sending "hello" to Azure.

Recommended fix: Convert tuple messages to a list before calling self.get_user_prompt, while preserving the list object for list-valued messages. Add a test with an override that uses a list operation on tuple input.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@devin-ai-integration devin-ai-integration Bot Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches the pre-#43786 parent, which passed the tuple unchanged to self.get_user_prompt. Converting to list is new behavior, so I'm leaving it for the maintainer

5 changes: 4 additions & 1 deletion litellm/proxy/guardrails/guardrail_hooks/azure/prompt_shield.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
GuardrailTracingDetail,
)

from .base import AZURE_CONTENT_SAFETY_TEXT_RECORD_LENGTH, AzureGuardrailBase
from .base import _RESPONSES_API_CALL_TYPES, AZURE_CONTENT_SAFETY_TEXT_RECORD_LENGTH, AzureGuardrailBase

if TYPE_CHECKING:
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
Expand Down Expand Up @@ -249,6 +249,9 @@ async def async_pre_call_hook(
"Azure Prompt Shield: Running pre-call prompt scan, on call_type: %s",
call_type,
)
if call_type not in _RESPONSES_API_CALL_TYPES and data.get("messages") is None:
verbose_proxy_logger.warning("Azure Prompt Shield: not running guardrail. No messages in data")
return data
user_prompt: Final = self.get_user_prompt_from_request(data, call_type)

if user_prompt:
Expand Down
5 changes: 4 additions & 1 deletion litellm/proxy/guardrails/guardrail_hooks/azure/text_moderation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from litellm.types.guardrails import GuardrailEventHooks
from litellm.types.utils import CallTypesLiteral, GenericGuardrailAPIInputs, LLMResponseTypes

from .base import AzureGuardrailBase
from .base import _RESPONSES_API_CALL_TYPES, AzureGuardrailBase

if TYPE_CHECKING:
from litellm.caching.caching import DualCache
Expand Down Expand Up @@ -231,6 +231,9 @@ async def async_pre_call_hook(
"Azure Text Moderation: Running pre-call prompt scan, on call_type: %s",
call_type,
)
if call_type not in _RESPONSES_API_CALL_TYPES and data.get("messages") is None:
verbose_proxy_logger.warning("Azure Text Moderation: not running guardrail. No messages in data")
return data
user_prompt: Final = self.get_user_prompt_from_request(data, call_type)

if user_prompt:
Expand Down
Loading
Loading