fix(guardrails): send only new messages since last assistant turn to CrowdStrike AIDR - #31230
Conversation
Greptile SummaryThis PR optimizes the CrowdStrike AIDR guardrail by filtering the conversation to only send new messages since the last assistant turn (plus all system messages), rather than the full history. When the API returns a transformation, the redacted content is stitched back into the complete message list using index tracking, and the response guardrail no longer forwards request history at all.
Confidence Score: 5/5Safe to merge — the change is scoped to the CrowdStrike AIDR guardrail hook, all edge cases are covered by new tests, and the 29-test suite passes cleanly. The filtering and stitching logic is correct across all traced paths: system messages are always retained, index bookkeeping maps filtered positions back to the full array without off-by-one errors, the id()-based writeback correctly falls back to structured_messages when identity cannot be confirmed, and the response-path simplification is intentional and well-tested. No regressions or behavior drift were identified. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/crowdstrike_aidr/crowdstrike_aidr.py | Core guardrail handler refactored to filter conversation to only new messages since the last assistant turn before calling CrowdStrike AIDR; adds index-tracking and stitching logic to correctly merge transformed texts and structured messages back into the full conversation. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_crowdstrike_aidr.py | Existing tests updated to reflect the new behavior (response guardrail no longer forwards conversation history; skipped-message assertion now validates redacted output); comprehensive new tests added covering message-filtering edge cases, transform stitching, and writeback logic. |
Reviews (5): Last reviewed commit: "fix(guardrails): send only new messages ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Greptile SummaryThis PR optimises the CrowdStrike AIDR guardrail integration by filtering the messages sent to the API on each turn: for request guardrails, only system messages plus the messages since the last assistant reply are forwarded; for response guardrails, only the assistant output texts are sent (conversation history is dropped). A new
Confidence Score: 3/5Safe to merge for the common path, but the response guardrail can now silently fire an unnecessary API call (with an empty messages array) when no output text is present, and the warning that previously caught this case was removed. The filtering logic and transformation stitching are well-tested and the core multi-turn optimisation is correct. The main concern is that _build_guard_input_for_response no longer returns None for empty output texts, making the None-guard in apply_guardrail dead code and silently changing behaviour for tool-call-only responses (an API request with an empty messages array is sent instead of an early return with a diagnostic warning). There is no test covering this edge case. crowdstrike_aidr.py — specifically _build_guard_input_for_response and the dead
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/crowdstrike_aidr/crowdstrike_aidr.py | Adds _messages_since_last_assistant filtering and index-based transformation stitching; response path now silently makes an API call with empty messages when no output texts exist (dead-code early-return guard). |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_crowdstrike_aidr.py | Existing response-path tests updated to remove history from expected payloads (matching new behaviour); 6 new TestMessageFiltering tests and 5 new top-level tests cover the filtering and stitching logic. No test for empty output_texts on the response path. |
Reviews (2): Last reviewed commit: "fix(guardrails): send only new messages ..." | Re-trigger Greptile
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
0348923 to
a9a9241
Compare
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 3 · PR risk: 0/10 |
|
Thanks for this optimization — the message filtering logic looks well thought out and the test coverage is thorough. The Greptile review is stale (it was run against an earlier commit). Triggering a fresh review against the current HEAD now:\n\n@greptileai |
3a14077 to
13163b7
Compare
13163b7 to
4cc94e3
Compare
4cc94e3 to
ae61dd3
Compare
…CrowdStrike AIDR Previously, every guardrail request forwarded the full conversation history to CrowdStrike AIDR. In a multi-turn conversation this means every prior message gets re-scanned on every new call, even though those messages were already evaluated in earlier turns. CrowdStrike AIDR internally has a conversation boundary optimization in place for just this scenario (ref. <https://aidr-docs.crowdstrike.com/docs/aidr/apis#messages-array-optional---array-of-message-objects-containing-a-conversation-segment-with-the-ai-system>). However, it is nevertheless wasteful to send so much data to the API when only a subset of it will be processed. It also risks hitting the documented 1 MiB request size limit. So now we filter down to system messages plus either the messages after the last assistant turn, or the last assistant message itself when that is what is being guarded. We also preserve the original, full message history within the guardrail in order to stitch back any transformations.
ae61dd3 to
b5d63b9
Compare
|
hey thanks for this, lgtm. there's an awkward quirk in our ci pipeline rn for oss as we are moving to different system, so i will need to port this to a internal staging branch! |
3 participants
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Details
Type
🐛 Bug Fix
Changes
Previously, every guardrail request forwarded the full conversation history to CrowdStrike AIDR. In a multi-turn conversation this means every prior message gets re-scanned on every new call, even though those messages were already evaluated in earlier turns.
CrowdStrike AIDR internally has a conversation boundary optimization in place for just this scenario (ref. https://aidr-docs.crowdstrike.com/docs/aidr/apis#messages-array-optional---array-of-message-objects-containing-a-conversation-segment-with-the-ai-system). However, it is nevertheless wasteful to send so much data to the API when only a subset of it will be processed. It also risks hitting the documented 1 MiB request size limit.
So now we filter down to system messages plus either the messages after the last assistant turn, or the last assistant message itself when that is what is being guarded. We also preserve the original, full message history within the guardrail in order to stitch back any transformations.