Fix block_code_execution: response-side bypass and phrase tightening - #22149
Merged
1 commit merged intoFeb 26, 2026
Merged
Conversation
…tighten no-execution phrases **Core bug fix**: Response-side blocking was silently disabled with detect_execution_intent=True (default) because execution-intent heuristics were applied to LLM output text, which doesn't contain phrases like 'run this'. Now input_type is threaded through _scan_text to skip intent checks for responses while still blocking detected code blocks. **Tightened overly broad no-execution phrases**: Replaced broad patterns like "what would ", "can you explain", and "explain what this " with more specific forms (e.g. "what would happen if", "can you explain this code") to prevent trivial bypass. **Added tests**: 7 new test cases covering response-side blocking with default settings, casual phrases in LLM output, and tightened phrase patterns. All 23 tests pass + 100% compliance dataset compliance (100/100). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Author
|
@greptile please review this |
Contributor
Greptile SummaryThis PR fixes two critical logic bugs in the
Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/block_code_execution/block_code_execution.py | Fixes critical response-side bypass by threading input_type through _scan_text and skipping execution-intent heuristics for responses. Tightens overly broad no-execution phrases. Logic is correct and well-documented with clear inline comments. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_block_code_execution.py | Adds 7 targeted tests covering the response-side bypass bug, phrase tightening, and ensuring legitimate explain requests still pass. All tests are mock-only with no network calls. Good coverage of both block and mask actions for response input type. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["apply_guardrail(input_type)"] --> B["_scan_text(text, detections, input_type)"]
B --> C{input_type?}
C -->|request| D{detect_execution_intent?}
C -->|response| H["Skip intent heuristics"]
D -->|True| E{_has_no_execution_intent?}
D -->|False| G["Find & enforce blocks"]
E -->|Yes| F["Return text, no block"]
E -->|No| G
G --> I{Blocks found?}
I -->|Yes| J{Compute effective_block}
I -->|No, request + intent| K["Block text-only exec request"]
I -->|No, otherwise| L["Return text, no block"]
H --> M["Find blocks"]
M --> N{Blocks found?}
N -->|Yes| O["Always enforce block action"]
N -->|No| L
J --> P{action == block?}
O --> P
P -->|block| Q["Raise HTTPException / ModifyResponseException"]
P -->|mask| R["Replace with CODE_BLOCK_REDACTED"]
Last reviewed commit: 8a3a787
This pull request was closed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Fixes Greptile review issues on #22056 (confidence score 2/5 → targeting 5/5)
Pre-Submission checklist
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
🐛 Bug Fix
Changes
Fixes the two critical logic bugs identified by Greptile in the block_code_execution guardrail, plus adds comprehensive test coverage.
1. Response-side guardrail bypass (critical): With
detect_execution_intent=True(default),_scan_textapplied execution-intent heuristics to LLM response text. Since LLM responses don't contain phrases like "run this", response-side blocking was silently disabled. Fix: threadinput_typethrough_scan_textand skip intent checks entirely for responses.2. Overly broad no-execution phrases: Replaced
"what would ","can you explain", and"explain what this "with more specific forms (e.g."what would happen if","can you explain this code") to prevent trivial guardrail bypass via adversarial prompts.3. Tests: Added 7 new test cases covering the exact
detect_execution_intent=True+input_type="response"combination that exposed the core bug, plus tightened phrase patterns. All 23 tests pass + 100/100 compliance dataset.