Skip to content

Fix block_code_execution: response-side bypass and phrase tightening - #22149

Merged
1 commit merged into
litellm_dev_02_23_2026_p2from
krrishdholakia/fix-code-exec-guard-v2
Feb 26, 2026
Merged

Fix block_code_execution: response-side bypass and phrase tightening#22149
1 commit merged into
litellm_dev_02_23_2026_p2from
krrishdholakia/fix-code-exec-guard-v2

Conversation

@ghost

@ghost ghost commented Feb 26, 2026

Copy link
Copy Markdown

Relevant issues

Fixes Greptile review issues on #22056 (confidence score 2/5 → targeting 5/5)

Pre-Submission checklist

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

Fixes the two critical logic bugs identified by Greptile in the block_code_execution guardrail, plus adds comprehensive test coverage.

1. Response-side guardrail bypass (critical): With detect_execution_intent=True (default), _scan_text applied execution-intent heuristics to LLM response text. Since LLM responses don't contain phrases like "run this", response-side blocking was silently disabled. Fix: thread input_type through _scan_text and skip intent checks entirely for responses.

2. Overly broad no-execution phrases: Replaced "what would ", "can you explain", and "explain what this " with more specific forms (e.g. "what would happen if", "can you explain this code") to prevent trivial guardrail bypass via adversarial prompts.

3. Tests: Added 7 new test cases covering the exact detect_execution_intent=True + input_type="response" combination that exposed the core bug, plus tightened phrase patterns. All 23 tests pass + 100/100 compliance dataset.

…tighten no-execution phrases

**Core bug fix**: Response-side blocking was silently disabled with detect_execution_intent=True (default) because execution-intent heuristics were applied to LLM output text, which doesn't contain phrases like 'run this'. Now input_type is threaded through _scan_text to skip intent checks for responses while still blocking detected code blocks.

**Tightened overly broad no-execution phrases**: Replaced broad patterns like "what would ", "can you explain", and "explain what this " with more specific forms (e.g. "what would happen if", "can you explain this code") to prevent trivial bypass.

**Added tests**: 7 new test cases covering response-side blocking with default settings, casual phrases in LLM output, and tightened phrase patterns. All 23 tests pass + 100% compliance dataset compliance (100/100).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel

vercel Bot commented Feb 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Building Building Preview, Comment Feb 26, 2026 4:46am

Request Review

@ghost

ghost commented Feb 26, 2026

Copy link
Copy Markdown
Author

@greptile please review this

@greptile-apps

greptile-apps Bot commented Feb 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes two critical logic bugs in the block_code_execution guardrail:

  • Response-side bypass fixed: When detect_execution_intent=True (the default), execution-intent heuristics were applied to LLM responses. Since LLM output doesn't contain user-intent phrases like "run this", _has_execution_intent() always returned False for responses, silently disabling all response-side blocking. The fix threads input_type through _scan_text and skips intent checks entirely for responses, so code blocks in LLM output are always enforced.
  • Overly broad no-execution phrases tightened: Replaced "what would ", "can you explain", and "explain what this " with more specific forms (e.g., "what would happen if", "can you explain this code") to prevent adversarial prompts from trivially bypassing the guardrail.
  • 7 new tests added covering the exact detect_execution_intent=True + input_type="response" combination, phrase tightening for adversarial inputs, and ensuring legitimate explain requests still pass through.

Confidence Score: 5/5

  • This PR is safe to merge — it fixes critical security bypass bugs with correct logic and comprehensive test coverage.
  • The changes are minimal, focused, and correct. The response-side bypass fix properly threads input_type through the scan pipeline and the boolean logic for effective_block is sound. Phrase tightening removes known bypass vectors. All 7 new tests target the exact scenarios that exposed the bugs, and no existing behavior is broken. No database queries, no network calls, no provider-specific code outside llms/.
  • No files require special attention

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_hooks/block_code_execution/block_code_execution.py Fixes critical response-side bypass by threading input_type through _scan_text and skipping execution-intent heuristics for responses. Tightens overly broad no-execution phrases. Logic is correct and well-documented with clear inline comments.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_block_code_execution.py Adds 7 targeted tests covering the response-side bypass bug, phrase tightening, and ensuring legitimate explain requests still pass. All tests are mock-only with no network calls. Good coverage of both block and mask actions for response input type.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["apply_guardrail(input_type)"] --> B["_scan_text(text, detections, input_type)"]
    B --> C{input_type?}
    C -->|request| D{detect_execution_intent?}
    C -->|response| H["Skip intent heuristics"]
    D -->|True| E{_has_no_execution_intent?}
    D -->|False| G["Find & enforce blocks"]
    E -->|Yes| F["Return text, no block"]
    E -->|No| G
    G --> I{Blocks found?}
    I -->|Yes| J{Compute effective_block}
    I -->|No, request + intent| K["Block text-only exec request"]
    I -->|No, otherwise| L["Return text, no block"]
    H --> M["Find blocks"]
    M --> N{Blocks found?}
    N -->|Yes| O["Always enforce block action"]
    N -->|No| L
    J --> P{action == block?}
    O --> P
    P -->|block| Q["Raise HTTPException / ModifyResponseException"]
    P -->|mask| R["Replace with CODE_BLOCK_REDACTED"]
Loading

Last reviewed commit: 8a3a787

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@ghost
ghost merged commit 490beb5 into litellm_dev_02_23_2026_p2 Feb 26, 2026
4 checks passed
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants