feat(add-new-block_code_execution-guardrail): prevent agent from executing code - #22154
feat(add-new-block_code_execution-guardrail): prevent agent from executing code#221541 commit merged into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryAdds a new
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/block_code_execution/block_code_execution.py | Core guardrail implementation (662 lines). Well-structured with confidence scoring, intent heuristics, response-side hardening. Has duplicate/redundant phrase entries and regex edge case with trailing spaces after language tags. |
| litellm/proxy/guardrails/guardrail_hooks/block_code_execution/init.py | Guardrail initialization and registry integration. Follows the auto-discovery pattern from guardrail_hooks directory. Clean implementation with proper config extraction. |
| litellm/types/proxy/guardrails/guardrail_hooks/block_code_execution.py | Type definitions for config model, detection output, and blocked languages options. Uses json_schema_extra for UI integration with multiselect and percentage slider. |
| litellm/types/guardrails.py | Adds BLOCK_CODE_EXECUTION to SupportedGuardrailIntegrations enum, adds multiselect/percentage UI param types, and mixes in the new config model to LitellmParams. |
| litellm/proxy/guardrails/guardrail_endpoints.py | Fixes ui_type handling for string-based values (was calling .value on plain strings), adds min/max/step copy from json_schema_extra, adds Literal options for select dropdowns. Solid improvements. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_block_code_execution.py | 26 unit tests covering detection, blocking, masking, escaped newlines, response-side behavior, phrase tightening, and conflict resolution. All mock-based, no network calls. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_block_code_execution_compliance.py | Compliance test running 100-entry dataset. Shared mutable request_data across iterations causes state accumulation but doesn't affect correctness. No network calls. |
| ui/litellm-dashboard/src/components/guardrails/guardrail_provider_fields.tsx | Adds Slider component for percentage fields and extends ProviderParam interface. Hardcoded "0%"/"50%"/"100%" labels assume 0–1 range. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Incoming Text] --> B[_normalize_escaped_newlines]
B --> C{input_type?}
C -->|response| F[_find_blocks]
C -->|request| D{detect_execution_intent?}
D -->|no| F
D -->|yes| E{_has_no_execution_intent AND NOT _has_execution_intent?}
E -->|yes: pure explain| Z[Allow through]
E -->|no| F
F --> G{Blocks found?}
G -->|no, request with exec intent| H[Block: execution_request]
G -->|no, no intent| Z
G -->|yes| I[For each block]
I --> J{Language blocked AND confidence >= threshold?}
J -->|no| K[Allow block through]
J -->|yes, response| L[effective_block = true]
J -->|yes, request| M{has_execution_intent OR no intent detection?}
M -->|yes| L
M -->|no| K
L --> N{action?}
N -->|block| O[Raise HTTPException / ModifyResponseException]
N -->|mask| P[Replace with CODE_BLOCK_REDACTED]
Last reviewed commit: 91b3e76
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91b3e76cfa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| # Regex: fenced code block with optional language tag. Handles ```lang\n...\n``` | ||
| # Content between fences; does not handle nested ``` inside body (documented edge case). | ||
| FENCED_BLOCK_RE = re.compile(r"```(\w*)\n(.*?)```", re.DOTALL) |
There was a problem hiding this comment.
The detector regex only matches fences of the form \w*\n...; this misses common fenced blocks such as language tags with punctuation (c++, objective-c) or CRLF line endings (\r\n). In block-all mode, those prompts are silently treated as having no code block, so executable snippets can bypass the guardrail by changing fence formatting rather than content.
Useful? React with 👍 / 👎.
| for h in event_hook | ||
| ] | ||
| else: | ||
| _event_hook = GuardrailEventHooks(event_hook) |
There was a problem hiding this comment.
This conversion path assumes every non-list event_hook can be cast to GuardrailEventHooks, but LitellmParams.mode also supports Mode objects for tag-based routing. Passing a Mode here raises at initialization time, so block_code_execution cannot be enabled in configs that use tag-based guardrail modes even though other guardrails support that mode type.
Useful? React with 👍 / 👎.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| _NO_EXECUTION_PHRASES: Tuple[str, ...] = ( | ||
| "don't run", | ||
| "do not run", | ||
| "don't execute", | ||
| "do not execute", | ||
| "no execution", | ||
| "without running", | ||
| "without execute", | ||
| "just reason", | ||
| "explain without running", | ||
| "explain without execute", | ||
| "what would happen if", | ||
| "what would this output", | ||
| "what would the result be", | ||
| "? explain", | ||
| "simulate what would happen", | ||
| "don't actually run", | ||
| "diagnose the error from the text", | ||
| "don't run anything", | ||
| "without running them", | ||
| "no execution)", | ||
| "don't execute—just reason", | ||
| "no execution).", | ||
| "(no execution)", | ||
| "no db access", | ||
| "no db access).", | ||
| "don't execute it", | ||
| "don't run).", | ||
| "(no execution)", | ||
| "no builds/run", | ||
| "(don't run)", | ||
| "no execution).", |
There was a problem hiding this comment.
Exact duplicate entries in phrase tuples
_NO_EXECUTION_PHRASES contains exact duplicate entries:
"(no execution)"appears at lines 88 and 93"no execution)."appears at lines 87 and 96
Additionally, many entries are functionally redundant because substring matching is used (p in lower). For example, "don't run" (line 66) already matches any input that would match "don't actually run" (line 81), "don't run anything" (line 83), "(don't run)" (line 95), "but don't run" (line 97), "don't run it" (line 98), etc. Similarly, "no execution" (line 70) subsumes all the punctuation variants on lines 85-96.
The same pattern exists in _EXECUTION_REQUEST_PHRASES: "run this " (line 137) subsumes all of "run this python", "run this javascript", "run this bash", "run this code", etc. And "run `" (line 142) subsumes all "run `curl", "run `git", "run `docker" entries.
While redundancy doesn't cause incorrect behavior, it adds ~30+ unnecessary iterations per text scan and makes the phrase lists harder to maintain. Consider deduplicating.
| request_data = {} | ||
| passed = 0 | ||
| failed = [] | ||
| for item in compliance_dataset: |
There was a problem hiding this comment.
Shared mutable request_data across iterations
request_data = {} is shared across all 100 compliance iterations. The apply_guardrail method's finally block calls add_standard_logging_guardrail_information_to_request_data, which appends logging entries to request_data["metadata"]["standard_logging_guardrail_information"]. This means the dict accumulates entries from every iteration, growing unbounded throughout the test.
This won't cause test failures, but it means later iterations carry state from earlier ones, which could mask bugs (e.g., if a detection list carried over). Consider resetting request_data inside the loop:
| request_data = {} | |
| passed = 0 | |
| failed = [] | |
| for item in compliance_dataset: | |
| for item in compliance_dataset: | |
| prompt = item["prompt"] | |
| expected = item["expected_result"] | |
| inputs = {"texts": [prompt]} | |
| request_data = {} |
|
|
||
| # Regex: fenced code block with optional language tag. Handles ```lang\n...\n``` | ||
| # Content between fences; does not handle nested ``` inside body (documented edge case). | ||
| FENCED_BLOCK_RE = re.compile(r"```(\w*)\n(.*?)```", re.DOTALL) |
There was a problem hiding this comment.
Regex misses code fences with spaces after language tag
The regex r"```(\w*)\n(.*?)```" requires \n immediately after the (\w*) capture group. If a code block has trailing spaces between the language tag and the newline (e.g., ```python \n), the regex won't match because \w* stops at the space characters and then \n fails.
While uncommon, some editors/formatters do insert trailing spaces. A more robust pattern would be:
| FENCED_BLOCK_RE = re.compile(r"```(\w*)\n(.*?)```", re.DOTALL) | |
| FENCED_BLOCK_RE = re.compile(r"```(\w*)[ \t]*\n(.*?)```", re.DOTALL) |
| <Slider | ||
| min={field.min} | ||
| max={field.max} | ||
| step={field.step ?? 0.1} | ||
| marks={{ | ||
| [field.min]: "0%", | ||
| [(field.min + field.max) / 2]: "50%", | ||
| [field.max]: "100%", | ||
| }} | ||
| /> |
There was a problem hiding this comment.
Hardcoded percentage labels assume 0–1 range
The Slider marks are hardcoded as "0%", "50%", and "100%" regardless of the actual min/max values. While the current block_code_execution guardrail uses 0.0–1.0 (making "0%"/"100%" technically correct), any future percentage field with a different range (e.g., 0.0–0.5) would show misleading labels. Consider deriving labels from the actual min/max values:
| <Slider | |
| min={field.min} | |
| max={field.max} | |
| step={field.step ?? 0.1} | |
| marks={{ | |
| [field.min]: "0%", | |
| [(field.min + field.max) / 2]: "50%", | |
| [field.max]: "100%", | |
| }} | |
| /> | |
| <Slider | |
| min={field.min} | |
| max={field.max} | |
| step={field.step ?? 0.1} | |
| marks={{ | |
| [field.min]: `${Math.round(field.min * 100)}%`, | |
| [(field.min + field.max) / 2]: `${Math.round(((field.min + field.max) / 2) * 100)}%`, | |
| [field.max]: `${Math.round(field.max * 100)}%`, | |
| }} | |
| /> |
…uting code Adds a new block_code_execution guardrail that detects markdown fenced code blocks in request/response content and blocks or masks them by language. Includes full UI integration, type definitions, compliance test dataset, and 26 unit tests. Key guardrail capabilities: - Regex-based fenced code block detection with configurable blocked languages - Confidence scoring with tunable threshold - Execution-intent heuristics (request-side only) with conflict resolution - Block or mask actions for detected code - Support for pre_call, post_call, and during_call event hooks Security hardening: - Response-side blocking skips intent heuristics (LLM output doesn't contain user intent phrases, so checking would silently disable post_call blocking) - No-execution short-circuit includes conflict resolution: if both no-execution and execution phrases match, execution intent wins - Tightened overly broad phrases to prevent trivial bypass - _normalize_escaped_newlines only applies to pure-escaped payloads to avoid corrupting content that discusses escape sequences Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
91b3e76 to
3448640
Compare
…uting code (#22154) Adds a new block_code_execution guardrail that detects markdown fenced code blocks in request/response content and blocks or masks them by language. Includes full UI integration, type definitions, compliance test dataset, and 26 unit tests. Key guardrail capabilities: - Regex-based fenced code block detection with configurable blocked languages - Confidence scoring with tunable threshold - Execution-intent heuristics (request-side only) with conflict resolution - Block or mask actions for detected code - Support for pre_call, post_call, and during_call event hooks Security hardening: - Response-side blocking skips intent heuristics (LLM output doesn't contain user intent phrases, so checking would silently disable post_call blocking) - No-execution short-circuit includes conflict resolution: if both no-execution and execution phrases match, execution intent wins - Tightened overly broad phrases to prevent trivial bypass - _normalize_escaped_newlines only applies to pure-escaped payloads to avoid corrupting content that discusses escape sequences Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…uting code (BerriAI#22154) Adds a new block_code_execution guardrail that detects markdown fenced code blocks in request/response content and blocks or masks them by language. Includes full UI integration, type definitions, compliance test dataset, and 26 unit tests. Key guardrail capabilities: - Regex-based fenced code block detection with configurable blocked languages - Confidence scoring with tunable threshold - Execution-intent heuristics (request-side only) with conflict resolution - Block or mask actions for detected code - Support for pre_call, post_call, and during_call event hooks Security hardening: - Response-side blocking skips intent heuristics (LLM output doesn't contain user intent phrases, so checking would silently disable post_call blocking) - No-execution short-circuit includes conflict resolution: if both no-execution and execution phrases match, execution intent wins - Tightened overly broad phrases to prevent trivial bypass - _normalize_escaped_newlines only applies to pure-escaped payloads to avoid corrupting content that discusses escape sequences
Relevant issues
Closes #22056
Pre-Submission checklist
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
🆕 New Feature
Changes
Adds a new
block_code_executionguardrail that detects markdown fenced code blocks in request/response content and blocks or masks them based on language, confidence threshold, and execution-intent heuristics.Core Guardrail
Security Hardening
_normalize_escaped_newlinesonly applies to pure-escaped payloads (no real newlines present) to avoid corrupting content discussing escape sequencesUI Integration
Testing