Skip to content

Prompt injection mitigation: update pattern-based detection - #9198

Merged
dorien-koelemeijer merged 9 commits into
mainfrom
fix/reduce-pattern-matching-sensitivity-prompt-injection
May 14, 2026
Merged

Prompt injection mitigation: update pattern-based detection#9198
dorien-koelemeijer merged 9 commits into
mainfrom
fix/reduce-pattern-matching-sensitivity-prompt-injection

Conversation

@dorien-koelemeijer

@dorien-koelemeijer dorien-koelemeijer commented May 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

This PR aims to decrease false positives in pattern-based detection, especially in rm -rf commands. Whenever the command injection classifier is not available (i.e. not enabled in settings or WARP disabled), there is an automatic fallback to pattern-based detection. The command injection classifier has reasonably high accuracy, but the pattern-based detection is harder to get right.

Changes

  • Removed rm_rf_system: Matched any rm -rf targeting system directory names (bin, etc, usr, var, tmp, opt, etc.). In practice this fired on build cleanup (rm -rf build), temp file deletion (rm -f /tmp/file.json), and Homebrew management (rm -rf /opt/homebrew/...). The rm_rf_root_bare pattern still catches the truly catastrophic rm -rf / and rm -rf /*.
  • Removed alternative_shell_invocation: Matched any bash -c or sh -c containing ;, |, or &. This is normal shell usage (build scripts, CI commands, conditional checks). The ML classifier distinguishes actual injection from legitimate shell invocations.
  • Simplified log_manipulation: Removed > /dev/null from the pattern. Stderr suppression (2>/dev/null) is standard in virtually every shell command. Kept the actual log tampering patterns: truncate.log, rm /var/log/, echo > /var/log. Also fixed the rm flag regex from (-[rf]\s+) to (-[rRfF]+\s+)* to correctly match rm -rf /var/log/... (aligned with rm_rf_root_bare).

Estimated impact

Eliminates ~257 of ~310 weekly pattern-based flags (~83%) while all true positives remain covered by:

  • The ML classifier (primary path)
  • rm_rf_root_bare for rm -rf /
  • curl_bash_execution for curl | bash
    All other 20+ patterns unchanged (reverse shells, SUID, credential access, data exfil, etc.)

Testing

Unit tests.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 13, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26791afd09

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/security/patterns.rs Outdated
Comment thread crates/goose/src/security/patterns.rs Outdated
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 14, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88ed696f2d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/security/patterns.rs Outdated
Comment thread crates/goose/src/security/patterns.rs Outdated
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 14, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa5c9ab851

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/security/patterns.rs Outdated
Comment thread crates/goose/src/security/patterns.rs Outdated
@lifeizhou-ap

lifeizhou-ap commented May 14, 2026

Copy link
Copy Markdown
Collaborator

Hi @dorien-koelemeijer,

Looks good! Would you please address the above codex comments before merge the PR?

Thanks!

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 14, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccc6ff5173

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/security/patterns.rs Outdated
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 14, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c01c94b65

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/security/patterns.rs
Comment thread crates/goose/src/security/patterns.rs
@dorien-koelemeijer
dorien-koelemeijer added this pull request to the merge queue May 14, 2026
Merged via the queue into main with commit 826cce0 May 14, 2026
23 checks passed
@dorien-koelemeijer
dorien-koelemeijer deleted the fix/reduce-pattern-matching-sensitivity-prompt-injection branch May 14, 2026 02:39
lifeizhou-ap added a commit that referenced this pull request May 15, 2026
@lifeizhou-ap
Merge branch 'main' into lifei/ui-acp-research
473ebcb 
* main: (102 commits)
  Dynamically refresh skill instructions each turn (#9217)
  Build non-vulkan linux variants using ubuntu 22.04 (#9211)
  fix(ui): show tool name in approval prompt (#9216)
  feat: add Atomic Chat as declarative OpenAI-compatible provider (#9210)
  chore: bump package.json versions from 0.19.1 to 0.20.0 (#9218)
  feat: support GOOSE_OAUTH_CALLBACK_PORT for stable OAuth redirect_uri (#9209)
  [RFC] feat(oauth): proactive token refresh to avoid re-auth on every session (#8386)
  fix: resolve Azure CLI on Windows by using az.cmd (#9215)
  fix: handle non-interactive terminal in goose configure on Windows (#9214)
  Better parsing of pasted html as markdown so agents understand (#9190)
  fix: persist accumulated cost in session DB to survive reload (#9191)
  fix(publish-npm): build binary from current SHA + add compat check (#9212)
  feat(desktop): add goose://new-session deep link to open fresh chat (#9196)
  Add PR previews using cloudflare pages (#9208)
  fix: prevent tool-use marker leakage in toolshim output (#8310)
  Prompt injection mitigation: update pattern-based detection (#9198)
  remove goose2 related skills (#9189)
  Switch GH pages deploy to actions/artifact workflow (#9025)
  fix(summon): re-apply canonical limits when delegate overrides model (#9183)
  Split code signing from build (#8587)
  ...
@github-actions github-actions Bot mentioned this pull request May 20, 2026
Closed
shafqatevo pushed a commit to shafqatevo/goose that referenced this pull request Aug 7, 2026
@dorien-koelemeijer
Prompt injection mitigation: update pattern-based detection (aaif-goo…
3b20557 
…se#9198)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@michaelneale michaelneale michaelneale approved these changes
@shellz-n-stuff shellz-n-stuff shellz-n-stuff approved these changes
@lifeizhou-ap lifeizhou-ap lifeizhou-ap approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@dorien-koelemeijer @lifeizhou-ap @michaelneale @shellz-n-stuff