diff --git a/documentation/docs/guides/config-file.md b/documentation/docs/guides/config-file.md index b5789135f8de..fb8f953a320d 100644 --- a/documentation/docs/guides/config-file.md +++ b/documentation/docs/guides/config-file.md @@ -36,6 +36,8 @@ The following settings can be configured at the root level of your config.yaml f | `GOOSE_ALLOWLIST` | URL for allowed extensions | Valid URL | None | No | | `GOOSE_RECIPE_GITHUB_REPO` | GitHub repository for recipes | Format: "org/repo" | None | No | | `GOOSE_AUTO_COMPACT_THRESHOLD` | Set the percentage threshold at which Goose [automatically summarizes your session](/docs/guides/sessions/smart-context-management#automatic-compaction). | Float between 0.0 and 1.0 (disabled at 0.0)| 0.8 | No | +| `security_prompt_enabled` | Enable [prompt injection detection](/docs/guides/security/prompt-injection-detection) to identify potentially harmful commands | true/false | false | No | +| `security_prompt_threshold` | Sensitivity threshold for [prompt injection detection](/docs/guides/security/prompt-injection-detection) (higher = stricter) | Float between 0.01 and 1.0 | 0.7 | No | :::info Automatic Multi-Model Configuration The experimental [AutoPilot](/docs/guides/multi-model/autopilot) feature provides intelligent, context-aware model switching. Configure models for different roles using the `x-advanced-models` setting. @@ -58,7 +60,7 @@ Here's a basic example of a config.yaml file: ```yaml # Model Configuration GOOSE_PROVIDER: "anthropic" -GOOSE_MODEL: "claude-3.5-sonnet" +GOOSE_MODEL: "claude-4.5-sonnet" GOOSE_TEMPERATURE: 0.7 # Planning Configuration @@ -73,8 +75,8 @@ GOOSE_CLI_MIN_PRIORITY: 0.2 # Recipe Configuration GOOSE_RECIPE_GITHUB_REPO: "block/goose-recipes" -# Experimental Features -ALPHA_FEATURES: true +# Security Configuration +security_prompt_enabled: true # Extensions Configuration extensions: diff --git a/documentation/docs/guides/logs.md b/documentation/docs/guides/logs.md index 3521af3121b9..013a6ddf455d 100644 --- a/documentation/docs/guides/logs.md +++ b/documentation/docs/guides/logs.md @@ -67,6 +67,10 @@ This log contains general application-level logging including: * Token usage statistics as well as token counts (input, output, total) * LLM information (model names, versions) +When [prompt injection detection](/docs/guides/security/prompt-injection-detection) is enabled, logs also include: +* Security findings with unique IDs (format: `SEC-{uuid}`) +* User decisions (allow/deny) associated with finding IDs + ### Desktop Application Log diff --git a/documentation/docs/guides/security/_category_.json b/documentation/docs/guides/security/_category_.json new file mode 100644 index 000000000000..7f2dcf2767a6 --- /dev/null +++ b/documentation/docs/guides/security/_category_.json @@ -0,0 +1,8 @@ +{ + "label": "Security", + "position": 43, + "link": { + "type": "doc", + "id": "guides/security/index" + } +} diff --git a/documentation/docs/guides/security/index.mdx b/documentation/docs/guides/security/index.mdx new file mode 100644 index 000000000000..7b02686179ac --- /dev/null +++ b/documentation/docs/guides/security/index.mdx @@ -0,0 +1,40 @@ +--- +title: Staying Safe with goose +hide_title: true +description: Keep your goose workflows safe with built-in security features and best practices +--- + +import Card from '@site/src/components/Card'; +import styles from '@site/src/components/Card/styles.module.css'; + +

Staying Safe with goose

+

+ Understanding security risks and following best practices can help you secure your workflows with goose. +

+ +
+

📚 Documentation & Guides

+
+ +
+
+ +
+

📝 Featured Blog Posts

+
+ + +
+
diff --git a/documentation/docs/guides/security/prompt-injection-detection.md b/documentation/docs/guides/security/prompt-injection-detection.md new file mode 100644 index 000000000000..38a2ca170522 --- /dev/null +++ b/documentation/docs/guides/security/prompt-injection-detection.md @@ -0,0 +1,102 @@ +--- +title: Prompt Injection Detection +sidebar_label: Prompt Injection Detection +description: Protect your workflow by detecting potentially harmful commands before they run. +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import { PanelLeft, Settings } from 'lucide-react'; + +Prompt injection happens when malicious instructions are hidden inside executable content. In the world of AI, prompt injection can be used to nudge AI agents (like goose) to run unsafe commands that compromise your environment or data. + +You can help protect your goose workflows by enabling prompt injection detection. This feature uses pattern matching to detect common attack techniques, including: +- Attempts to delete system files or directories +- Commands that download and execute remote scripts +- Attempts to access or exfiltrate sensitive data like SSH keys +- System modifications that could compromise security + +:::important +These checks provide a safeguard, not a guarantee. They detect known patterns but cannot catch all possible threats, especially novel or sophisticated attacks. +::: + +## How Detection Works + +When enabled, goose scans tool calls for risky patterns before they run: + +1. **Tool call is intercepted and analyzed** - When goose prepares to execute a tool, the security system extracts the tool parameter text and checks it against [threat patterns](https://github.com/block/goose/blob/main/crates/goose/src/security/patterns.rs) +2. **Risk is assessed** - Detected threats are assigned confidence scores +3. **Execution pauses** - Threats that exceed your configured threshold need your decision +4. **Security alert appears** - The alert displays the confidence level, a description of the finding, and a unique finding ID. For example: + ``` + 🔒 Security Alert: This tool call has been flagged as potentially dangerous. + + Confidence: 95% + Explanation: Detected 1 security threat: Recursive file deletion with rm -rf + Finding ID: SEC-abc123... + + [Allow Once] [Deny] + ``` +5. **You choose** whether to proceed or cancel after reviewing the alert details. Note that: + - Each decision is logged with its finding ID in the [goose system logs](/docs/guides/logs#system-logs) + - Allowed commands still run with your full permissions + +**Responding to Alerts:** + +- Read the explanation to understand what triggered the detection +- Consider your context—does this match what you're trying to do? +- Try rephrasing your request more specifically +- Check the source and be extra cautious with prompts from unknown sources + +When in doubt, deny. + +## Enabling Detection + + + + + 1. Click the button in the top-left to open the sidebar + 2. Click `Settings` on the sidebar + 3. Click the `Chat` tab + 4. Toggle `Enable Prompt Injection Detection` to the on setting + 5. Optionally adjust the `Detection Threshold` to [configure the sensitivity](#configuring-detection-threshold) + + + + + Add these settings to your [`config.yaml`](/docs/guides/config-file): + + ```yaml + security_prompt_enabled: true + security_prompt_threshold: 0.7 # Optional, default is 0.7 + ``` + + + + +:::info Other Security Features +Beyond prompt injection detection, goose automatically: +- Warns you before running new or updated recipes +- Warns you when importing recipes that contain invisible Unicode Tag Block characters +- [Checks for known malware](/docs/troubleshooting#malicious-package-detected) when installing extensions for locally-run MCP servers +::: + +### Configuring Detection Threshold + +The threshold (0.01-1.0) controls how strict detection is: + +| Threshold | Sensitivity | Use When | +|-----------|------------|----------| +| **0.01-0.50** | Very lenient | You're experienced and understand the risks | +| **0.50-0.70** | Balanced | General development work (good default) | +| **0.70-0.90** | Strict | Working with sensitive data or systems | +| **0.90-1.00** | Maximum | High-security environments | + +When the injection prompt detection feature is enabled, the default threshold is 0.7 (recommended for most users). + +Lower thresholds mean fewer alerts but might miss threats. Higher thresholds catch more potential issues but may flag legitimate operations. You can control this sensitivity/convenience tradeoff based on your needs. + +## See Also + +- [goose Permission Modes](/docs/guides/goose-permissions) - Control goose's autonomy level +- [Managing Tool Permissions](/docs/guides/managing-tools/tool-permissions) - Fine-grained tool control