Skip to content

Conversation

@dorien-koelemeijer
Copy link
Collaborator

@dorien-koelemeijer dorien-koelemeijer commented Nov 12, 2025

Summary

There were errors exporting metrics to Datadog, although local testing didn't cause any issues which makes it harder to troubleshoot. Suspicion is that the issues may be due to formatted string messages with {} placeholders, gauge.goose.* as attribute keys, newlines in explanation strings, or emojis in message strings.

Part of error log (not super descriptive):

Field | Value
-- | --
attributes.message | Failed to export metrics
attributes.reason | Metrics exporter otlp failed with http request failed with error sending request for url (xxx)

Type of Change

  • Feature
  • Bug fix
  • Refactor / Code quality
  • Performance improvement
  • Documentation
  • Tests
  • Security fix
  • Build / Release
  • Other (specify below)

AI Assistance

No AI assistance on this PR

Testing

Manual testing - I set up a local Docker OTLP collector to capture and inspect the raw metric payloads from goose, which allowed me to verify that the prompt injection metrics were emitted correctly (along with looking at the already existing metrics).

Screenshot 2025-11-12 at 4 21 41 pm

Copilot AI review requested due to automatic review settings November 12, 2025 06:19
@dorien-koelemeijer dorien-koelemeijer changed the title update prompt injection metrics due to errors exporting to datadog Update prompt injection detection metrics (due to errors exporting to datadog) Nov 12, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR attempts to fix Datadog metrics export errors by removing problematic elements from log messages: format string placeholders with {}, gauge.goose.* metric keys, newlines in explanation strings, and emojis.

Key changes:

  • Converted gauge.goose.* metrics to counter.goose.*
  • Sanitized explanation strings to replace newlines with " | " for metrics
  • Simplified log messages to remove format placeholders in some locations

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
crates/goose/src/security/mod.rs Changed gauge metrics to counters, added explanation sanitization for metrics, and partially simplified log messages (but emojis and some format placeholders remain)
crates/goose/src/agents/tool_execution.rs Removed format placeholders from user decision logging, converted to structured logging with fields
Comments suppressed due to low confidence (1)

crates/goose/src/security/mod.rs:102

  • The log message still uses a format placeholder with "{}" on line 97 and contains emojis (🔒) on lines 99 and 101. According to the PR description, both format placeholders and emojis may be causing Datadog export errors and should be removed.
                        "{}",
                        if above_threshold {
                            "🔒 Current tool call flagged as malicious after security analysis (above threshold)"
                        } else {
                            "🔒 Security finding below threshold - logged but not blocking execution"
                        }

@dorien-koelemeijer dorien-koelemeijer merged commit 387d76c into main Nov 13, 2025
15 checks passed
tlongwell-block added a commit that referenced this pull request Nov 14, 2025
* origin/main: (51 commits)
  Compaction resiliency improvements (#5618)
  docs: ask goose button (#5730)
  Update prompt injection detection metrics (due to errors exporting to datadog) (#5692)
  Spence/icon2 tokyo drift (#5728)
  docs: logs rotation and misc updates (#5727)
  docs: automatic ollama model detection (#5725)
  Fix context progress bar not resetting after /clear command (#5652)
  docs: removing double announcements (#5714)
  docs: mcp sampling support (#5708)
  hackathon banner  (#5710)
  Fix documentation-only change detection for push events (#5712)
  Added transaction commits to multi sql functions in session_manager (#5693)
  fix: improve and simplify tool call chain rendering (#5704)
  Fix: Always show autocompact threshold ui (#5701)
  chore: Update governance to include Discord (#5690)
  Ollama improvements (#5609)
  feat: add Supabase MCP server to registry (#5629)
  Unlist VS Code extension tutorials from MCP and experimental sections (#5677)
  fix: make image processing work in github copilot provider (#5687)
  fix: do not take into account gitignore in developer mcp (#5688)
  ...
wpfleger96 added a commit that referenced this pull request Nov 14, 2025
* main: (65 commits)
  Fix: Recipes respect the quiet flag (#5743)
  docs: update cli commands (#5744)
  Run smoke tests on a free runner (#5745)
  faster, cheaper (pick two): improve CI workflow and switch to free github runner (#5702)
  Compaction resiliency improvements (#5618)
  docs: ask goose button (#5730)
  Update prompt injection detection metrics (due to errors exporting to datadog) (#5692)
  Spence/icon2 tokyo drift (#5728)
  docs: logs rotation and misc updates (#5727)
  docs: automatic ollama model detection (#5725)
  Fix context progress bar not resetting after /clear command (#5652)
  docs: removing double announcements (#5714)
  docs: mcp sampling support (#5708)
  hackathon banner  (#5710)
  Fix documentation-only change detection for push events (#5712)
  Added transaction commits to multi sql functions in session_manager (#5693)
  fix: improve and simplify tool call chain rendering (#5704)
  Fix: Always show autocompact threshold ui (#5701)
  chore: Update governance to include Discord (#5690)
  Ollama improvements (#5609)
  ...
michaelneale added a commit that referenced this pull request Nov 17, 2025
* main:
  scan recipe for security when saving recipe (#5747)
  feat: trying grok for live test (#5732)
  Platform Extension MOIM (Minus One Info Message) (#5027)
  docs: remove hackathon banner (#5756)
  Fix: Recipes respect the quiet flag (#5743)
  docs: update cli commands (#5744)
  Run smoke tests on a free runner (#5745)
  faster, cheaper (pick two): improve CI workflow and switch to free github runner (#5702)
  Compaction resiliency improvements (#5618)
  docs: ask goose button (#5730)
  Update prompt injection detection metrics (due to errors exporting to datadog) (#5692)
  Spence/icon2 tokyo drift (#5728)
  docs: logs rotation and misc updates (#5727)
  docs: automatic ollama model detection (#5725)
  Fix context progress bar not resetting after /clear command (#5652)
  docs: removing double announcements (#5714)
  docs: mcp sampling support (#5708)
Surendhar-N-D pushed a commit to Surendhar-N-D/goose that referenced this pull request Nov 17, 2025
arul-cc pushed a commit to arul-cc/goose that referenced this pull request Nov 17, 2025
BlairAllan pushed a commit to BlairAllan/goose that referenced this pull request Nov 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants