Skip to content

fix(gateway): skip MEDIA: tags inside code blocks and blockquotes - #35699

Closed
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/extract_media_code_block_false_positives
Closed

fix(gateway): skip MEDIA: tags inside code blocks and blockquotes#35699
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:fix/extract_media_code_block_false_positives

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes extract_media() false positives where MEDIA:<path> tags inside fenced code blocks, inline code spans, and blockquotes were incorrectly extracted as real media delivery directives. This caused example paths in agent responses to be stripped from user-visible text and potentially delivered as unintended files.

Related Issue

Fixes #35695

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • gateway/platforms/base.py: Added _mask_protected_spans() helper that replaces content inside fenced code blocks, inline code spans, and blockquotes with equal-length whitespace before regex matching. Modified extract_media() to use masked content for both finditer() (extraction) and sub() (cleanup). The helper skips masking backtick-quoted paths in MEDIA: tags (e.g. MEDIA:`/path/to/file.png`) to preserve existing path extraction behavior.
  • tests/gateway/test_platform_base.py: Added 5 regression tests covering MEDIA tags in fenced code blocks, inline code, blockquotes, mixed code+prose, and real tags outside protected regions.

How to Test

  1. Run pytest tests/gateway/test_platform_base.py -k "TestExtractMedia" -v — all 24 tests should pass (19 existing + 5 new)
  2. Verify the false-positive scenario: agent response containing To send an image, include \MEDIA:/path/to/image.jpg`should NOT extract/path/to/image.jpg` as a media file
  3. Verify real MEDIA tags still work: MEDIA:/tmp/report.pdf in normal prose should still be extracted
  4. Run pytest tests/gateway/test_send_image_file.py tests/gateway/test_media_extraction.py -v — all related tests should pass

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Code Intelligence

  • Analyzed: BasePlatformAdapter.extract_media (callers: ~10 across gateway dispatch paths)
  • Analyzed: BasePlatformAdapter._mask_protected_spans (new, called from extract_media only)
  • Blast radius: LOW — additive helper, existing extraction paths unchanged
  • Related patterns: extract_local_files already uses code-span detection (lines 2753-2761) for the same purpose; this fix applies the same concept to extract_media

extract_media() scanned the full response text without distinguishing
live delivery tags from example paths in fenced code blocks, inline code
spans, and blockquotes. This caused false positives where the agent's
explanation of MEDIA: syntax (or tool output containing example paths)
was stripped from user-visible text and the path was added to the media
delivery list.

Added _mask_protected_spans() helper that replaces protected regions
with equal-length whitespace before regex matching, preserving match
offsets. The helper skips backtick-quoted paths in MEDIA: tags to
maintain existing path extraction behavior.

Fixes NousResearch#35695
@daimon-nous daimon-nous Bot added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels May 31, 2026
@daimon-nous

daimon-nous Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

Duplicate of #16585 — both fix the same extract_media() false-positive bug on MEDIA: tags inside code blocks. This PR also handles blockquotes and adds tests; #16585 also covers stream_consumer.py. See also #16434 and #35695.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Overview

Well-crafted fix for extract_media() false positives where MEDIA:<path> tags inside fenced code blocks, inline code spans, and blockquotes were incorrectly extracted as real media delivery directives.

Looks Good

  • _mask_protected_spans correctly skips MEDIA: path quotes (e.g. MEDIA:`/path/to/file.png` )
  • Preserves character count for offset-valid regex matching
  • 5 regression tests covering code blocks, inline code, blockquotes, mixed content, and real tag preservation
  • Additive — no changes to existing extraction paths
  • Low blast radius; mirrors existing extract_local_files pattern

Reviewed by Hermes Agent

@liuhao1024

Copy link
Copy Markdown
Contributor Author

@daimon-nous[bot] Acknowledged. This PR passes all CI checks, while the competing PR #16585 has a test failure. If maintainers prefer the duplicate flag to take precedence, feel free to close this. Otherwise, I believe this PR is ready for merge.

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Thanks @liuhao1024 — salvaged into a batch PR alongside the related auto-append guard from #16721: #36275.

Your code-block / inline-code / blockquote masking commit was cherry-picked with authorship preserved; it applied cleanly to current main. Verified together with the other fix — 139 passed, 2 skipped across the gateway media tests, including your new cases.

Closing in favor of #36275.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: extract_media() false-positives on example paths in quoted text / code blocks

3 participants