Skip to content

fix(gateway): restrict auto-appended media to producer tools - #16721

Closed
VinciZhu wants to merge 1 commit into
NousResearch:mainfrom
VinciZhu:fix/gateway-media-auto-append-tool-allowlist
Closed

fix(gateway): restrict auto-appended media to producer tools#16721
VinciZhu wants to merge 1 commit into
NousResearch:mainfrom
VinciZhu:fix/gateway-media-auto-append-tool-allowlist

Conversation

@VinciZhu

@VinciZhu VinciZhu commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR prevents the gateway from auto-appending MEDIA: tags found in arbitrary tool results.

Previously, when the final response did not contain a media tag, the gateway scanned all current-turn tool/function output for MEDIA: strings and appended any matches to the final gateway reply. That is too broad: ordinary tools can return documentation, skill content, logs, search results, or user-authored text containing literal media examples.

For example, a docs/skill tool can legitimately return:

MEDIA:/absolute/path/to/file.png

That string is only documentation, but the old broad scanner treats it as a real attachment and may try to send a nonexistent file on any gateway platform.

The fix keeps the behavior intentionally narrow: automatic append only considers tool outputs from known media-producing tools (text_to_speech and the legacy text_to_speech_tool name). Future media-producing tools can opt in deliberately by adding their tool name to this list or by introducing a registry-level capability flag.

Fixes #16720

Changes

  • Adds a small gateway helper for collecting auto-appended media tags.
  • Maps tool results back to the assistant tool call that produced them.
  • Stops scanning arbitrary tool/function output for media tags.
  • Keeps existing auto-append behavior for real TTS media outputs.
  • Adds regression coverage for documentation/skill examples and real TTS media artifacts.

Test plan

  • Added regression coverage that skill_view output containing a MEDIA: example is not auto-appended.
  • Added positive coverage that text_to_speech media output is still auto-appended, including the voice directive.
  • Tested on macOS Darwin 25.3.0 arm64 with Hermes v0.11.0 local checkout.
venv/bin/python -m pytest tests/gateway/test_media_extraction.py -q

Result:

6 passed in 2.24s

Follow-up note

This PR intentionally avoids changing tool registration APIs. If maintainers prefer a more extensible design, the allowlist can be replaced in a follow-up with a registry-level capability flag such as auto_append_media=True on media-producing tools.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery labels Apr 27, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Thanks @VinciZhu — salvaged into a batch PR alongside another related MEDIA: false-positive fix: #36275.

Your producer-tool allowlist commit was cherry-picked with authorship preserved. Since gateway/run.py had moved on a fair bit since this was opened (it gained the #34608 current-turn slice isolation and the #160 compression-safe fallback), the cherry-pick was reconciled to keep both of those and layer your allowlist on top as the primary guard, plus use the extension-anchored matcher already on main. Net behavior is strictly tighter with no regression. Also added you to the release AUTHOR_MAP so you're credited in the notes.

Closing in favor of #36275.

kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request Jun 1, 2026
kossteg pushed a commit to kossteg/hermes-agent that referenced this pull request Jun 16, 2026
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
xyshanren pushed a commit to xyshanren/hermes-agent-cn that referenced this pull request Jun 25, 2026
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
donbowman pushed a commit to donbowman/hermes-agent that referenced this pull request Jul 13, 2026
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Gateway auto-appends MEDIA tags from arbitrary tool results

3 participants