fix(gateway): deliver media before queued follow-ups

[Qwinty]

What does this PR do?

Fixes a gateway delivery path where a completed agent response is sent before draining a queued follow-up message. That path previously called the first-response delivery from inside _run_agent, where it could either bypass normal response media post-processing or, in the helper-based path, reference an out-of-scope event variable.

The change adds a small helper that mirrors normal response delivery for this queued path: send only displayable text, then deliver extracted MEDIA: / local files via the existing post-stream media delivery helper. The call site now builds a synthetic MessageEvent from the current source before invoking the helper, so queued first-response delivery has the chat/thread target it needs without relying on a non-existent local event.

This prevents raw MEDIA:/path/to/file text from leaking to users, preserves attachments when a user sends a follow-up while the first turn is completing, and avoids warnings like:

Failed to send first response before queued message: name 'event' is not defined

Related Issue

No related upstream issue or PR found. Searched PRs and issues for MEDIA queued follow-up, final stream delivery, already_sent, MEDIA directive, and send_document MEDIA Telegram.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Security fix
• Documentation update
• Tests (adding or improving test coverage)
• Refactor (no behavior change)
• New skill (bundled or hub)

Changes Made

• gateway/run.py
  • Add _send_queued_first_response() and use it in the queued follow-up first-response branch instead of raw adapter.send().
  • Build a synthetic response MessageEvent from the current source at the queued call site so media delivery has chat/thread metadata and does not reference an undefined local event.
• tests/gateway/test_queued_followup_media_delivery.py
  • Cover mixed text plus MEDIA: and media-only queued first responses.

How to Test

1. Reproduce by completing a response that includes MEDIA:/path/to/file.html while a follow-up message is already queued.
2. Before this fix, the queued branch can send the raw MEDIA: text and skip native attachment delivery, or fail the helper call with name 'event' is not defined.
3. After this fix, the display text is sent without MEDIA:, send_document() receives the referenced file, and the queued first-response branch has a valid response event.

Commands run:

/usr/local/lib/hermes-agent/venv/bin/python -m pytest -q -o addopts='' tests/gateway/test_queued_followup_media_delivery.py tests/gateway/test_media_extraction.py tests/gateway/test_duplicate_reply_suppression.py
/usr/local/lib/hermes-agent/venv/bin/python -m ruff check gateway/run.py tests/gateway/test_queued_followup_media_delivery.py
git diff --check

Results:

39 passed, 1 warning
All checks passed!
git diff --check passed

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes
• I've tested on my platform: Ubuntu 24.04

Documentation & Housekeeping

• Documentation update N/A
• cli-config.yaml.example update N/A
• CONTRIBUTING.md or AGENTS.md update N/A
• Cross-platform impact considered; change is platform-neutral gateway response handling
• Tool descriptions/schemas update N/A

Screenshots / Logs

Observed live on Telegram: a queued first response containing text plus MEDIA:/root/pivin-scenario.html rendered the raw MEDIA: line instead of delivering the HTML attachment; a later media-only retry delivered the file.

A later live text-batch split also hit the same queued first-response branch and logged:

Queued follow-up ... final stream delivery not confirmed; sending first response before continuing.
Failed to send first response before queued message: name 'event' is not defined

The regression tests encode the media delivery failure mode without depending on Telegram network state; the call-site fix removes the undefined event reference from the queued branch.

[Qwinty]

Rebased this PR onto current upstream/main; it applied cleanly.

Local verification after rebase:

• python -m pytest -q -o 'addopts=' tests/gateway/test_queued_followup_media_delivery.py → 2 passed
• python -m ruff check gateway/run.py tests/gateway/test_queued_followup_media_delivery.py → passed

[teknium1]

Thanks for isolating the queued-delivery path; the current direct send at gateway/run.py:19845 does bypass the normal media pipeline, matching the report in #60845.

Problems

• gateway/run.py:12764 discards the images returned by adapter.extract_images(). The later _deliver_media_from_response() call also discards extracted images (gateway/run.py:13250), whereas the normal response path dispatches them with send_multiple_images() (gateway/platforms/base.py:4930, gateway/platforms/base.py:5055). A queued first response containing a remote markdown/HTML image would therefore lose its attachment.

Suggested changes

• Preserve and dispatch extracted image URLs with the queued response metadata, and add a remote-image regression test.
• Exercise the queued fallback call site in addition to the helper directly, so synthetic-event routing and ordering are covered.

This is an automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

Two PRs address the same queued-follow-up delivery defect: both replace the bare first-response send with clean-text delivery followed by native media handling, while #46223 additionally preserves remote extracted images and #51805 provides broader queued-call-site coverage.

Related pull requests

• #46223 related — (+173/-6) — keep open with a salvage path: The focused helper removes media directives from queued-response text, routes local files and remote images through the attachment pipeline, and constructs the routing event required by the resend branch. This aligns with the keep_open review on #46223 and already addresses its remote-image concern, but the diff still needs a regression test that exercises the queued fallback call site rather than only the helper.
• #51805 duplicate — (+426/-32) — close as duplicate of #46223 after salvaging its integration test: It fixes the same bare-send cause and uniquely exercises the full queued-follow-up path, but it makes a broader helper-signature refactor and does not deliver extracted remote markdown/HTML images. Despite the keep_open, high-salvageability maintainer-bot verdict on #51805, the diff of #46223 contains the narrower current implementation and remote-image handling; the valuable end-to-end test from #51805 should be ported to #46223.

Duplicates

#51805 and #46223 implement substantially the same clean-text-plus-media resend path; treat #51805 as a duplicate of #46223, with its full queued-follow-up regression test salvaged into #46223.

Suggested consolidation

Keep #46223 open with a salvage path: add the queued fallback call-site and ordering coverage demonstrated by #51805, while retaining #46223's remote-image dispatch and synthetic-event routing. Then close #51805 as duplicate of #46223; this differs from the maintainer-bot keep_open verdict on #51805 because the competing diff in #46223 covers the same root cause more narrowly and handles remote extracted images, while #51805's distinct value is its portable end-to-end regression test.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    subgraph Dup46223 ["PRs duplicating each other"]
        P46223["PR #46223 (open)"]
        P51805["PR #51805 (open)"]
    end
    class P46223 open
    class P51805 open
    class P46223 target
    click P46223 "https://github.com/NousResearch/hermes-agent/pull/46223"
    click P51805 "https://github.com/NousResearch/hermes-agent/pull/51805"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 30 kB of PR diffs, 6 kB of issue/PR text, 1 kB of discussion (2 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[will-lynas]

+1, also getting this problem on Slack and I'm running a local patch

[teknium1]

Closing as superseded: #82162 (salvage of #71031) landed on main and fixes queued follow-up MEDIA delivery for both branches — the non-streamed fallback that leaked literal MEDIA: text (this PR's target) and the streamed branch that silently dropped attachments — while routing through the existing extract_media + _deliver_media_from_response machinery so image batching, voice routing, path-security filtering, and the explicit-only policy are preserved. Thanks for attacking this bug; your report and approach helped establish the fix's requirements.