Skip to content

fix(gateway): preserve queued follow-up media delivery - #25119

Closed
vKongv wants to merge 2 commits into
NousResearch:mainfrom
pebble-tech:fix/queued-followup-media-delivery-upstream
Closed

vKongv wants to merge 2 commits into
NousResearch:mainfrom
pebble-tech:fix/queued-followup-media-delivery-upstream

Conversation

@vKongv

@vKongv vKongv commented May 13, 2026 •

Copy link
Copy Markdown

What does this PR do?

When a second inbound message queues behind an in-progress gateway turn, the queued resend path previously replayed only the text body. MEDIA-backed attachments from send_asset could be dropped silently. This routes queued first-response resends through a text-plus-media delivery helper instead of a plain adapter text send.

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • gateway/run.py — route queued first-response resends through _deliver_queued_first_response; preserve MEDIA attachments the queued path can replay natively
  • tests/gateway/test_tts_media_routing.py — regression test for queued follow-up media delivery

How to Test

  1. uv run pytest -o addopts= tests/gateway/test_tts_media_routing.py -k queued -q
  2. Trigger a gateway turn that returns a response with MEDIA: attachments, then queue a second inbound before delivery completes — confirm images are delivered on the queued resend path

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

N/A — delete this section otherwise.

Screenshots / Logs

N/A

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists labels May 13, 2026
@alt-glitch

Copy link
Copy Markdown

Likely duplicate of #18546 — same bug: gateway queue drain loop skips extract_media() for queued follow-up responses, dropping MEDIA file delivery. #19011 was already flagged as a duplicate of #18546 for the same issue.

@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from b327673 to 9655de3 Compare May 13, 2026 18:34
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 9655de3 to 0d0a378 Compare May 24, 2026 08:27
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 0d0a378 to cff4d66 Compare May 24, 2026 08:37
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch 2 times, most recently from 8a85668 to 8bfe8dc Compare May 25, 2026 07:18
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 8bfe8dc to 9030100 Compare May 26, 2026 06:50
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 9030100 to 727c310 Compare May 27, 2026 07:07
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 727c310 to 4a4cf20 Compare May 28, 2026 04:14
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 4a4cf20 to 570aa36 Compare May 28, 2026 06:58
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from ea665aa to e69b5e3 Compare June 3, 2026 08:25
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from e69b5e3 to d9803ab Compare June 4, 2026 07:58
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from d9803ab to 4c800dd Compare June 5, 2026 07:15
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 4c800dd to b53f79d Compare June 6, 2026 06:35
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from b53f79d to 1165cea Compare June 7, 2026 10:43
@cursor
cursor Bot force-pushed the fix/queued-followup-media-delivery-upstream branch from 1165cea to 5d152ef Compare June 9, 2026 06:59
@vKongv
vKongv force-pushed the fix/queued-followup-media-delivery-upstream branch from 5d152ef to 0a0a27a Compare June 10, 2026 07:15

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for narrowing this to the queued first-response resend path. The defect is present on current main: gateway/run.py:19724-19728 directly sends first_response, while native attachment routing lives in _deliver_media_from_response() at gateway/run.py:13097-13216.

Problems

  • The added broad cleanup at gateway/run.py:2188 strips every remaining MEDIA: token after extract_media(). That conflicts with the base contract: protected code/inline-code spans and unsupported or unvalidated tags are deliberately preserved (gateway/platforms/base.py:3623-3675, 1460-1481).
  • The new tests call the helper directly, rather than taking the queued resend branch at gateway/run.py:19718-19730; they do not verify the actual integration point.

Suggested changes

  • Drop the broad regex and retain the base extractor's cleaned text semantics.
  • Add a queued-follow-up integration test, including attachment routing and a protected MEDIA: example.

Automated hermes-sweeper review.

Comment thread gateway/run.py
@vKongv

vKongv commented Jul 22, 2026

Copy link
Copy Markdown
Author

Thanks for the heads-up on #18546 / #19011 — same gap on the queued first-response resend path.

This PR keeps that path, but routes delivery through the existing text + media helper so attachments still get the normal allowlist / type-aware routing instead of a one-off extract-and-send loop. Also dropped the broad leftover MEDIA: strip after extract_media() (it was wiping protected code/inline examples), and added a regression that hits the real queued resend branch in _run_agent.

Rebased on current main. Happy to close/consolidate if maintainers want a single landing PR — otherwise this should be ready for another look.

@alt-glitch

Copy link
Copy Markdown

This was generated by AI during triage.

Current-head correction: #25119 predates the other open queued-media PRs. It shares the queued first-response delivery outcome but uses a distinct helper shape, so this is a competing implementation rather than a PR-to-issue duplicate. Related: #18546, #46223, and #51805.

@SemonCat

SemonCat commented Aug 3, 2026

Copy link
Copy Markdown

Opened a small incremental follow-up directly on top of this branch: pebble-tech#56.

It covers the one remaining gap in the queued-follow-up flush: the "already streamed" branch (where streaming already delivered the first response's text) never ran the MEDIA: upload pass, so an attachment carried by that same response was silently dropped — streaming only ever delivers text. The fix reuses _deliver_media_from_response (the same helper _deliver_queued_first_response already calls for its media half) via a synthetic event; no new helper, no change to the direct-send branch this PR already fixes.

Also closing #77732 — a duplicate that a triage pass flagged as part of this same family — in favor of consolidating that one missing piece here rather than competing with your work.

vKongv added 2 commits August 8, 2026 04:15
Ensure queued follow-up resends keep MEDIA-backed attachments by replaying the
first response through the gateway's text-plus-media delivery flow instead of a
plain adapter text send.
Drop the broad MEDIA: regex after extract_media so code/inline examples
survive, and cover the real queued first-response resend path in tests.
@teknium1

teknium1 commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Closing as superseded — with credit: #82162 (salvage of #71031) landed on main and fixes queued follow-up MEDIA delivery for both branches. Your #25119 commits were consolidated into that PR branch with attribution preserved before the salvage, so your work is part of what shipped. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants