fix(slack): make thread images/files visible to the agent - #70188
Merged
Conversation
…context Images and files posted in a Slack thread before the bot joins were invisible to the agent: _fetch_thread_context renders text only, and a caption-less image post was dropped from context entirely (empty text → skip). "@bot what do you think of the chart above?" read as a question about nothing. _render_message_text now appends a compact, sanitized marker per file attachment — [image: chart.png], [video: demo.mp4], [audio: note.m4a], [file: report.pdf (application/pdf)] — so the agent can SEE that prior thread messages carried attachments and ask for a re-share when it needs the bytes. Filenames are stripped of newlines/brackets so a hostile name can't fake context structure. Because both thread-context formatting and parent-text rendering go through _render_message_text, markers appear on the cold-start hydrate, the explicit-mention delta refresh, restart rehydration, and reply_to_text. Reapplied from #32315 onto the current adapter (original patched the pre-plugin gateway/platforms/slack.py, moved in the plugin migration; annotation labels reworked to per-file typed markers, download side handled separately).
When the bot is mentioned mid-thread for the first time, the thread root
is very often the artifact the mention is about ("@bot what's in this
chart?" posted as a reply under an image) — but the root's image never
reached the agent, so it answered blind.
On the cold-start hydrate path (and only there), _collect_thread_root_images
reads the root message from the thread-context cache the immediately
preceding _fetch_thread_context call just populated (zero extra Slack API
calls in the normal case), downloads its image/* attachments through the
existing authenticated _download_slack_file helper, and delivers them as
media_urls/media_types on the same MessageEvent — upgrading the message
type to PHOTO so vision routing engages.
Scope and safety:
- One-time delivery by construction: the cold-start path is guarded by
_has_active_session_for_thread, so later turns in the same session can
never re-download or re-deliver. No new gateway/session plumbing needed.
- Bounded by _THREAD_ROOT_IMAGE_MAX (4); non-image root attachments stay
text-only markers.
- Slack Connect stubs (file_access=check_file_info) resolve via files.info.
- Best-effort: a failed download degrades to the [image: ...] marker from
the thread context — never an error turn.
- Also hardens the video mimetype fallback (mimetype can be empty) so
media_types entries are always non-None strings.
Adapted from #69185 by @KCAYAAI — the original plumbed MessageEvent media
through gateway/base.py, run.py and session.py with durable one-time
delivery markers (2,441 lines); this lands the user-visible behavior
adapter-locally by reusing the session guard already on the hydrate path.
…lity Covers cluster C1-images (#69185, #32315, #66136): - _slack_file_marker unit tests: typed markers per mimetype family, hostile-filename sanitization (newlines/brackets can't fake context structure). - _render_message_text appends markers; a caption-less image post no longer vanishes from thread context. - Cold-start hydrate integration: prior-message images surface as markers in channel_context; the thread root's image is downloaded, delivered as media_urls, and upgrades message_type to PHOTO. - Failure path: root-image download failure degrades to the marker, never blocks the turn. - Bounds: root delivery capped at _THREAD_ROOT_IMAGE_MAX; non-image root attachments stay marker-only (no download). - One-time delivery: active thread session skips the hydrate → no re-download/re-delivery on later turns. - Composition: the trigger's own event files still ride alongside a delivered root image; Slack Connect stubs resolve via files.info; the collector never issues its own conversations.replies call. - Delta refresh (#23918 path): images in new replies past the watermark surface as markers, with no root re-download. A/B: 14 of 15 tests fail with the adapter fix reverted, all pass with it applied.
…almarket Contributor-email mapping for the #32315 salvage (thread image/file markers reapplied onto the plugin adapter).
Contributor
૮ >ﻌ< ა ci reviewran on 5d0d402 ℹ️ InfoDesktop E2E visual evidence · View test artifacts · View job1 visual diff. inline evidence upload failed. Failed to upload diff-665a0833239e-onboarding-overlay-diff.png with gh image (exit code 1): Error uploading /home/runner/work/_temp/e2e-evidence/diff-665a0833239e-onboarding-overlay-diff.png: step 0 (get upload token): uploadToken not found on repo page — do you have write access to NousResearch/hermes-agent? (or, if NousResearch enforces SAML SSO, authorize at https://github.com/orgs/NousResearch/sso) |
alt-glitch
added
type/bug
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Images and files posted earlier in a Slack thread are now visible to the agent: every thread-context path annotates them with typed markers (
[image: chart.png],[file: report.pdf (application/pdf)]), and on cold-start the thread-root image is downloaded and delivered so "look at this chart" mentions actually work.Changes
_render_message_textappends sanitized media markers for every file attachment — covers cold-start hydrate, delta refresh, restart rehydration, and reply_to_text; hostile filenames (newlines/brackets) sanitized._collect_thread_root_images()on the cold-start path only: reads the root from the just-populated context cache (zero extra API calls), downloadsimage/*via the existing_download_slack_file, delivers as PHOTO. Capped at 4; download failure degrades to the marker. One-time by construction — no gateway/session plumbing.Credits
Salvaged from #32315 (@yemi-lagosinternationalmarket — first implementation of context markers) and adapted from #69185 (@KCAYAAI — thread-root delivery design; the full 2,400-line MessageEvent plumbing across gateway core was judged overreach for the user-visible goal). #66136 (lazy file loading) deferred as complementary.
Supersedes #69185, #32315.
Validation
tests/gateway/ -q -k slackInfographic