Skip to content

fix(slack): prevent duplicate rich-text message content - #80240

Open
nikitaBarkov wants to merge 1 commit into
NousResearch:mainfrom
JetBrains:nikita.barkov/fix-slack-rich-text-duplication
Open

fix(slack): prevent duplicate rich-text message content#80240
nikitaBarkov wants to merge 1 commit into
NousResearch:mainfrom
JetBrains:nikita.barkov/fix-slack-rich-text-duplication

Conversation

@nikitaBarkov

@nikitaBarkov nikitaBarkov commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Stops one authored Slack message from reaching the agent twice.

Slack sends the same message flat in event.text and structurally in event.blocks. The blocks are rendered so quoted and forwarded content is not lost, and whatever the render carries beyond the flat text is appended. That comparison had several ways to fail on the same sentence, each of which showed the author their own words a second time:

  1. HTML entities. The flat copy escapes &/</> while blocks[].link.url stays raw, so any link with query parameters — every "Copy link" on a thread — mismatched.
  2. Permalink unfurls. The live inbound path skips is_msg_unfurl attachments; thread/parent hydration did not, so the linked message's body was appended again.
  3. The Block Kit dump. It serialized the authored rich_text alongside the UI blocks it exists for, and its allowlist drops url, so the sentence reappeared with every link removed. Scoped back to non-rich_text blocks, which is what fix: inline Slack block and attachment context #11426 described.
  4. Unknown inline elements. _render_inline_elements() knew eight types and silently dropped the rest. A pasted message permalink arrives as message_mention, so the link vanished from the render and the two sides stopped comparing equal.
  5. message_mention without a url. url is optional on that element while channel_id and message_ts are not, so it rendered as nothing and the sentence came back with a blank in the link's place.
  6. date elements. fallback and url are both optional, and the flat <!date^…> form was never read down to what the rich text renders.
  7. Labelled mentions. Slack may attach a label (<@U…|name>, <#C…|general>, <!subteam^S…|@marketing>, <!here|@here>) in the flat text while the blocks carry the bare id. The bot's own mention is one of these, and stripping only its bare form left it in the flat copy.
  8. Autolink schemes. Only https and mailto were matched, so a tel: link kept its angle brackets and mismatched too.
  9. A permalink reduction that ran too far. A labelled link is canonicalized to label (url), and reducing the permalink to its tail consumed everything up to the next space — the closing parenthesis included — so the two sides mismatched again whenever only one of them carried ?thread_ts=….

Cause 4 is the one confirmed against a live gateway log: one inbound message line per message (a rendering bug, not repeated delivery), the transcript carrying the sentence twice with a blank where the URL had been, and the same messages arriving once after the fix. The rest are reproducible with the adapter's own helpers.

Related Issue

Follows the same bug class as #66204, #26309 and #59903, which fixed individual mismatches without test coverage on the comparison path.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • plugins/platforms/slack/adapter.py: inline-element rendering moved into _render_slack_inline_element(). message_mention is read like link, and any unknown type by its url/text/fallback, so a type Slack adds later still renders; team, color and a fallback-less date render into the flat form Slack sends.
  • plugins/platforms/slack/adapter.py: channel_id and message_ts are the permalink's own components, so a url-less message_mention renders the permalink's tail; the workspace host and the thread query cannot be rebuilt from the element, so a permalink on either side is reduced to that same tail, stopping at the query so a label (url) form keeps its closing parenthesis.
  • plugins/platforms/slack/adapter.py: _normalize_slack_text_for_dedupe() unescapes &amp;/&lt;/&gt; before canonicalizing links, reads down the <!date^…> form and the optional mention label, strips the bot's mention after that label, and matches any autolink scheme.
  • plugins/platforms/slack/adapter.py: _extract_text_from_slack_attachments() skips is_msg_unfurl, matching the live path; _serialize_slack_blocks_for_agent() serializes only non-rich_text blocks.
  • plugins/platforms/slack/adapter.py: every field is read as a string or not at all, through one helper. Block Kit carries text as an object in many places, so an element -- an unknown one above all -- may hold one where a string belongs, and it would raise in the renderer's str.join and cost the whole message. Four such payloads raise on main and none do here.
  • tests/gateway/test_slack.py: TestSlackAuthoredTextDeduplication — 45 tests over both merge sites.

Canonicalization is used for matching only: the authored text still reaches the agent verbatim, so a mistake there can cost an unrendered element, never an altered or missing message.

Known gap, deliberately left open: an element carrying neither a url nor a label still renders as nothing, and a message containing one is still appended twice. Suppressing such a render is worse — an app message whose body lives only in the blocks disappears, and a forwarded quote is dropped. An unrendered element is the lesser cost.

How to Test

  1. Paste a Slack permalink to a message (.../archives/C…/p…) and send it to the bot — it arrives once, with the link intact.
  2. Send a thread "Copy link" URL (it carries ?thread_ts=…&cid=…), plain and with a label of your own — one copy each.
  3. Send a message mentioning a channel, a user group or @here, and one with a tel: link — one copy each.
  4. Post a permalink in a thread, then have the agent read the thread — the linked message's body is not repeated.
  5. Send a message alongside a real quote or forward — the quoted content is still delivered.
  6. scripts/run_tests.sh tests/gateway/test_slack.py222 passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit message follows Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate
  • This PR contains one commit and touches only the two files related to this fix
  • I've run all affected tests and they pass
  • I've added regression tests for the bug and its edge cases, including negative cases: quotes, lists, code blocks, alert attachments and interactive bot blocks must still be delivered
  • I've verified the behavior in a live Slack workspace
  • Prompt caching, message-role alternation and the system prompt are untouched

Documentation & Housekeeping

  • Documentation updates — N/A, no user-facing behavior or configuration changed
  • cli-config.yaml.example — N/A, no config changes
  • CONTRIBUTING.md / AGENTS.md — N/A, no architecture or workflow changes
  • Cross-platform impact considered — pure Python Slack adapter behavior
  • Tool descriptions/schemas — N/A, no tool changes

Notes

Out of scope: Slack also re-delivers a message as message_changed when it attaches an unfurl preview, while _processed_message_ts[ts] is written only at the end of _handle_slack_message, after every long await. That is repeated delivery rather than a duplicated render, it did not occur in the logs behind this report, and a naive fix breaks "an @mention added by an edit still wakes the bot" (#64957). #73450 already rewrites that branch.

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins platform/slack Slack app adapter P3 Low — cosmetic, nice to have sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Aug 6, 2026
@nikitaBarkov
nikitaBarkov force-pushed the nikita.barkov/fix-slack-rich-text-duplication branch from d83ec1c to 6fcfec9 Compare August 10, 2026 15:13
@nikitaBarkov
nikitaBarkov force-pushed the nikita.barkov/fix-slack-rich-text-duplication branch 17 times, most recently from ae151cd to 6c24001 Compare August 15, 2026 15:30
Slack sends an authored message twice: flat in `event.text` and structurally
in `event.blocks`. The blocks are rendered so quoted and forwarded content is
not lost, and whatever the render carries beyond the flat text is appended to
the message. That comparison had several ways to fail on the *same* sentence,
each of which showed the author their own words a second time:

1. HTML entities — the flat copy escapes `&`/`<`/`>` while `blocks[].link.url`
   stays raw, so any link with query parameters (every "Copy link" on a
   thread) mismatched.
2. Permalink unfurls — the live inbound path skips `is_msg_unfurl`
   attachments, thread/parent hydration did not, so the linked message's body
   was appended again.
3. The Block Kit dump — it serialized the authored `rich_text` alongside the
   UI blocks it exists for, and its allowlist drops `url`, so the sentence
   reappeared with every link removed.
4. Unknown inline elements — the renderer knew eight types and silently
   dropped the rest. A pasted message permalink arrives as `message_mention`,
   so the link vanished from the render and the sides stopped comparing equal.
5. `message_mention` without a url — `url` is optional on that element while
   `channel_id` and `message_ts` are not, so the element rendered as nothing
   and the sentence came back with a blank in the link's place.
6. `date` elements — `fallback` and `url` are both optional, and the flat
   `<!date^…>` form was never read down to what the rich text renders.
7. Labelled mentions — Slack may attach a label (`<@U…|name>`,
   `<#C…|general>`, `<!subteam^S…|@marketing>`, `<!here|@here>`) in the flat
   text while the blocks carry the bare id. The bot's own mention is one of
   these, and stripping only its bare form left it in the flat copy.
8. Autolink schemes — only `https` and `mailto` were matched, so a `tel:` link
   kept its angle brackets and mismatched too.

Unknown inline types are now read by their `url`/`text`/`fallback` so a type
Slack adds later still renders, and `team`, `color` and a fallback-less `date`
render into the flat form Slack sends. Every field is read as a string or
not at all: Block Kit carries text as an object in many places, and a
non-string one reaches the renderer's `str.join` and raises there, which
costs the whole message. `channel_id` and `message_ts` are the
permalink's own components, so a url-less `message_mention` renders the
permalink's tail; the workspace host and the thread query cannot be rebuilt
from the element, so a permalink on either side is reduced to that same tail.
Canonicalization is used for matching only -- the authored text still reaches
the agent verbatim, so a mistake here can cost an unrendered element, never an
altered or missing message.

An element carrying neither a url nor a label still renders as nothing, and a
message containing one is still appended twice. Suppressing such a render was
tried and is worse: an app message whose body lives only in the blocks
disappears, and a forwarded quote is dropped. Genuinely additional content --
quotes, lists, code blocks, attachments, interactive bot blocks -- is
unaffected throughout.

Tests cover both merge sites (live inbound and thread hydration) and the
negative cases.
@nikitaBarkov
nikitaBarkov force-pushed the nikita.barkov/fix-slack-rich-text-duplication branch from 6c24001 to 672ecb3 Compare August 15, 2026 15:41
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference, author can ignore or act on any point.

fix(slack): prevent duplicate rich-text message content

A well-tested, well-documented normalization effort (HTML-entity unescape, link/date/permalink/mention canonicalization, fenced/inline code, inline styles, message-unfurl skip, block-dump scoping). Points:

  1. Aggressive substring dedupe: _extract_additional_text_from_slack_blocks treats a rendered element as duplicate when normalized == primary or normalized in primary (adapter.py line 233). A distinct rich-text element whose text is merely a substring of the flat text (e.g. a quote repeating one sentence fragment, or a single repeated word as its own element) is silently dropped, even when it's genuinely additional quoted content. The tests pin the important cases; a length guard (only treat as duplicate when it's a large fraction of the primary, or exact-match) would reduce false drops.
  2. Nested-inline-style loop cost: _normalize_slack_text_for_dedupe's while True (line 196-200) peels one style layer per pass (_SLACK_INLINE_STYLE_RE with \1 backreference). Adversarial deeply-nested text (*_~…~_*` repeated) is O(depth × length) — bounded in practice, but a pathological 10KB message with hundreds of nesting levels could add noticeable latency on the message hot path. A cap on iterations (or a single-pass parse) would bound it.
  3. Message-unfurl skip is flag-based: is_msg_unfurl (line 586-587) skips only attachments Slack marks as unfurls. An unfurl-like attachment without the flag (some bot frameworks) would still be appended — acceptable, but worth noting the skip is best-effort on Slack's metadata.
  4. Permalink canonicalization is workspace-agnostic by design_slack_permalink_path rebuilds only archives/<channel>/p<ts> — and the tests cover thread-query vs raw forms well.
  5. Overall this closes the "second copy without the link" bug class cleanly; the scale of the test additions (700+ lines) matches the normalization surface.

@nikitaBarkov

Copy link
Copy Markdown
Contributor Author

Thanks for the read. I checked both actionable points against the code and against a measurement rather than reasoning about them, and neither holds up as a defect — details below so the conclusion is falsifiable. Points 3–5 I agree with as stated.

1. Substring dedupe — deliberate, and it cannot drop content the agent would otherwise see

Two facts make this safe:

The flat text is always delivered. _extract_additional_text_from_slack_blocks is named for what it does: at both call sites (adapter.py:6447 and :8228) the message text is the base and this function only appends what the blocks add on top of it. A rendered element that is a substring of the flat text is, by definition, words the agent is already receiving. The worst case of a false positive is losing the fact that a fragment was also quoted — never the fragment itself.

Exact match would reintroduce the bug. primary is Slack's serialization of the whole message, while the loop compares top-level elements of the rich_text block. Any message with more than one element — a line plus a quote, a paragraph plus a list — has every element as a proper substring of the flat text and nothing equal to it. Under normalized == primary alone the dedupe would fire only for single-element messages, and every richer message would get its whole body appended a second time: exactly the duplication this PR fixes. A length-fraction guard has the same failure mode, just on a threshold: a short quote under a long message is precisely the shape that duplicates.

The one thing that would genuinely be additional-yet-substring is a repetition Slack itself put in the flat text as well — so it is still not lost.

2. Nested inline styles — measured, and the premise is wrong

The loop does not peel one level per pass. re.sub replaces all non-overlapping matches in a single pass, so each pass removes an entire nesting layer across the whole string and the pass count is logarithmic in depth, not linear. Measured on the real _normalize_slack_text_for_dedupe (Python 3.13, M-series, perf_counter):

nested depth 1k    len=6007    12 passes   0.53 ms
nested depth 10k   len=60007   14 passes   4.88 ms
40k "*"            len=40000   11 passes   2.68 ms
40k alternating    len=39996    —          2.76 ms
realistic 4k msg   len=4000     —          0.22 ms

Slack's message limit is 40k characters, and the 60k row above is already past what can be posted. A realistic message costs ~0.2 ms; the adversarial 40k cases cost under 3 ms, on a path that then makes network calls. There is no pathological input to cap against, and an iteration cap would silently leave styled text un-normalized — turning a non-problem into a real one (a message that fails to dedupe and gets duplicated).

3. Unfurl skip is flag-based — agreed, and left as is on purpose

is_msg_unfurl keys on Slack's own metadata, so an unfurl-shaped attachment from a third-party bot that does not set the flag is still appended — i.e. it keeps exactly the pre-PR behaviour for those senders, nothing regresses for them. Guessing at unflagged attachments would mean heuristics on arbitrary bot payloads, with the failure mode of dropping genuine content, and I have no concrete producer to validate it against. That belongs in its own change with a real reproduction, not bolted onto this one.

4 & 5

Agreed — the permalink path is workspace-agnostic by design (the same message is linked from different hosts and with/without ?thread_ts), and the test volume tracks the normalization surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/slack Slack app adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants