fix(slack): drop message_changed re-emits whose body is unchanged - #89908
fix(slack): drop message_changed re-emits whose body is unchanged#89908nikitaBarkov wants to merge 2 commits into
Conversation
Slack re-dispatches message_changed for its own metadata updates (async language detection, unfurl) with the message body byte-identical, usually while the original message is still in ingress. Neither existing guard in _handle_slack_message stops it: _processed_message_ts is armed only at the end of the method, after users.info, thread hydration and file downloads, and the dedup cache keys such an event under the synthetic _slack_changed_event_ts rather than the message ts. The same user message reaches the gateway twice and the agent answers twice. Record a digest of the rendered message body (flat text plus the Block Kit payload the agent is shown) before the first await, and drop a message_changed whose body digest is unchanged. An edit that changes the body — including one that adds a bot mention, in the text or in the blocks — still passes, so edited-in mentions keep working.
andrexibiza
left a comment
There was a problem hiding this comment.
Exact-head review at 4ddebec11ca66bd392f3a21dd99443ecec7a1a6d.
Merge recommendation: changes required. The production diagnosis is convincing and recording a pre-await baseline is the right primitive. The current implementation, however, still conflates metadata replay with real edits in several normal Slack shapes.
[P1] The completed-message guard still suppresses every genuine later edit
The message_changed branch first does:
if original_message_ts and original_message_ts in self._processed_message_ts:
returnThat happens before the new body comparison. Once the original message has completed ingress, a text edit, block-only edit, or attachment-only edit is discarded unconditionally. The new test_block_only_update_during_ingress_still_routes holds the original inside users.info, so it proves only the narrow race window where _processed_message_ts is not yet armed; it does not prove the PR body's broader claim that a genuine body change still passes.
Replace the terminal timestamp-only short circuit with the content/revision decision. A completed original should suppress a message_changed only when the agent-visible body and revision semantics prove it is Slack's metadata-only re-emission.
[P1] The signature omits attachments that the adapter later shows to the agent
_slack_message_body_signature() hashes flat text and blocks only. Later in the same inbound path, attachments are rendered into agent-visible text from title, title_link/from_url, text, footer, and fallback (excluding only is_msg_unfurl). An attachment/card can therefore change while text and blocks remain identical, and this guard will drop it as unchanged.
Do not solve Slack-generated unfurls by excluding the entire attachment channel. Normalize the exact attachment fields the agent consumes and distinguish Slack-owned metadata enrichment from authored attachment/card changes. Add both an attachment-only real edit and a metadata-only unfurl re-emission test.
[P1] A bare Slack timestamp is not a canonical message identity
_seen_message_body_digest is adapter-wide and keyed only by message.ts. The rest of this adapter already scopes event/thread markers because timestamps can collide across workspaces; Slack message operations themselves identify a message by channel plus timestamp. A same-ts/same-body message in another workspace or channel can seed the digest and cause a legitimate message_changed to be dropped. Thread roots and replies also need to remain distinct.
Key the baseline by the full routing identity available at ingress: profile/route or adapter instance, team/workspace, channel, message ts, and thread root where relevant. Cover:
- same text with different message timestamps;
- same timestamp in different channels and workspaces;
- parent versus reply identities inside one thread;
- identical repeated user messages that are separate Slack messages.
[P1] The baseline is arrival-ordered, not revision-ordered
The map stores only a digest. If message_changed arrives first (reconnect/replay ordering) it records the edited body under the original ts; a later delivery of the original message then overwrites that baseline with the stale body and can itself route as a turn because its dedup key differs from the edit event's synthetic timestamp. A later metadata re-emission of the edited body then appears changed again.
Store a monotonic Slack revision alongside the digest (edited.ts/event timestamp plus the original generation), and advance the baseline only when the incoming version is newer. The first-event/edit matrix needs both orderings and concurrent completion, not only original-first with an artificial ingress gate.
The fix should remain narrowly targeted: deduplicate only a proven metadata-only re-emission of the same canonical message revision. It must not deduplicate by text, by bare timestamp, or by arrival order.
…r workspace The unchanged-body guard in _handle_slack_message compared the flat text and the Block Kit payload only, and both it and the delivered-message registry keyed on the bare Slack ts. The inbound path also renders the legacy attachments into the text the agent is shown, so a chat.update that rewrites a card while keeping a static fallback text was dropped as unchanged. Fold the authored attachments into the compared signature. Attachments Slack generates itself while unfurling a link (is_msg_unfurl, from_url, original_url, app_unfurl_url, service_name) stay out: treating Slack's own enrichment as a body change would restore the extra turn on every message with a link. Slack timestamps are unique within one workspace only, so a delivered message in one workspace could swallow a genuine message_changed carrying the same inner ts in another. Key both _seen_message_body_digest and _processed_message_ts by _workspace_event_id(team_id, ts), on read and on write, and resolve the workspace id from the normalized message so the lookup matches what the delivery path recorded.
|
Thanks for the review — this was a careful read and two of the four points were real holes I'd left open. Both are fixed in [P1] Signature omits attachments — fixedYou're right, and the failure mode was exactly as described:
[P1] A bare timestamp is not an identity — fixedAlso correct, and it was worse than the digest map alone: On the wider identity you sketched: I stopped at workspace + ts deliberately. A Slack [P1] The completed-message guard suppresses later edits — acknowledged, out of scopeFair as a fact, and my PR description overstated things — I've corrected that sentence. But that [P1] The baseline is arrival-ordered — acknowledged, out of scopeThe ordering you describe is real, and I don't have it in the production data behind this PR (127 Verification after the follow-up: |
What does this PR do?
Stops Slack from turning one user message into two agent turns.
Slack re-dispatches
message_changedfor updates it makes to a message on its own, with the body byte-identical and nobody having touched the message. This is documented behavior: "This event can also sometimes be triggered by our automatic language detection, which can add or update language or locale information to the metadata for the message, prompting the event to be dispatched" (message_changedreference). The re-emit arrives about a second after the original — while the original is still being ingested.Neither barrier in
_handle_slack_message()stops it:_processed_message_tsguard at the top of themessage_changedbranch is armed only at the very end of the method, afterusers.info, thread-context hydration and attachment downloads. That leg takes 0.2–2s, and the re-emit lands inside it, so the guard sees an empty map and lets the event through._slack_changed_event_ts, not under the message ts — deliberately, otherwise an edit that adds a bot mention would be swallowed. So it does not match the original delivery either.The event is then normalized into a plain message and handed to the gateway a second time. The gateway parks it as a follow-up and drains it as a brand-new turn: a second identical answer to the user, a second full API call, and a duplicated user turn in the transcript.
Evidence from a production workspace (one gateway session, five days of
gateway.logplus the session DB):queued follow-up (FIFO)→Draining queued follow-up … as a new turn. One inbound message, two turns, every time.message+ optionalapp_mention(suppressed by the dedup — same ts, same key) +message_changed(a different key — the one that gets through). If a proxy or a second listener were duplicating traffic, theapp_mentiontwin would be dropped twice; it never is.message_changedevents in that window, 108 arrived less than 2s after the original (median 0.8s) — machine speed, no human involved. The only three that arrived minutes later (172s, 417s, 417s) were genuine edits.Because the body is always identical, the fix does not have to reason about timing at all: record a digest of the rendered body before the first
await, and drop amessage_changedwhose body is unchanged. An edit that genuinely changes the body — including one that adds a bot mention, in the text, the blocks or an authored attachment — passes this new guard, so the edited-in-mention behavior is untouched. (To be precise about the scope: once the original has finished ingress, the pre-existing_processed_message_tsguard from49497bcddbstill drops everymessage_changedfor it, edits included. That behavior is onmaintoday and this PR does not change it — see Notes.)Related Issue
No linked issue — diagnosed from production logs, mechanism confirmed against the Slack documentation above. The same trigger has bitten other Slack integrations: n8n issue 23782 describes Slack's asynchronous locale enrichment re-triggering a bot reply for a message nobody edited.
Adjacent open PRs, complementary rather than overlapping:
message_changedevents by comparing the event's ownprevious_messagesnapshot, and target replay after a restart. This PR is about the message the process is ingesting right now: it compares against the body this process actually saw, needs noprevious_message(Slack's metadata re-emit does not have to carry a usable one), and closes the in-flight race those PRs do not touch. The guards are independent and compose — if both land, an unchanged body is simply dropped one check earlier.Type of Change
Changes Made
plugins/platforms/slack/adapter.py: new_slack_message_body_signature(message)— the part of a message body the agent is actually shown, as one value: the flattextplus the same renderings the inbound path uses for blocks (_extract_text_from_slack_blocks,_serialize_slack_blocks_for_agent) and for the legacyattachments(_extract_text_from_slack_attachments). Keying ontextalone would silently drop achat.updatethat rewrites a bot card while keeping a static fallback text, which underallow_bots=allis real content.plugins/platforms/slack/adapter.py: new_slack_authored_attachments(attachments)— only the attachments an author or an app set take part in the signature. The ones Slack generates itself while unfurling a link are recognised by the keys only it sets (_SLACK_GENERATED_ATTACHMENT_KEYS:is_msg_unfurl,from_url,original_url,app_unfurl_url,service_name) and are deliberately left out: Slack attaches them on its own, so counting them as a body change would hand its own re-emit a turn — exactly the bug being fixed.plugins/platforms/slack/adapter.py: new_slack_body_digest(body)(SHA-256,surrogatepassso an unpaired surrogate in a payload cannot raise) and the boundedself._seen_message_body_digestmap (_SEEN_MESSAGE_BODY_DIGEST_MAX = 5000, evicted oldest-first through the existing_trim_oldest_dict_entries). Only the digest is stored — a rendered Block Kit payload can be 6 KB and never needs to stay in memory.plugins/platforms/slack/adapter.py:_remember_message_body(message_key, body)is called in_handle_slack_message()before the firstawait, which is the whole point — that is why a baseline exists while_processed_message_tsis still unarmed. It sits below the two synchronous filters that precede it (the redelivery dedup and the ignored-channel check), so neither a replay nor an ignored channel takes a slot. An empty body is never recorded, so a message carrying only files is never mistaken for "unchanged".plugins/platforms/slack/adapter.py: both maps are keyed by_workspace_event_id(team_id, ts), not by the bare Slackts— a timestamp identifies a message only within one workspace, and the adapter already scopes its other markers that way. This covers_seen_message_body_digestand, on the same key, the pre-existing_processed_message_tsregistry, on read and on write; themessage_changedbranch normalizes the routing fields before either guard so the id it resolves is the one the delivery path recorded under.plugins/platforms/slack/adapter.py: themessage_changedbranch gains a second guard — when the body digest matches the one recorded for that message, the event is dropped with one INFO line ([Slack] dropped message_changed with unchanged body ts=… channel=…), so a future duplicate report can be told apart from this class by the presence or absence of that line.tests/gateway/test_slack.py: seven new tests around a shared_IngressGatehelper that holdsusers.infoopen so the second event genuinely races an in-flight ingress (asyncio.Event-driven, no polling).test_metadata_only_edit_during_ingress_routes_once(parametrized with and without therich_textblocks a composed message really carries) andtest_link_unfurl_during_ingress_routes_onceassert a singlehandle_message;test_real_edit_during_ingress_still_routes,test_block_only_update_during_ingress_still_routesandtest_attachment_only_update_during_ingress_still_routesassert two, for a changed text, changed blocks and a changed authored card under a static fallback text;test_same_ts_in_another_workspace_still_routescovers the same innertsin another workspace against both maps — while the original is still in ingress and after it has been delivered;test_ignored_channel_body_is_not_rememberedasserts the baseline is not recorded for an ignored channel.tests/gateway/test_slack_mention.py: the hand-built adapter stub gains the two new attributes.No config key, no env var, no schema change, and nothing in
gateway/— no other platform's behavior moves.How to Test
scripts/run_tests.sh tests/gateway/test_slack.py tests/gateway/test_slack_mention.py— 200 passed, 0 failed on this branch.== "mutation-never-matches") and both parametrizations oftest_metadata_only_edit_during_ingress_routes_oncefail withthe re-emit entered the ingress instead of being dropped; restore it and they pass. The tests asserting two turns stay green either way, which is the point — they prove the guard does not swallow real changes. Two more mutations pin the other half: dropattachmentsback out of the signature andtest_attachment_only_update_during_ingress_still_routesfails; unscope either the read or the write of_processed_message_tsandtest_same_ts_in_another_workspace_still_routesfails.main,gateway.logshows twodelivering event to gatewaylines with the samets, a queued follow-up drained as a new turn, and the user gets two identical answers. On this branch the second event ends atdropped message_changed with unchanged bodyand there is exactly one turn.test_message_edit_with_new_mention_processed, unchanged, plustest_real_edit_during_ingress_still_routesfor the racing variant).Checklist
Code
fix(scope):,feat(scope):, etc.)main)pytest tests/for the affected area and all tests passDocumentation & Housekeeping
docs/, docstrings) — N/A, no user-facing setting or documented behavior changes; the reasoning lives in the new helpers' docstringscli-config.yaml.exampleif I added/changed config keys — N/A, no config keysCONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/ANotes
Deliberately out of scope, to keep the change on the confirmed mechanism:
_queue_or_replace_pending_event). A cross-platform "this message_id is already the running turn" guard would cover more, but it drops inbound messages on a key the platforms do not all populate the same way — a worse failure than a duplicate if the key is wrong.messageevents:_processed_message_tsis already armed beforehandle_message()on that path, and a body-equality drop there could suppress a legitimate re-delivery._processed_message_tsshort circuit is left as it is. It drops everymessage_changedfor a message that has finished ingress, genuine edits included — behavior introduced with themessage_changedbranch itself in49497bcddband live onmain. Replacing it with a content/revision decision would make every later edit a new agent turn, which is a behavioral change for the edit flow rather than a fix for the duplicate-turn bug.tsis already unique inside a workspace, andchannelis not carried on everymessage_changedshape the adapter accepts, so adding it buys no discrimination and adds a way to miss the lookup and let a duplicate through.message_changedis delivered before the original (reconnect replay), it records the edited body and the original then overwrites it. Both events already produce a turn onmainin that ordering, so this is not a regression; closing it needs a stored Slack revision (edited.ts) and a version comparison, which is more machinery than the confirmed mechanism justifies.message_changedonly to attach a link unfurl, the agent no longer sees that preview — by design, since the alternative is an entire extra turn per link. The URL itself is still in the message text and can be fetched.