Skip to content

fix(slack): humanize inbound user mentions + ground bot identity - #55340

Closed
benbarclay wants to merge 1 commit into
mainfrom
fix/slack-humanize-mentions
Closed

fix(slack): humanize inbound user mentions + ground bot identity#55340
benbarclay wants to merge 1 commit into
mainfrom
fix/slack-humanize-mentions

Conversation

@benbarclay

@benbarclay benbarclay commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Infographic

slack-identity-grounding

Problem

Slack delivers user mentions as opaque IDs (<@U123>). The adapter stripped only the bot's own mention and passed every other participant's <@U…> token to the agent raw:

# the ONLY inbound mention rewrite — the bot's own UID, nothing else:
text = text.replace(f"<@{bot_uid}>", "").strip()

With no name map and no knowledge of its own handle, the agent receives something like:

[Alice Example] <@U07BOB> I think thread is prob the right default

In a thread the bot is already participating in (correctly auto-responding to follow-ups), the model has no way to tell that <@U07BOB> is a human and not itself — so it reads the mention as directed at it and replies. This is the reported symptom: the bot treats a participant's mention as if it named the bot, and even insists it was "explicitly mentioned." Discord never hits this because it feeds the agent message.clean_content (IDs already rendered as @DisplayName); Slack was the outlier.

Note: this is not a mention-gate routing collision — the channel gate is a literal f"<@{bot_uid}>" in text match that a human's ID cannot satisfy. The bug is in what the agent perceives, not in whether the message was routed. (The separate user-token-vs-bot-token guard is #55332; this PR is the actual fix for the reported symptom.)

Fix (two cooperating parts)

1. Humanize inbound mentions_humanize_user_mentions() rewrites remaining <@UID> and <@UID|label> tokens in the trigger text, thread context, and reply-to text to @DisplayName, resolved via the cached _resolve_user_name() (one users.info lookup per distinct user per process). Unresolvable IDs fall back to the bare ID rather than emptying the mention. This is the Slack equivalent of Discord's clean_content.

2. Ground the bot's identity_build_identity_prompt() returns a line naming the bot's own Slack handle:

You are connected to this Slack workspace as the bot "@Yourbot". … Only treat a message as directed at you when it mentions "@Yourbot" specifically; a mention of any other participant is not a mention of you, even if their name is similar.

It's injected through the per-turn channel_prompt ephemeral seam — applied at API-call time, never persisted to history, so per-conversation prompt caching is preserved. The bot display name is captured per-workspace at connect time (multi-workspace safe; cleared on reconnect alongside the existing _bot_user_id reset).

Tests

tests/gateway/test_slack_mention_humanization.py:

  • mention humanization: single, multiple-distinct, labelled (<@UID|handle>), repeated, unresolvable→ID fallback, and no-mention no-op
  • identity prompt: names the bot, prefers the per-team name over the primary, and returns empty before connect (so callers skip injecting a half-formed line)

Prove-fail verified: all 9 fail with the adapter change stashed, all 9 pass with it. The existing 256 Slack tests stay green. An out-of-test E2E run against the real _humanize_user_mentions / _build_identity_prompt confirms a human's <@UID> is rendered to @DisplayName and the identity anchor names the bot's own handle.

Reinstall caveat

None — this is a pure runtime change. No manifest/scope change, so no Slack app reinstall is required; it takes effect on the next gateway restart.

@benbarclay
benbarclay requested a review from teknium1 June 30, 2026 01:40
@github-actions

github-actions Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: fix/slack-humanize-mentions vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11727 on HEAD, 11724 on base (🆕 +3)

🆕 New issues (3):

Rule Count
unresolved-attribute 2
unresolved-import 1
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
run_agent.py:3069: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`
tests/gateway/test_slack_mention_humanization.py:19: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`

✅ Fixed issues (1):

Rule Count
invalid-assignment 1
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`

Unchanged: 6173 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery platform/slack Slack app adapter P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jun 30, 2026
Slack delivers user mentions as opaque IDs (<@u123>). The adapter stripped
only the bot's OWN mention and passed every other participant's <@uid> to
the agent raw. With no name map and no knowledge of its own handle, the
agent could read a human's mention as a self-mention and reply to messages
merely addressed to that person — it would treat a participant's mention as
if it named the bot. Discord never hits this because it feeds the agent
message.clean_content (IDs already rendered as names).

Two cooperating fixes:

- _humanize_user_mentions: rewrite remaining <@uid> (and <@uid|label>)
  tokens in the trigger text, thread context, and reply-to text to
  @DisplayName via the cached _resolve_user_name (one users.info lookup per
  distinct user per process). The Slack equivalent of clean_content.

- _build_identity_prompt: an ephemeral system-prompt line naming the bot's
  own Slack handle ("you are @x; a mention of any other participant is not a
  mention of you"), injected through the per-turn channel_prompt seam —
  applied at API-call time, never persisted to history, so per-conversation
  prompt caching is preserved. Bot display name is captured per-workspace at
  connect time (multi-workspace safe, cleared on reconnect).

Tests in tests/gateway/test_slack_mention_humanization.py cover single /
multiple / labelled / repeated / unresolvable / no-op mention cases and the
identity-prompt builder (per-team name preference, empty-before-connect).
Prove-fail verified: all 9 fail with the adapter change stashed, pass with
it; the existing 256 Slack tests stay green.
@benbarclay
benbarclay force-pushed the fix/slack-humanize-mentions branch from 6e008db to 4a150a0 Compare June 30, 2026 01:59

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Well-documented fix for the Slack bot identity confusion bug. Humanizes user mentions and grounds the bot's own identity via ephemeral channel_prompt. Clean with tests.

@tonydwb

tonydwb commented Jun 30, 2026

Copy link
Copy Markdown

Code Review Summary

Verdict: Approved

Replaces raw <@UID> Slack mention tokens with @DisplayName and adds bot identity grounding so the agent can distinguish self-mentions from human mentions. This fixes the "bot thinks it's @Someone-Else" bug.

✅ Looks Good

  • Clean approach: humanize mentions + identity prompt
  • Good test coverage: single/multiple mentions, labelled form, repeated, unresolvable, no mentions
  • Identity prompt is ephemeral (applied at API-call time, not persisted)
  • Preserves prompt caching by not mutating history
  • Per-team bot name support for multi-workspace setups

Reviewed by Hermes Agent

teknium1 pushed a commit that referenced this pull request Jul 22, 2026
Slack delivers user mentions as opaque IDs (<@u123>). The agent had no
way to tell one participant from another — or from itself — so it could
misread a mention of a human as a self-mention and answer messages
addressed to that person (the "bot thinks it's @Someone-Else" bug).

Two cooperating fixes:
- _humanize_user_mentions rewrites remaining <@uid> tokens (the bot's
  own mention is stripped earlier) to @DisplayName in the trigger text
  and reply_to_text — the Slack equivalent of Discord's clean_content.
  Handles the labelled <@uid|handle> form; unresolvable IDs fall back
  to the raw ID.
- _build_identity_prompt injects an ephemeral per-turn system-prompt
  line via the channel_prompt seam (applied at API-call time, never
  persisted — prompt caching preserved) naming the bot's own workspace
  handle (per-team in multi-workspace installs) so the agent has a
  positive "that's me" anchor.

Salvaged from #55340 by @benbarclay, rebased over the workspace-scoped
user-name cache (team_id-aware resolution) on main.
teknium1 pushed a commit that referenced this pull request Jul 22, 2026
Slack delivers user mentions as opaque IDs (<@u123>). The agent had no
way to tell one participant from another — or from itself — so it could
misread a mention of a human as a self-mention and answer messages
addressed to that person (the "bot thinks it's @Someone-Else" bug).

Two cooperating fixes:
- _humanize_user_mentions rewrites remaining <@uid> tokens (the bot's
  own mention is stripped earlier) to @DisplayName in the trigger text
  and reply_to_text — the Slack equivalent of Discord's clean_content.
  Handles the labelled <@uid|handle> form; unresolvable IDs fall back
  to the raw ID.
- _build_identity_prompt injects an ephemeral per-turn system-prompt
  line via the channel_prompt seam (applied at API-call time, never
  persisted — prompt caching preserved) naming the bot's own workspace
  handle (per-team in multi-workspace installs) so the agent has a
  positive "that's me" anchor.

Salvaged from #55340 by @benbarclay, rebased over the workspace-scoped
user-name cache (team_id-aware resolution) on main.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via #69320 — your commit was cherry-picked onto current main with your authorship preserved in git history: your mention humanization + bot identity grounding was cherry-picked directly.

Thanks for the contribution!

@tw0316

tw0316 commented Jul 23, 2026

Copy link
Copy Markdown

@benbarclay I may have found a secondary regression. _build_identity_prompt() currently tells the model:

Only treat a message as directed at you when it mentions “@{name}” specifically.

That instruction conflicts with valid unmentioned messages the Slack adapter has already chosen to route, including:

  • Channels configured under slack.free_response_channels
  • Thread follow-ups where the agent is already participating
  • Configurations where slack.require_mention is disabled

The gateway delivers these messages correctly, but the model is simultaneously told not to treat them as directed at it. Because this is natural-language guidance rather than a hard routing gate, the result is intermittent: the agent sometimes proceeds based on thread context and sometimes refuses to act without another explicit mention.

I reproduced this on current main (91546b833). The Slack configuration was unchanged before and after the regression appeared; restarting the gateway simply loaded the newly merged prompt wording.

A narrow fix would preserve the identity protection without overriding channel routing, maybe like the following:

A mention of any other participant is not a mention of you, even if their name is similar. Messages delivered to you without a mention may still be directed at you according to the channel's routing policy.

@tw0316

tw0316 commented Jul 23, 2026

Copy link
Copy Markdown

After reviewing the original PR more closely and tracing the current Slack routing logic, I want to refine my earlier comment.

The original failure this PR aims to resolve appears narrower than free-response behavior:

  1. In a require_mention channel, the Slack adapter drops top-level messages unless they mention the bot.
  2. Once the bot is mentioned and participating in the resulting thread, subsequent thread replies are intentionally passed to the agent without requiring another mention.
  3. If one of those replies mentions another participant, Slack previously exposed that mention as an opaque <@UID>.
  4. The model could not distinguish that participant’s mention from a mention of itself, so it sometimes replied and claimed it had been explicitly mentioned.

That means free-response channels were not the original bug surface. They became collateral damage because _build_identity_prompt() is injected unconditionally and says:

Only treat a message as directed at you when it mentions “@{name}” specifically.

I also do not think conditioning that existing sentence only on require_mention completely solves the problem. In an active thread, valid plain follow-ups such as “yes, go ahead” or “use option B” are intentionally delivered without another mention. The current sentence can cause the model to reject those too.

The intended behavior seems to be:

  • Require @bot to start a conversation in mention-required channels.
  • Once the bot is participating in the thread, process ordinary follow-ups.
  • Ignore a message that opens by addressing @someone-else, unless the bot is also mentioned.
  • Leave free-response channel behavior unchanged.

Hermes now has related adapter-level logic through ignore_other_user_mentions, although that setting is opt-in and currently applies across free-response channels as well.

My revised view is that routing should remain the adapter’s responsibility. The prompt should provide identity grounding only, for example:

You are @{name}. A mention of another participant is not a mention of you.

Then, for auto-followed threads in mention-required channels, the adapter can suppress only messages that begin by mentioning another participant without also mentioning the bot. This preserves normal unmentioned thread follow-ups without changing free-response channels.

randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Slack delivers user mentions as opaque IDs (<@u123>). The agent had no
way to tell one participant from another — or from itself — so it could
misread a mention of a human as a self-mention and answer messages
addressed to that person (the "bot thinks it's @Someone-Else" bug).

Two cooperating fixes:
- _humanize_user_mentions rewrites remaining <@uid> tokens (the bot's
  own mention is stripped earlier) to @DisplayName in the trigger text
  and reply_to_text — the Slack equivalent of Discord's clean_content.
  Handles the labelled <@uid|handle> form; unresolvable IDs fall back
  to the raw ID.
- _build_identity_prompt injects an ephemeral per-turn system-prompt
  line via the channel_prompt seam (applied at API-call time, never
  persisted — prompt caching preserved) naming the bot's own workspace
  handle (per-team in multi-workspace installs) so the agent has a
  positive "that's me" anchor.

Salvaged from NousResearch#55340 by @benbarclay, rebased over the workspace-scoped
user-name cache (team_id-aware resolution) on main.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/slack Slack app adapter sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants