Skip to content

fix(gateway): canonicalize WhatsApp identity in session keys - #14904

Closed
keiravoss94 wants to merge 1 commit into
NousResearch:mainfrom
pebble-tech:feature/whatsapp-dm-canonical-session-key
Closed

fix(gateway): canonicalize WhatsApp identity in session keys#14904
keiravoss94 wants to merge 1 commit into
NousResearch:mainfrom
pebble-tech:feature/whatsapp-dm-canonical-session-key

Conversation

@keiravoss94

@keiravoss94 keiravoss94 commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Problem

On WhatsApp, the bridge routinely hands the gateway two different JIDs for the same contact within the same conversation — the LID form (999999999999999@lid) and the phone form (15551234567@s.whatsapp.net) — and flips between the two. Because build_session_key used raw identifiers verbatim, the flip produced two distinct session keys for the same human in two places:

  1. DM chat_id — a user's DM sessions split in half; transcripts and per-sender state diverge.
  2. Group participant_id (with group_sessions_per_user=True) — a member's per-user session inside a group splits in half for the same reason.

This is a different concern from the existing _expand_whatsapp_auth_aliases helper in gateway/run.py, which consults the same lid-mapping-*.json bridge files only for authorisation. Session keying never ran through that mapping, so the split persisted even when auth succeeded.

Fix

Collapse WhatsApp user-identity aliases into a single canonical identifier when building the session key. Three small helpers are added to gateway/session.py:

  • normalize_whatsapp_identifier (public) — strips @lid, @s.whatsapp.net, device suffix (:N), and leading + down to the bare numeric form.
  • _expand_whatsapp_aliases (private) — walks the bridge's $HERMES_HOME/whatsapp/session/lid-mapping-<id>{,_reverse}.json files (same mapping the auth path already uses) to resolve LID/phone pairs to a single set.
  • canonical_whatsapp_identifier (public) — picks the shortest identifier (ties broken lexicographically) as the stable key component.

build_session_key now routes both the WhatsApp DM chat_id and the WhatsApp group participant_id through canonical_whatsapp_identifier. All other platforms and chat types are untouched.

Result: both JID shapes produce the same key, e.g.

  • DM: agent:main:whatsapp:dm:15551234567
  • Group (per-user sessions): agent:main:whatsapp:group:120363…@g.us:15551234567

so the transcript is shared across alias flips.

Why expose two of the helpers

Plugins that need per-sender behaviour on WhatsApp (role-based routing, per-contact authorization, policy gating in a gateway hook) must resolve the same JID↔LID aliases the core already resolves — otherwise their bookkeeping keys drift apart from Hermes' session keys. Without a public helper, each such plugin would have to reimplement the walker against the bridge's internal on-disk format.

Exposing canonical_whatsapp_identifier and normalize_whatsapp_identifier keeps the resolution authoritative in one place. _expand_whatsapp_aliases stays private — it's an implementation detail of how the mapping files are walked, not a contract callers should depend on.

Test plan

  • TestWhatsAppSessionKeyConsistency::test_whatsapp_dm_aliases_share_one_session_key — DM LID and phone JID share one key.
  • TestWhatsAppSessionKeyConsistency::test_whatsapp_group_participant_aliases_share_session_key — group participant LID and phone JID share one per-user key when group_sessions_per_user=True.
  • TestWhatsAppSessionKeyConsistency::test_whatsapp_group_shared_sessions_untouched_by_canonicalisationgroup_sessions_per_user=False path is a no-op for canonicalisation (participant isn't in the key anyway).
  • TestWhatsAppIdentifierPublicHelpers — contract tests for the public helpers (JID/LID/device-suffix/plus-prefix stripping, transitive mapping walk, empty input).
  • tests/gateway/test_session.py — 75/75 pass locally.
  • Adjacent group_sessions_per_user-dependent suites (test_config.py, test_slack.py, test_shared_group_sender_prefix.py) — 160/160 pass locally.

Notes for reviewers

  • Helpers equivalent to normalize_whatsapp_identifier / _expand_whatsapp_aliases already exist in gateway/run.py for auth-alias matching. They are kept locally in gateway/session.py here to keep the diff small; happy to extract both usages into a shared module (e.g. gateway/whatsapp_identity.py) as a follow-up if you prefer — the helpers can't be imported directly from gateway.run because run.py already imports from gateway.session.
  • website/docs/user-guide/sessions.md is updated to reflect the new DM key shape.
  • Behaviour on non-WhatsApp platforms is unchanged; when no lid-mapping file is present, the result is identical to stripping @... off the raw identifier, so there is no migration path needed for existing sessions other than the normal drift that already happens when a user's JID form changes.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/whatsapp WhatsApp Business adapter labels Apr 24, 2026
@github-actions
github-actions Bot force-pushed the feature/whatsapp-dm-canonical-session-key branch from 7c4771d to 7ee1795 Compare April 24, 2026 05:01
@keiravoss94
keiravoss94 force-pushed the feature/whatsapp-dm-canonical-session-key branch 3 times, most recently from d0f0a9d to 5459465 Compare April 24, 2026 10:28
@keiravoss94 keiravoss94 changed the title fix(gateway): canonicalize WhatsApp DM identity in session keys fix(gateway): canonicalize WhatsApp identity in session keys Apr 24, 2026
@keiravoss94
keiravoss94 force-pushed the feature/whatsapp-dm-canonical-session-key branch from 5459465 to 08352d0 Compare April 24, 2026 10:35
Hermes' WhatsApp bridge routinely surfaces the same person under either
a phone-format JID (60123456789@s.whatsapp.net) or a LID (…@lid),
and may flip between the two for a single human within the same
conversation. Before this change, build_session_key used the raw
identifier verbatim, so the bridge reshuffling an alias form produced
two distinct session keys for the same person — in two places:

  1. DM chat_id — a user's DM sessions split in half, transcripts and
     per-sender state diverge.
  2. Group participant_id (with group_sessions_per_user enabled) — a
     member's per-user session inside a group splits in half for the
     same reason.

Add a canonicalizer that walks the bridge's lid-mapping-*.json files
and picks the shortest/numeric-preferred alias as the stable identity.
build_session_key now routes both the DM chat_id and the group
participant_id through this helper when the platform is WhatsApp.
All other platforms and chat types are untouched.

Expose canonical_whatsapp_identifier and normalize_whatsapp_identifier
as public helpers. Plugins that need per-sender behaviour (role-based
routing, per-contact authorization, policy gating) need the same
identity resolution Hermes uses internally; without a public helper,
each plugin would have to re-implement the walker against the bridge's
internal on-disk format. Keeping this alongside build_session_key
makes it authoritative and one refactor away if the bridge ever
changes shape.

_expand_whatsapp_aliases stays private — it's an implementation detail
of how the mapping files are walked, not a contract callers should
depend on.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via #15191#15191

Your fix and regression suite were cherry-picked onto current main with your authorship preserved (commit 10deb1b on main). A small follow-up commit on top extracts the JID/LID helpers into a new gateway/whatsapp_identity.py module so the session-key path and the existing auth-alias path in gateway/run.py share one resolver — addressing your own note about keeping the two in sync.

Thanks for the careful writeup and the public-helper contract tests.

@teknium1 teknium1 closed this Apr 24, 2026
@keiravoss94
keiravoss94 deleted the feature/whatsapp-dm-canonical-session-key branch April 26, 2026 09:25
github-actions Bot pushed a commit to pebble-tech/hermes-agent that referenced this pull request Apr 27, 2026
This is the sole commit on ops-overlay. It carries:

  - .github/workflows/sync-upstream.yml: daily rebase + rebuild of main
    on top of upstream/main + ops-overlay + each feature branch.
  - FORK.md: branch layout, sync model, recovery procedure.

Steady state (2026-04-26 onward): both feature branches we used to
carry have merged upstream (PRs NousResearch#13445 and NousResearch#14904 via NousResearch#15050 and
NousResearch#15191). FEATURE_BRANCHES is now empty; main collapses to
upstream/main + ops-overlay. The fork stays alive so customer VPSes
have a stable deploy target rebuilt on our schedule.
github-actions Bot pushed a commit to pebble-tech/hermes-agent that referenced this pull request Apr 30, 2026
This is the sole commit on ops-overlay. It carries:

  - .github/workflows/sync-upstream.yml: daily rebase + rebuild of main
    on top of upstream/main + ops-overlay + each feature branch.
  - FORK.md: branch layout, sync model, recovery procedure.

Steady state (2026-04-26 onward): both feature branches we used to
carry have merged upstream (PRs NousResearch#13445 and NousResearch#14904 via NousResearch#15050 and
NousResearch#15191). FEATURE_BRANCHES is now empty; main collapses to
upstream/main + ops-overlay. The fork stays alive so customer VPSes
have a stable deploy target rebuilt on our schedule.
github-actions Bot pushed a commit to pebble-tech/hermes-agent that referenced this pull request May 6, 2026
This is the sole commit on ops-overlay. It carries:

  - .github/workflows/sync-upstream.yml: daily rebase + rebuild of main
    on top of upstream/main + ops-overlay + each feature branch.
  - FORK.md: branch layout, sync model, recovery procedure.

Steady state (2026-04-26 onward): both feature branches we used to
carry have merged upstream (PRs NousResearch#13445 and NousResearch#14904 via NousResearch#15050 and
NousResearch#15191). FEATURE_BRANCHES is now empty; main collapses to
upstream/main + ops-overlay. The fork stays alive so customer VPSes
have a stable deploy target rebuilt on our schedule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/whatsapp WhatsApp Business adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants