feat(gateway): add session_keying=gmail_thread_id mode for email adapter

[ciscodebs]

Summary

Opt-in platforms.email.extra.session_keying setting that lets the email adapter route each visible-to-user mail thread to its own gateway session, keyed by Gmail's X-GM-THRID IMAP extension. Brings email parity with Slack / Discord / Telegram / Matrix, which already split sessions per thread.

Stacked on #11418 (precursor _thread_context clobber bugfix). The diff below includes both commits; they should land in order.

The user-visible problem

Today the email adapter omits thread_id when building the SessionSource, so every inbound message from a given sender collapses into one shared session forever:

1. Context poisoning across unrelated topics — memories the agent persists while handling Email A bleed into replies to unrelated Email B from the same sender.
2. Monotonic token-cost bloat — every new email's first turn replays the entire sender's prior conversation history.
3. Broken /reset semantics — resetting one thread destroys context for every other concurrent thread from the same sender.
4. Reply-header clobber — addressed in the precursor fix(gateway): rekey email _thread_context by (sender, message_id) to prevent reply clobber #11418.

Why Gmail-only (for now)

X-GM-THRID is Gmail's stable, server-assigned thread identifier, returned in the same FETCH round-trip as the body:

UID FETCH <uid> (RFC822 X-GM-THRID)

It matches what users see as a thread in Gmail (web/mobile/Apple Mail), inherits Gmail's threading heuristics (subject changes mid-thread, forwards, cross-client replies), and is documented at https://developers.google.com/gmail/imap/imap-extensions.

The alternative (RFC 5322 References walking + subject-hash) is ~150 LoC of error-prone parsing; defer to a follow-up that adds a generic thread_root mode for non-Gmail deployments. Default stays sender — non-Gmail users see zero change.

Changes

• Config: new extra.session_keying value, \"sender\" (default) or \"gmail_thread_id\". Unknown values warn-and-degrade to sender.
• Capability probe: connect() runs imap.capability() and records _has_gmail_ext. If gmail_thread_id is requested but X-GM-EXT-1 is absent, the adapter logs a warning and degrades to sender mode.
• Extended FETCH: _fetch_new_messages switches the FETCH payload to (RFC822 X-GM-THRID) when Gmail mode is active. Parses the THRID via a small _parse_gm_thrid helper.
• Dispatch: _dispatch_message derives thread_id:
  • f\"gthr-{thrid}\" when THRID is present, or
  • f\"mid-{sha1(message_id)[:12]}\" when it's missing (treats the message as its own thread root — always safe).
    Then passes thread_id to build_source(...) and stores _thread_context under the (sender, thread_id) tuple.
• Outbound plumbing: send() extracts thread_id from metadata[\"thread_id\"] and forwards it to _send_email. _send_email and _send_email_with_attachment accept thread_id and resolve via the helper added in fix(gateway): rekey email _thread_context by (sender, message_id) to prevent reply clobber #11418.
• send_image / send_document extension: these methods were missing the metadata parameter — without that fix, attachment replies would drop thread_id and fall back to the latest-subject heuristic. Signatures extended; metadata plumbed through send/_send_email_with_attachment.

Tests

• TestSessionKeying (14 new tests): default-mode behavior, mode validation, Gmail thread-id derivation (with and without THRID), same-thrid → same key, distinct-thrid → distinct keys, capability probe degradation, FETCH payload shape per mode, THRID parser unit tests, metadata propagation through send_image/send_document.
• TestEmailThreadIdSessionKey in tests/gateway/test_session.py (3 new tests): regression that build_session_key for an email DM with thread_id produces agent:main:email:dm:<addr>:<thread_id>.

Local verification:

• Email + session focused: `pytest tests/gateway/test_email.py tests/gateway/test_session.py` → 154 passed.
• Broader sweep (excluding pre-existing flaky env-dependent tests for matrix/slack-approval/whatsapp): 2816 passed, 0 failed.

Migration / risk

• Default stays sender. Existing installs: zero behavior change.
• Opting a profile into gmail_thread_id causes a one-time session fragmentation: the next message from a sender lands in a fresh thread session rather than the pre-existing flat one. Existing sessions remain accessible at their old keys; they simply stop accumulating new messages.
• Capability probe handles delegated-mailbox / Workspace-restricted edge cases gracefully (degrade with warning log). All Gmail / Google Workspace deployments advertise X-GM-EXT-1.

Test plan

• Unit tests pass (154/154 in scope).
• Smoke test on real Gmail IMAP: two back-to-back inbound emails on different topics from the same sender; confirm two distinct session keys (agent:main:email:dm:<addr>:gthr-<digits>) appear in the supervisord log.
• Multi-turn continuity: 3+ replies in one Gmail thread accumulate on a single session.
• /reset scope: resetting thread A leaves thread B's session untouched.
• Subject-change robustness: replying with a renamed subject still resolves to the same Gmail-assigned THRID.

[teknium1]

Thanks for the focused Gmail-thread proposal. The underlying issue is still present: current main stores email reply context only by sender at plugins/platforms/email/adapter.py:867-870 and emits no thread_id at plugins/platforms/email/adapter.py:872-878.

Problems

• This needs a current-main port: the PR changes gateway/platforms/email.py, but commit 560010547 moved the active adapter to plugins/platforms/email/adapter.py (whose EmailAdapter begins at line 422).
• Please cover the newer batch-image path. gateway/platforms/base.py:5075-5083 passes thread metadata into send_multiple_images, but plugins/platforms/email/adapter.py:1023-1031 currently drops it before _send_email_with_attachments, which reads sender-global context at line 1047. A Gmail-thread session could therefore still produce a batched-image reply threaded to the wrong message.

Suggested changes

• Salvage both stacked changes into the bundled plugin and update the email tests' imports.
• Carry metadata["thread_id"] through send_multiple_images and _send_email_with_attachments, with a regression test for that route.

Automated hermes-sweeper review.

[teknium1]

When salvaging this metadata plumbing onto current main, please carry the same thread_id through EmailAdapter.send_multiple_images() and _send_email_with_attachments(). The current base delivery path sends extracted image batches with metadata, so otherwise batched-image replies would still resolve sender-global reply context.

[GottZ]

This was generated by AI during triage.

Summary

Three PRs address or document the same sender-wide email-session defect. #11422 adds opt-in Gmail X-GM-THRID session isolation plus thread-scoped outbound context and tests, #27508 derives thread IDs from RFC 5322 headers but leaves outbound state sender-wide, and #11423 documents #11422's configuration without implementing it.

Related pull requests

• #11422 related — (+718/-21) — preferred salvage: This is the most complete diff against the reported cause, adding Gmail-thread session keys, tuple-keyed reply context, metadata plumbing, bounded state, and regression tests. Before merge, port it from the removed gateway adapter to plugins/platforms/email/adapter.py and preserve thread metadata through the current send_multiple_images path, as required by the keep_open contributor review.
• #11423 [closed] related — (+48/-0) — fold into implementation: This closed PR documents the proposed gmail_thread_id mode and its migration behavior, but the blocking contributor objection was valid because the option did not exist on current main. Its documentation remains relevant only if incorporated alongside the ported implementation rather than reopened or merged independently.
• #27508 duplicate — (+107/-4) — superseded by #11422: The RFC 5322 References-root approach targets the same session-collapse cause and improves References chaining, but its diff retains sender-wide outbound context, contains a broken Hermes-root fallback due to overwriting context before lookup, lacks regression tests, and targets the obsolete adapter path. Despite the keep_open reviews on #27508, the diff shows these unresolved correctness gaps, while #11422 already supplies the stronger context-keying and test foundation.

Duplicates

#11422 and #27508 substantially duplicate the per-thread email-session change, using Gmail X-GM-THRID and RFC 5322 root Message-IDs respectively; #11423 is the documentation companion to #11422.

Suggested consolidation

Merge #11422 after porting both stacked changes to plugins/platforms/email/adapter.py, carrying thread metadata through text, document, image, and batched-image sends, and folding in #11423's documentation; this explicitly resolves the contributor reviews rather than bypassing them. Then close #27508 as superseded by #11422, while considering its RFC 5322 References-chain extension as a separate follow-up for non-Gmail providers; keep #11423 closed because its blocking no-implementation objection is addressed by folding the docs into the implementation PR.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    subgraph Dup11422 ["PRs duplicating each other"]
        P11422["PR #11422 (open)"]
        P27508["PR #27508 (open)"]
    end
    class P11422 open
    class P27508 open
    class P11422 target
    click P11422 "https://github.com/NousResearch/hermes-agent/pull/11422"
    click P27508 "https://github.com/NousResearch/hermes-agent/pull/27508"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed or no verify verdict yet (state tag in the node label).

Cross-PR triage: Reviewed 3 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 54 kB of PR diffs, 13 kB of issue/PR text, 5 kB of discussion (5 comments), 1 verify verdict. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[rudidev08]

This only works with Gmail's own thread key (yes stating the obvious :) ), the X-GM-THRID IMAP extension. Would be better if this was universal. In my case Fastmail, but really any IMAP mailbox.

As written, non-Gmail IMAP logs a warning and falls back to per-sender keying, so the mode does nothing there. Provider-independent options: normalized-subject keying (#26277, attempted in #45600), or the thread_root follow-up you mention, walking Message-ID/References.