feat(email): per-thread session routing + configurable display name

[andrewKnown]

Summary

The Email platform adapter previously used the sender's email address as the session key (chat_id), meaning all emails from the same person shared a single conversation. This caused context pollution when an email agent handled diverse requests (research, calendar, scheduling) from the same sender.

Changes

1. Per-thread session routing (session_routing)

New config option with two modes:

• thread (default): Each email subject gets its own session. Replies within the same thread (same subject, stripped of Re:/Fwd:) stay in the same session.
• sender: One session per sender address (legacy behaviour).

The normalized subject is used as chat_id in build_source(). The real recipient address is stored in _thread_context and resolved back in send() when replying.

2. Configurable display name (display_name)

Adds a display_name option for the From: header in outgoing emails. Previously the From header was the bare EMAIL_ADDRESS with no display name.

• If set: From: "Iris Sloane" <iris@known.ltd>
• If not set: From: iris@known.ltd (legacy behaviour)

Configuration

platforms:
  email:
    session_routing: thread    # or: sender (default: thread)
    display_name: 'Iris Sloane' # optional, defaults to bare address

Or via environment variables:

EMAIL_SESSION_ROUTING=thread
EMAIL_DISPLAY_NAME=Iris Sloane

Backward compatibility

• session_routing: sender preserves the exact legacy behaviour
• display_name unset = bare address (same as before)
• All changes are opt-in via config; existing setups with no config changes get thread mode by default (the better behaviour)

Send method resolution

When session_routing is thread, chat_id is the subject string (not an email address). The send() method resolves the real recipient via:

1. Look up sender_addr in _thread_context by chat_id
2. Scan all thread contexts for any stored sender_addr
3. Fall back to EMAIL_HOME_ADDRESS env var
4. Fall back to chat_id itself if it contains @

This ensures replies always go to the right recipient, even for Kanban notification deliveries that don't have thread context set.

Files changed

• plugins/platforms/email/adapter.py — session routing logic, display name, send resolution
• gateway/config.py — wire EMAIL_SESSION_ROUTING and EMAIL_DISPLAY_NAME env vars

---

Follow-up (commit 52b5410): one threaded reply per turn

Dogfooding the branch on a live mailbox surfaced three defects in the reply
path that the original commits did not cover. An email answer that carried
attachments did not look like a person's reply: it arrived as two messages, the
one with the files landed in an unrelated conversation, and everyone who had
been cc'd was dropped.

1. Attachment mail built its own reply headers

_send_email_with_attachments and _send_email_with_attachment each resolved
context with a bare self._thread_context.get(to_addr) lookup. Under
session_routing: thread that always misses, because keys are
"sender::thread_id". So attachment mail went out as:

header	before	after
Subject	Re: Hermes Agent	Re: <the actual thread subject>
In-Reply-To	(absent)	parent Message-ID
References	(absent)	full inbound chain + parent
Cc	(dropped)	inbound To/Cc chain preserved

Mail clients had no threading headers to work with, so every attachment mail
collapsed into one unrelated conversation.

Subject / Cc / In-Reply-To / References now come from a single
_apply_reply_headers() that every send path calls — the divergence was the
bug, so the fix is one authority rather than three copies. References now
extends the inbound chain (capped at 20, root + most recent) instead of being
reset to just the parent, which had made long threads drift apart in Outlook.
An empty context logs a warning and attachment mail with no parent message-id
logs an error, rather than silently emitting a generic subject.

2. A reply to the agent's own mail landed on the wrong thread

_msgid_context only ever indexed inbound Message-IDs. A reply to one of
the agent's own mails carries In-Reply-To: <hermes-...>, which missed the
index and fell through to the most-recent-thread scan in _context_for_send —
answering on a different thread, with that thread's Cc list. Outbound
Message-IDs are now registered against their thread as they are sent, and the
thread's stored anchor advances so the next reply chains correctly.

3. One turn produced several emails

The gateway dispatches one answer as send() for the body, then
send_multiple_images() once for an image batch and send_document() once
per remaining file, with human_delay sleeps in between. On chat those are
separate bubbles; on email each is a separate message in the recipient's thread.
Folding the body into just the first attachment would still leave one email per
subsequent file, so outbound parts are buffered per thread and flushed as a
single MIME once the turn goes idle.

• fold_window_seconds (default 12) — idle debounce, re-armed by each new part.
  Measured body→attachment gaps on real turns were 4.1–7.9s.
• fold_max_wait_seconds (default 90) — ceiling from the first part, so a long
  attachment loop cannot stall a reply.
• disconnect() flushes anything still buffered, so a restart mid-turn cannot
  silently drop a reply (worse than sending two).

Also in this commit

• Thread state persists to <HERMES_HOME>/state/email_thread_state.json
  (atomic write, 0600). It was memory-only, so every restart made the next
  reply on a live thread lose its subject and Cc — the reported symptom,
  reappearing because of a deploy. Resolved via get_hermes_home() so it is
  per-profile and sandboxed under test rather than one shared file.
• Identical attachments are not re-sent on later turns of the same thread
  (content sha256, thread-scoped, so a genuinely revised file still goes out).

Configuration

platforms:
  email:
    session_routing: thread
    display_name: 'Iris Sloane'
    fold_window_seconds: 12      # optional
    fold_max_wait_seconds: 90    # optional

Tests

78 passed in tests/gateway/test_email.py,
test_email_robustness.py, test_email_secret_scope.py.

Four existing send/send_document tests asserted the old synchronous
contract (SMTP called during send()); they now flush before asserting the
wire message. New coverage:

• test_one_turn_body_plus_files_is_a_single_email — body + 3 files produce
  one send_message call carrying the body, all three attachments, and the
  thread's Subject/Cc/In-Reply-To; asserts nothing is on the wire before
  the flush.
• test_reply_to_own_message_resolves_original_thread — with a newer decoy
  thread present for the same address, a reply to an outbound <hermes-...>
  id still resolves the original thread and its Cc list. Without the decoy this
  passes by accident, which is how the bug went undetected.
• test_thread_state_survives_restart — state written, reloaded into a fresh
  adapter, reply still resolves subject + Cc.

Verified end-to-end against a real Gmail mailbox: a 6-file request produced
exactly one message in [Gmail]/Sent Mail with Subject: Re: <thread>,
In-Reply-To equal to the inbound Message-ID, Cc preserved, all six
attachments and a non-empty body; and a reply to the agent's own message on an
older thread — with a newer thread live for the same sender — stayed on the
correct thread.

[teknium1]

Thanks for addressing the real sender-scoped-session limitation: current main does route Email DMs from one sender through one chat_id (plugins/platforms/email/adapter.py:866-878).

Problems

• The PR uses the normalized subject as the entire DM chat_id (plugins/platforms/email/adapter.py:898-918). build_session_key() uses that value directly for DMs (gateway/session.py:907-915), so two allowed senders with the same subject share a session and overwrite reply context.
• The fallback at plugins/platforms/email/adapter.py:954-959 chooses the first stored sender for an unresolved thread. That can deliver a reply to an unrelated recipient.
• Attachment paths bypass the resolver: send_multiple_images() and send_document() pass chat_id directly to SMTP helpers (plugins/platforms/email/adapter.py:1100-1105, 1177-1184), whose To headers receive that value (1120, 1200). In thread mode it is a subject, not an address.
• The PR adds no tests or documentation updates for the new routing/configuration behavior.

Suggested changes

• Preserve the recipient as chat_id; use a recipient-scoped, stable thread identifier separately for session isolation.
• Fail closed for unresolved reply targets; do not select an arbitrary cached sender.
• Route text and attachment sends through one recipient/context resolver, then add coverage for collisions, unknown threads, attachments, compatibility mode, and display names.

Automated hermes-sweeper review.

[teknium1]

A subject alone cannot identify a private email thread: two allowed senders can both send Re: Invoice, and build_session_key() uses this chat_id directly for Email DMs. That merges their agent history and overwrites _thread_context; retain the recipient identity and use a separate recipient-scoped thread identifier.

[teknium1]

This fallback selects the first cached sender for any unresolved chat_id, which can send a response to an unrelated recipient. Fail closed here; only use an explicit home target for proactive delivery.

[andrewKnown]

@teknium1 reshaped per your review — no more composite chat_id.

What changed: chat_id is now always the sender's mailbox and the thread key moved to SessionSource.thread_id, so build_session_key() isolates per (sender, thread) exactly like Telegram topics/Discord threads (gateway/session.py DM path already appends thread_id — no gateway changes needed).

Your four points:

1. Recipient identity — chat_id is the address; subject/composite no longer touches session identity.
2. Fail-closed — _resolve_recipient() only ever returns chat_id (an address) or explicit EMAIL_HOME_ADDRESS; no cached-sender scanning.
3. Attachments — text/images/documents all resolve through _resolve_recipient(chat_id, metadata=…, reply_to=…) and pass an explicit ctx; To: is provably an address (regression test).
4. Tests/docs — 105 passing incl. cross-sender isolation, same-sender multi-thread isolation, context-leak, bounded trim, metadata + Message-ID send resolution; new "Session Routing" section in website/docs/user-guide/messaging/email.md.

Improvements beyond the review: Message-ID anchor index (replies thread correctly even when a tool sends without thread metadata) and bounded thread-context maps (2000, oldest-half trim) for long-lived gateways.

Context resolution order at send time: metadata.thread_id (gateway already passes it through _thread_metadata_for_source) → Message-ID anchor → sender-mode key → latest thread for the same mailbox (subject/headers only; never changes recipient).

Migration note: email session key format changes one time (per-sender → per-thread sessions), which is the intended behavior of the feature.

[andrewKnown]

Follow-up found during live end-to-end verification of the reshape on a real gateway: two silent-drop bugs in the pre-dispatch sender gate (this PR's code path):

1. Allow-all precedence — allow-all was only consulted when EMAIL_ALLOWED_USERS was empty; a stale allowlist silently dropped every sender. Now matches authz_mixin.py (allow-all first).
2. @domain tokens — documented allowlist format (@known.ltd) never matched; membership was exact-address only. Now both token types work.

The failure mode is nasty: IMAP fetch marks the mail \Seen, dispatch drops at logger.debug, no New message log, no reply, poller looks healthy. Fixed + 3 regression tests (108 passing).

[andrewKnown]

Rebased onto current main (3be565fbd) and re-verified: 108 tests passing, all patch markers present, and a live mail round-trip on a fleet gateway (inbound dispatch → per-thread session → From: Iris Sloane <…> reply with correct per-thread subject + In-Reply-To). No code changes vs the previous head — same feature, fresh base.

[andrewKnown]

Contribution packaging update

This is no longer an always-on Iris-only patch. It's an opt-in platform feature with first-class switches:

Surface	Switch
config.yaml	platforms.email.session_routing: thread|sender
config.yaml	platforms.email.display_name: "…"
Env	EMAIL_SESSION_ROUTING, EMAIL_DISPLAY_NAME
hermes gateway setup → Email	Interactive prompts for routing + display name
Dashboard Channels env editor	Same env vars labeled

Default is sender (stock Hermes: one session per person). No session-key migration for existing users until they opt into thread.

Recommended for email agents:

platforms:
  email:
    session_routing: thread
    display_name: Iris Sloane

Core model still matches the Jul 16 review: chat_id = mailbox, thread_id = thread, fail-closed delivery, attachments unified, tests+docs.

109 tests passing. Live-verified on a fleet gateway with real IMAP/SMTP mail.

[andrewKnown]

Hi @teknium1 — just following up on this one. Since the July 16 review I've reworked the PR along all four points and verified it end-to-end:

1. Recipient identity / thread isolation — chat_id is now always the sender's mailbox; the thread key moved to SessionSource.thread_id, so build_session_key() isolates per (sender, thread) exactly like Telegram topics/Discord threads. No subject-based chat_id.
2. Fail-closed resolve — unresolved reply targets no longer fall back to an arbitrary cached sender; the callers bail with a logged error instead.
3. Attachment sends — send_multiple_images() and send_document() now both route through the same _resolve_recipient() as text sends, so To: is always an address, never a thread/subject.
4. Tests + docs — added coverage in tests/gateway/test_email.py (+473 lines) for collisions, unknown threads, attachments, compatibility (sender) mode, and display names, plus website/docs/user-guide/messaging/email.md.

Also folded in two silent-drop fixes surfaced during live end-to-end verification (allow-all precedence and @domain token matching). Rebased onto current main (3be565f), 108 tests passing, and a live mail round-trip confirmed inbound dispatch → per-thread session → From: Iris Sloane <…> reply with correct per-thread subject + In-Reply-To.

I believe all review feedback is now addressed — would appreciate a re-review whenever you have a moment. Happy to make any further adjustments.

[andrewKnown]

Status update (conflicts cleared)

Rebased feat/email-thread-sessions onto current main and force-pushed (0b1eca1f → 3bc612966).

Conflicts resolved

• plugins/platforms/email/adapter.py — kept allow-all precedence + @domain allowlist gate
• tests/gateway/test_email.py — kept PR test coverage
• gateway/config.py — kept main structure
• hermes_cli/web_server.py — kept main file (large drift); re-applied only intentional Email dashboard env labels (EMAIL_SESSION_ROUTING, EMAIL_DISPLAY_NAME, EMAIL_ALLOW_ALL_USERS, etc.)

Local verification (canonical scripts/run_tests.sh)

• tests/gateway/test_email.py — 65 passed
• tests/hermes_cli/test_web_server*.py (+ dashboard dist) — 160 passed
• 225 total, 0 failed

PR should no longer show “This branch has conflicts.” Ready for re-review whenever you have a moment, @teknium1.

[andrewKnown]

Pushed 52b5410 — follow-up fixing the reply path, found by running this branch on a live mailbox.

The original commits fixed threading for text replies only. Any answer that carried attachments still went out as two messages: the one with the files had no In-Reply-To/References and a Re: Hermes Agent subject (the bare _thread_context.get(to_addr) lookup always misses under session_routing: thread), so it landed in an unrelated conversation with the Cc chain dropped.

Three fixes:

1. _apply_reply_headers() is now the only place Subject/Cc/In-Reply-To/References are set, and every send path calls it. References extends the inbound chain instead of being reset to the parent.
2. Outbound Message-IDs are registered in _msgid_context. They previously weren't, so a reply to the agent's own mail missed the index and fell through to the most-recent-thread scan — replying on the wrong thread with the wrong Cc.
3. One turn now produces one email. The gateway calls send() for the body and send_document() once per file, so parts are buffered per thread and flushed as a single MIME when the turn goes idle (fold_window_seconds, default 12s; measured body→attachment gaps were 4.1–7.9s).

Thread state also persists to <HERMES_HOME>/state/email_thread_state.json — it was memory-only, so a restart made the next reply on a live thread lose its subject and Cc.

78 passed. Four existing send tests asserted the old synchronous contract and now flush before asserting the wire message; new tests cover one-email folding, reply-to-own-message resolution with a decoy thread present, and state surviving a restart. Also verified end-to-end against a real Gmail mailbox (6 files → exactly one Sent message with correct Subject/Cc/In-Reply-To and a non-empty body).

[kiwipaulrob]

+1 for this PR. Can it be added to code?

[kiwipaulrob]

@andrewKnown Friendly ping on this one 🙂 It closes two long-standing issues (#26277, #27804) and is still the most complete fix for email session/thread isolation. It's been about three weeks since the last status update — is there anything needed from our side to move it toward review?

For triage context: my two smaller fixes in the same area are also still open — #56581 (strip compound channel prefix before SMTP To) and #58055 (extract SMTP recipient from session keys). If it helps review, I'm happy to fold either into this branch, or rebase them once this lands.

[santiagomalter]

Not stacking on the subject-as-chat_id approach (sweeper's SMTP To: / cross-sender collision notes). Filing a plugin-layer fix on #26277 that keeps chat_id as the sender address and puts isolation in source.thread_id instead, per the guidance here and on #27811.