Skip to content

feat(simplex): add SimpleX Chat platform adapter - #4666

Closed
jooray wants to merge 5 commits into
NousResearch:mainfrom
jooray:fix/simplex-voice-file-transfer
Closed

feat(simplex): add SimpleX Chat platform adapter#4666
jooray wants to merge 5 commits into
NousResearch:mainfrom
jooray:fix/simplex-voice-file-transfer

Conversation

@jooray

@jooray jooray commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new platform adapter for SimpleX Chat — a privacy-focused messenger with no user identifiers. The adapter connects to a locally-running `simplex-chat` process via its WebSocket API.

Features

  • Direct message and group chat support
  • Image, audio, and file attachment handling
  • Auto-accept contact requests (configurable)
  • Group allowlist via `SIMPLEX_GROUP_ALLOWED`
  • Health monitor with automatic reconnection
  • Correct voice/file transfer handling (see below)
  • Outbound voice messages using SimpleX inline voice note format

Voice / File Transfer

SimpleX file transfers complete asynchronously. This PR handles three cases:

  1. `rcvFileDescrReady` event — fires before `newChatItems` for XFTP transfers. The adapter immediately sends `/freceive ` so the download starts, then waits for the chat item.

  2. `newChatItems` with pending file — when a chat item arrives with a `fileId` but no `filePath` yet, the adapter stores it in `_pending_file_transfers` and sends `/freceive ` (fire-and-forget, since simplex-chat never sends a corr-id reply for this command — blocking on it would freeze the event loop for 30s).

  3. `rcvFileComplete` — when the download finishes, the pending item is retrieved, the real `filePath` is patched in, and the message is dispatched for processing.

Outbound Voice Messages

When sending audio replies, `send_voice()` uses the `/_send json` API with `msgContent.type: "voice"` and a `fileSource`, so recipients see an inline voice note player rather than a generic downloadable file attachment.

Configuration

platforms:
  simplex:
    ws_url: ws://127.0.0.1:5225
    auto_accept: true

Environment variables:

  • `SIMPLEX_WS_URL` — WebSocket URL (required)
  • `SIMPLEX_HOME_CHANNEL` — home channel contact/group ID
  • `SIMPLEX_HOME_CHANNEL_NAME` — display name
  • `SIMPLEX_ALLOWED_USERS` — comma-separated contact ID allowlist
  • `SIMPLEX_ALLOW_ALL_USERS` — set true to allow all contacts
  • `SIMPLEX_GROUP_ALLOWED` — comma-separated group IDs (or `*` for all)

Adds full SimpleX Chat support as a new platform adapter using the
simplex-chat WebSocket API.

Features:
- Direct message and group chat support
- Image, audio, and file attachment handling
- Auto-accept contact requests (configurable)
- Group allowlist via SIMPLEX_GROUP_ALLOWED
- Health monitor with automatic reconnection
- Correct deferred handling of voice/file transfers:
  on newChatItems the filePath may be null while the transfer is in
  progress; the adapter now issues /fa <fileId>, stores the pending
  chat item, and dispatches it once rcvFileComplete fires with the
  real path.

Configuration via environment variables:
  SIMPLEX_WS_URL            WebSocket URL of simplex-chat process
  SIMPLEX_AUTO_ACCEPT       Auto-accept contact requests (default true)
  SIMPLEX_HOME_CHANNEL      Home channel contact/group ID
  SIMPLEX_HOME_CHANNEL_NAME Display name for home channel
  SIMPLEX_ALLOWED_USERS     Comma-separated allowlist of contact IDs
  SIMPLEX_ALLOW_ALL_USERS   Set true to allow all contacts
  SIMPLEX_GROUP_ALLOWED     Comma-separated group IDs to allow
Copilot AI review requested due to automatic review settings April 2, 2026 22:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new SimpleX Chat platform adapter so the gateway can communicate with a locally running simplex-chat process via its WebSocket API, including DM/group messaging and basic attachment handling.

Changes:

  • Introduces SimplexAdapter with a WebSocket listener, reconnection health monitor, and file-transfer deferral logic for voice/media.
  • Wires SimpleX into gateway startup/authorization via new SIMPLEX_* allowlist env vars and adapter creation in run.py.
  • Extends configuration to enable/configure SimpleX via environment variables and include it in “connected platforms” detection.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File Description
gateway/run.py Registers SimpleX allowlist env vars and adds adapter creation for Platform.SIMPLEX.
gateway/platforms/simplex.py New SimpleX WebSocket-based platform adapter with message handling, send APIs, and file-transfer deferral.
gateway/config.py Adds Platform.SIMPLEX, enables env-based config overrides, and marks SimpleX as connected when ws_url is set.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +231 to +244
while self._running:
try:
logger.debug("SimpleX WS: connecting to %s", self.ws_url)
async with websockets.connect(
self.ws_url,
ping_interval=20,
ping_timeout=20,
close_timeout=10,
) as ws:
self._ws = ws
backoff = WS_RETRY_DELAY_INITIAL # Reset on successful connection
self._last_ws_activity = time.time()
logger.info("SimpleX WS: connected")

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the reconnect loop, adapter runtime status is only marked connected once in connect() and never marked disconnected/connected on subsequent WS drops/reconnects. This can leave the gateway reporting the platform as connected while the socket is down (and vice versa). Consider calling _mark_disconnected() when the WS connection closes and _mark_connected() after a successful reconnect (or otherwise updating runtime status inside _ws_listener).

Copilot uses AI. Check for mistakes.
Comment on lines +130 to +133
# Track sent message IDs to prevent echo loops
self._recent_sent_ids: set = set()
self._max_recent_ids = 50

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_recent_sent_ids/_max_recent_ids are initialized but never used (echo suppression currently relies on chatDir filtering). This unused state adds complexity and can mislead future readers—either remove it or implement the intended recent-ID tracking so it actually prevents echo loops.

Suggested change
# Track sent message IDs to prevent echo loops
self._recent_sent_ids: set = set()
self._max_recent_ids = 50

Copilot uses AI. Check for mistakes.
Comment on lines +615 to +626
"""Send a text message."""
if chat_id.startswith("group:"):
group_id = chat_id[6:]
# Use the /_send structured API with numeric group ID (#<id>) to
# avoid ambiguity when multiple groups share the same display name.
# The plain '#<name>' chat command looks up by display name and
# fails if the name is not unique or matches the wrong group.
escaped = content.replace('"', '\\"')
command = f'/_send #{group_id} json [{{"msgContent":{{"type":"text","text":"{escaped}"}}}}]'
else:
command = f"@{chat_id} {content}"

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

send() defines MAX_MESSAGE_LENGTH=8000 but never enforces it. Other adapters split long messages via BasePlatformAdapter.truncate_message(); without chunking, SimpleX will likely reject oversized outputs. Recommend formatting + chunking content to MAX_MESSAGE_LENGTH and sending sequential chunks (respecting reply_to_mode if applicable).

Copilot uses AI. Check for mistakes.
Comment thread gateway/platforms/simplex.py Outdated
Comment on lines +622 to +631
escaped = content.replace('"', '\\"')
command = f'/_send #{group_id} json [{{"msgContent":{{"type":"text","text":"{escaped}"}}}}]'
else:
command = f"@{chat_id} {content}"

# SimpleX CLI uses @ prefix for DMs and # prefix for groups:
# @<contactId> <message> or #<groupId> <message>
# The structured API form is:
# /_send @<contactId> json [{"msgContent":{"type":"text","text":"..."}}]
# The simpler chat command form also works for plain text.

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The group send path builds a JSON payload by string interpolation and only escapes double quotes. Content containing backslashes, newlines, control characters, or " sequences can produce invalid JSON or unintended payloads. Build the JSON part with json.dumps (or reuse the structured /_send API builder) instead of manual escaping, and prefer the same structured form for DMs for consistent escaping.

Suggested change
escaped = content.replace('"', '\\"')
command = f'/_send #{group_id} json [{{"msgContent":{{"type":"text","text":"{escaped}"}}}}]'
else:
command = f"@{chat_id} {content}"
# SimpleX CLI uses @ prefix for DMs and # prefix for groups:
# @<contactId> <message> or #<groupId> <message>
# The structured API form is:
# /_send @<contactId> json [{"msgContent":{"type":"text","text":"..."}}]
# The simpler chat command form also works for plain text.
target = f"#{group_id}"
else:
target = f"@{chat_id}"
# Build the message payload structurally and serialize it so content
# is escaped correctly for quotes, backslashes, newlines, and other
# control characters.
payload = json.dumps(
[
{
"msgContent": {
"type": "text",
"text": content,
}
}
]
)
command = f"/_send {target} json {payload}"
# SimpleX CLI uses @ prefix for DMs and # prefix for groups.
# Use the structured API form consistently for both:
# /_send @<contactId> json [{"msgContent":{"type":"text","text":"..."}}]
# /_send #<groupId> json [{"msgContent":{"type":"text","text":"..."}}]

Copilot uses AI. Check for mistakes.
Comment on lines +489 to +497
if not file_path and _is_audio_ext(ext) and file_id is not None:
# File transfer not yet complete — accept and wait for rcvFileComplete
logger.info(
"SimpleX: voice message file %d not yet received, accepting transfer",
file_id,
)
self._pending_file_transfers[file_id] = chat_item
await self._send_command(f"/fa {file_id}")
return

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending file transfers are stored in _pending_file_transfers but never expire if simplex-chat never emits rcvFileComplete (or if /fa fails). This can grow unbounded over time. Consider adding a TTL/size cap and periodic cleanup (e.g., drop oldest entries after N minutes or when dict exceeds a limit) and/or removing the pending entry if the /fa command errors.

Copilot uses AI. Check for mistakes.
Comment on lines +679 to +685
# Send file via simplex-chat command
if chat_id.startswith("group:"):
group_id = chat_id[6:]
command = f"/f #{group_id} {file_path}"
else:
command = f"/f @{chat_id} {file_path}"

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

send_image()/send_document() build /f ... {file_path} commands without quoting/escaping the path. If the cached/local path contains spaces or shell-significant characters, simplex-chat may parse it incorrectly and the send will fail. Quote/escape the file path (and verify the simplex-chat command parser rules) before sending.

Copilot uses AI. Check for mistakes.
Comment on lines +686 to +694
result = await self._send_command(command)

# Send caption as a separate message if provided
if caption and result is not None:
await self.send(chat_id, caption)

if result is not None:
return SendResult(success=True)
return SendResult(success=False, error="Failed to send image")

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

send_image()/send_document() treat a missing correlated response (result is None) as a hard failure, but send() treats the same situation as success when the WS is still connected. This inconsistency can cause false negatives (and skip sending captions) if simplex-chat doesn't emit a correlation response for /f. Consider applying the same fallback success logic used in send() when self._ws is still connected.

Copilot uses AI. Check for mistakes.
Comment on lines +356 to +374
# Handle file transfer completion — deliver pending voice messages
if resp_type == "rcvFileComplete":
chat_item = resp.get("chatItem", {})
chat_item_data = chat_item.get("chatItem", {})
file_info = chat_item_data.get("file", {})
file_id = file_info.get("fileId") if isinstance(file_info, dict) else None
if file_id is not None and file_id in self._pending_file_transfers:
pending = self._pending_file_transfers.pop(file_id)
file_source = file_info.get("fileSource", {})
file_path = file_source.get("filePath") if isinstance(file_source, dict) else None
if file_path:
pending_item_data = pending.get("chatItem", {})
pending_item_data.setdefault("file", {})["fileSource"] = {"filePath": file_path}
pending["chatItem"] = pending_item_data
try:
await self._handle_chat_item(pending)
except Exception:
logger.exception("SimpleX: error processing deferred voice message")
return

Copilot AI Apr 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deferred voice/file-transfer handling (storing chat items on newChatItems and dispatching on rcvFileComplete) introduces non-trivial event ordering/state logic but has no automated tests. Add unit tests that cover: (1) receiving a chat item with fileId and null filePath triggers /fa and queues the item, (2) rcvFileComplete with matching fileId dispatches exactly once with the resolved filePath, and (3) cleanup behavior for unmatched/expired pending transfers.

Copilot uses AI. Check for mistakes.
jooray added 4 commits April 3, 2026 00:37
…age support

- Replace /fa with /freceive (correct simplex-chat CLI command)
- Use fire-and-forget for /freceive (no corr-id reply ever arrives;
  the old _send_command approach blocked the event loop for 30s)
- Handle rcvFileDescrReady event to accept file transfers that arrive
  before the newChatItems event (common for XFTP transfers)
- Add send_voice() method that sends audio as type:"voice" via the
  /_send JSON API so recipients see an inline voice note player
  instead of a generic downloadable file attachment
Add a new 'script' TTS provider that delegates synthesis to any
user-specified external command. The command receives --text and --out
arguments and is expected to produce an audio file at the output path.

Config example:
  tts:
    provider: script
    script:
      command: /path/to/tts-wrapper.sh
      args: [--voice, default]
      timeout: 120

This enables integration with custom or self-hosted TTS systems without
requiring changes to hermes-agent source code.
Group send was using '#<name>' which fails when multiple groups share the
same display name. Switch to '/_send #<id>' which uses the numeric group
ID and is unambiguous.

Also add 'simplex' entry to PLATFORMS dict in tools_config.py to fix
KeyError when resolving default toolset for simplex platform.
The previous f-string approach only escaped double quotes but not
newlines, backslashes, or other control characters. Multi-line
agent responses (e.g. nightly reports) would produce malformed JSON
causing simplex-chat to return 'commandError: Failed reading: empty'.
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery labels May 1, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #2558 and #4870 — both are also SimpleX Chat adapter PRs. Please coordinate to avoid duplicated effort.

@alt-glitch alt-glitch added the duplicate This issue or pull request already exists label May 1, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #2558 and #4870 — both are also SimpleX Chat adapter PRs.

brandon-btcgroup added a commit to brandon-btcgroup/hermes-agent that referenced this pull request May 29, 2026
…emon

Four independent bugs prevented end-to-end SimpleX messaging against a
real simplex-chat daemon:

1. WS listener never sent /_start, so the daemon stored inbound
   messages but never pushed events to the subscriber.
2. newChatItems batch events nest `chatItems` under `resp`, not at the
   top level — every batched message was silently dropped.
3. Group sender was read from the legacy `chatItemMember` key; current
   simplex-chat reports it under `chatItem.chatDir.groupMember`, so
   sender_id fell back to the chat_id and failed allowlist matching.
   Falls back to chatItemMember for older payloads.
4. Outbound send used `@[id]`/`#[id]` bracket syntax the daemon reads
   as a literal contact name; switched to the `/_send @id text` /
   `/_send #id text` API form in both send() and _standalone_send().

Bugs 1, 2 and 4 mirror upstream PR NousResearch#26433 (issue NousResearch#30150). Bug 3 is not
covered there; the chatDir.groupMember approach matches PRs NousResearch#4666/NousResearch#27978.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
brandon-btcgroup added a commit to brandon-btcgroup/hermes-agent that referenced this pull request May 30, 2026
Switch send() and _standalone_send() from the /_send <ref> text shorthand
to the structured /_send <ref> json <ComposedMessage[]> form for both group
and DM sends. The text shorthand truncates the body at the first newline, so
multi-line agent replies were silently cut off; the json form escapes
newlines, backslashes and other special characters correctly.

This also aligns outbound text with the approach taken upstream in NousResearch#4666 /
NousResearch#27978, easing a future rebase. Adds a regression test asserting a multi-line
body is escaped rather than truncated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
brandon-btcgroup added a commit to brandon-btcgroup/hermes-agent that referenced this pull request May 30, 2026
…ening

Adds a 2026-05-29 part-2 update: sender fix upstreamed as issue NousResearch#35045 / PR
NousResearch#35046; rationale for NOT opening a group-send PR (already covered by
NousResearch#4666/NousResearch#27978 with a more robust json form); production branch hardened to
the /_send <ref> json form (commit d31a043) to stop multi-line reply
truncation; and a rebase note for the eventual NousResearch#26433 merge conflict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@brandon-btcgroup

Copy link
Copy Markdown

Field report in support of this approach — with the caveat that I didn't run
this exact branch; I independently implemented the same techniques in my own
SimpleX deployment and have them running in production, so this is
corroboration rather than a test of your PR.

The two choices this PR makes both held up live:

  • Group send via /_send #<id> json [{"msgContent": …}]. I landed on the
    same form. Your rationale — numeric id to avoid ambiguity when group display
    names aren't unique, and json to escape newlines/backslashes — is exactly
    right in practice. A 3-line reply (a haiku) sent to a group arrives intact
    on the phone; the /_send #<id> text <body> shorthand I used first
    truncated the body at the first newline, so multi-line model replies were
    silently cut off. I also extended the json form to the DM path
    (/_send @<id> json …) for the same escaping benefit.
  • Group sender from chatItem.chatDir.groupMember. Current simplex-chat
    reports the member there, not under the legacy chatItemMember key; reading
    chatDir.groupMember (with memberProfile for the display name) gives the
    correct member id, which downstream allowlist matching needs.

#27978 takes the same /_send #<id> json approach. From a live deployment,
this is the form that works end-to-end. Happy to share more detail if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants