Repository navigation
Conversation
alt-glitch
added
type/feature
FarelRA
force-pushed
the
feat/waha-platform-plugin
branch
from
September 6, 2026 13:33
91a31b1 to
ba6fe09
Compare
PR #104247 — feat: WhatsApp via WAHA platform plugin (third WhatsApp transport)Big PR (~213KB, 30 files): new What looks right
Non-blocking points (large-PR focus areas)
|
FarelRA
force-pushed
the
feat/waha-platform-plugin
branch
from
September 6, 2026 16:36
ba6fe09 to
bc0fcd6
Compare
Author
New in this push + responses to the automated reviewRebased on the reworked #104245 (rolling observed-context compaction, platform-neutral Point-by-point on the automated review:
Test suite on this branch: 21 WAHA tests, full gateway + plugins + conformance green (9646 passed, 0 failed). |
FarelRA
force-pushed
the
feat/waha-platform-plugin
branch
2 times, most recently
from
September 8, 2026 00:58
684802b to
5349304
Compare
…t-py) Replace the regex transcoder in WhatsAppBehaviorMixin.format_message with an AST-based CommonMark renderer so every markdown construct is handled — emphasis, headings, fenced code (language tag kept as a caption), GFM tables, links, images, lists, blockquotes, horizontal rules and HTML — with nothing silently dropped. - gateway/platforms/whatsapp_renderer.py: CommonMarkToWhatsApp walks the markdown-it-py token stream and emits WhatsApp's native dialect (*bold* / _italic_ / ~strike~ / backticks). Constructs WhatsApp cannot render are down-rendered, never discarded. Prose containing literal * _ ~ backtick passes through with stray open→close pairs broken by space-flanking the offending open (WhatsApp pairs delimiters per line only; a lone ~ meaning 'approx' would otherwise strike on-device). - gateway/platforms/whatsapp_unicode.py: optional Unicode font styling (default ON, WHATSAPP_UNICODE_FORMATTING=false to disable) recovers heading hierarchy native WhatsApp flattens; unassigned math glyphs (U+1D455 → ℎ) are substituted to avoid tofu. - Tables default to a wrap-safe key/value flatten; aligned monospace fence stays available via WHATSAPP_TABLE_MODE=monospace. - MAX_MESSAGE_LENGTH raised 4096 → 65,536 (WhatsApp's real per-message cap): send and edit both stay ONE bubble — no UX fragmentation, and a split edit can no longer surface chunk 2+ as unlinked duplicates. - Declare markdown-it-py==4.0.0 explicitly (already resolved transitively via rich; the renderer imports it directly) and exempt the exact pin from exclude-newer per the release-day brick pattern.
send() already converted markdown, but three paths sent raw text: - edit_message: streaming edits delivered the accumulated buffer without conversion, so previews AND the final message showed raw # / ** markers. Also surface the bridge's messageIds when an oversized edit splits, so the stream consumer re-targets subsequent edits at the last visible bubble (mirrors send()'s continuation contract) instead of reporting the original id while chunks 2+ landed as unlinked new messages. - media captions (bridge /send-media and Cloud API media block): captions are rendered with WhatsApp formatting on-device, so a raw markdown caption leaked # / ** onto the media bubble. - _standalone_send (cron delivery): runs in a separate process and bypasses adapter.send(); body text and single-file captions now go through format_message with the same never-raise guard as the gateway path.
… 65,536 cap - splitLongMessage moves to bridge_helpers.js and counts/splits by code points (codePointLength): astral characters (emoji, the Unicode math glyphs the formatter emits) count once and can never be split mid- surrogate-pair. The legacy UTF-16 splitter counted them at 2x, so a ~3000-char heading-formatted edit crossed a 4096 threshold and split. - /send and /edit both split at WhatsApp's real per-message cap (WHATSAPP_MAX_MESSAGE_LENGTH, default 65,536 code points). There is no smaller edit cap: an edit past a smaller limit sends chunk 2+ as brand-new unlinked messages — the duplicate-bubble regression — and a smaller send cap just fragments what WhatsApp renders fine as one message. /edit keeps reporting messageIds so the adapter can surface continuation ids.
- test_whatsapp_formatting.py: full-construct CommonMark→WhatsApp matrix (inline emphasis, headings incl. setext, fenced code captions, links, images, lists, task lists, blockquotes, HR, tables both modes, HTML, entities, mixed documents, unterminated-fence degradation), stray delimiter flanking (per-line pairing, heading/list contexts, star and underscore literals never flanked), GlyphStyler + unassigned-glyph substitution, unicode_formatting toggle (env + config.extra), format+truncate pipeline (balanced fences per chunk at the 65,536 cap), and every-send-path-formats coverage (streaming edits, media captions, standalone/cron) incl. edit continuation ids. - test_whatsapp_cloud.py: Cloud API caption conversion + long-message chunking at the real cap regressions. - test_whatsapp_send_message_media.py: standalone body + caption conversion regressions. - test_whatsapp_native_delivery.py: sanitizer output is fully consumed as paragraph indentation now (clean 'text', not ' text'). - generate_conformance_vectors.py: pin the whatsapp oracle to the default pipeline config (committed vectors must not depend on the developer's WHATSAPP_UNICODE_FORMATTING), add tilde-approx / tilde-space-safe / tilde-mixed / header-h3-italic-h grid vectors, table vectors expect parity under the default flatten mode, backslash-in-code documented as divergent (invalid CommonMark kept losslessly). - Regenerated conformance vectors (whatsapp 4 new; telegram/slack/discord re-stamped with the generator run).
…nd cap register() still advertised max_message_length=4096 while the adapter and formatter run at the 65,536 code-point cap; setup surfaces derived from the registry understated the real limit.
The formatter raises the per-message limit from the legacy 4,096 UX split to WhatsApp's actual cap; the docs section still described the old behavior.
…ng compaction - gateway/observed_context.py (new platform-neutral sibling): marker constants (single source for Telegram + WhatsApp prompts), injection limits (gateway.observed_context_max_chars default 512,000 ~ 128K tokens for common 256K-window models; observed_context_max_rows default 4,000), read assembly, and rolling compaction. - gateway/run.py: generalize the Telegram-only observed-context marker check to the shared predicate so WhatsApp observed rows are separated from replayable history the same way Telegram's are (previously they replayed as ordinary user turns); header text is platform-neutral; assembly delegates to the sibling. - Rolling compaction instead of truncation: when the verbatim observed set overflows roughly half the char budget, a background pass summarizes the OLDEST rows via the aux compression LLM (task routing, secret redaction, per-row and total input bounds) into ONE iterative summary row appended to the transcript. Originals stay searchable; nothing is destroyed. Later passes update the summary in place. Read path serves latest summary + verbatim rows after the watermark; the hard cap (oldest dropped, omission note) remains as the last-resort valve. Per-session lock + failure cooldown; cheap pending-chars gate so a busy group does not reload its transcript per message. - Write-side trigger in the shared WhatsApp mixin and the Telegram adapter (fire-and-forget, never raises); prompt markers built from the shared constants. - Attribution scoping: _apply_whatsapp_group_observe_attribution applies only to mention-gated, allowlisted groups; open / free-response groups keep per-user source and clean text (ports the NousResearch#38750 post-audit fix). - Real media in observations: the observe path downloads media to the local cache and records inspectable references ([image: path] + vision_analyze pointer, [voice note: path], [document: path]) instead of kind placeholders; failed downloads degrade to an unavailable note. - End-to-end runner tests through the real _build_gateway_agent_history + wrapper (separation for both platforms, fail-closed replay without a marker, cap relationships, alternation survival, unwrapped persistence contract) plus compaction unit tests (watermark assembly, iterative update, redaction bounds, aux-failure safety, scheduling gate). Sabotage-checked.
Adds plugins/platforms/waha/ (manifest + adapter): inbound webhook receiver with
timing-safe token auth, payload mapping onto the bridge-shaped dict the shared
WhatsApp mixin gates on, outbound sendText/media/edit/typing/sendSeen, and cron
standalone sending. Zero core changes: Platform("waha") resolves via the enum's
bundled-plugin discovery; hermes-waha is the implicit plugin-platform toolset
bundle. Reuses WhatsAppBehaviorMixin for gating, attribution, observed context
(with real media references), and formatting. Docs cover transport selection and
the one-transport-per-profile rule.
NOWEB sendText returns the Baileys key ({remoteJid, fromMe, id}) with no
top-level id, so SendResult.message_id was always None and the stream
consumer fell back to fresh sends per frame. WAHA's chat-message edit
endpoint also rejects bare ids (HTTP 500, expects
<fromMe>_<chatJid>_<id>). Extract key.id, serialize it (remoteJid
preferred), and serialize bare ids on the edit path.
WhatsApp now delivers many DMs keyed by a privacy LID (addressingMode: lid, from <n>@lid). NOWEB exposes the phone form as _data.key.remoteJidAlt; without using it, phone-form allowlists silently reject every LID-addressed message (200-acked, never dispatched).
The gateway converts markdown to WhatsApp formatting itself; telling the model to emit WhatsApp syntax double-formats (literal asterisks become Unicode bold). Ask for markdown instead, mirroring the builtin whatsapp platform hint.
LID-addressed groups break two more gates: replies/mentions carry the bot's LID while botIds held only me.id (phone), and the group sender arrives as key.participant (@lid) with the phone form in participantAlt. Track me.lid alongside me.id in _bot_ids and map participantAlt for the sender. Also log gate rejections at debug so silent drops are diagnosable.
Normalize every outbound chatId to the docs-recommended @c.us form (@s.whatsapp.net is internal-only per waha.devlike.pro chat-ids; @lid targets pass through). Inbound LID resolution now canonicalizes remoteJidAlt/participantAlt to @c.us and remembers learned pairs so LIDs that arrive without their alt field still resolve. Works with the NOWEB store on or off: alt fields and the cache need no store; the Lids API remains an optional future resolver.
The discovery-order test purges plugins.memory.mem0* from sys.modules and re-imports them; restoring sys.modules alone left plugins.memory package attributes pointing at the throwaway modules, so later tests importing from plugins.memory.mem0 got fresh class identities and isinstance routing assertions failed. Rebind the top-level attributes from the saved modules.
_handle_webhook discarded the message handler's return value, so every slash command (/help, /status, ...) plus drain/limit notices vanished silently on the WAHA transport: no reply, no log trace. Route the return through the shared _dispatch_inline_reply helper — the same contract the other adapters use. Agent turns already delivered via streaming return None, so this cannot double-send.
FarelRA
force-pushed
the
feat/waha-platform-plugin
branch
from
September 8, 2026 13:49
5349304 to
1dfc235
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds WAHA (WhatsApp HTTP API) as a bundled platform plugin — the third WhatsApp transport alongside the Baileys bridge and Meta Cloud API. Shares
WhatsAppBehaviorMixin, so gating, mention rules, allow-lists, markdown formatting, and observed group context all behave identically. Zero core changes:Platform("waha")resolves via the enum's bundled-plugin_missing_hook, andhermes-wahais the implicit plugin-platform core bundle (_plugin_platform_bundle).Transport
/api/sendText(chunked at the WhatsApp cap, shared formatter), media via/api/send{Image,Video,Voice,Audio,File}(local → base64 BinaryFile, remote → RemoteFile passthrough), edits viaPUT /api/{session}/chats/{chatId}/messages/{messageId}(streaming parity on NOWEB/WEBJS/GOWS),startTyping/stopTyping,sendSeen(default off, privacy)./webhooks/waha), timing-safeX-Hermes-Tokenauth (WAHA customHeaders are static, so plain-token is the only sound scheme). Payloads map onto the bridge-shaped dict the mixin gates on —participantas group sender,mentionedJidextracted from the engine_datablob (NOWEB/GOWS),replyTo, media → local cache.standalone_sender_fn+WAHA_HOME_CHANNELrouting.requires_env/optional_envfor the setup wizard;env_enablement_fnso env-only setups appear ingateway status.Why a plugin (not core)
Per ADDING_A_PLATFORM: the plugin path needs zero core-tree changes and this is exactly the sibling-adapter shape the guide documents ("unofficial vs official APIs → two adapters sharing a behavior mixin" — WAHA is a third transport of the same platform). Bundled in-tree because it ships with the repo like the other bundled platform plugins.
Tests
20 adapter tests: payload mapping (DM/group/participant-sender/mentions-from-_data/replyTo), gating through the REAL shared mixin (allowed DM dispatches, unknown DM dropped, unmentioned group observed-not-dispatched, mention dispatches, fromMe skipped at the webhook layer), outbound REST shapes (formatting, chunk continuations, edit endpoint, base64 vs URL media, typing/receipts), webhook auth, standalone cron delivery. Plugin loads through the real discovery pipeline (58 manifests → registered, check_fn green).
Docs
website/docs/user-guide/messaging/waha.md: WAHA docker run, session bootstrap, webhook registration with curl, transport comparison table, troubleshooting. Index entries added.Inherited from #104245 (this branch rebases on it)
gateway/run.py— no WAHA-specific runner code._collect_waha_mediaand records inspectable references ([image: path]+vision_analyzepointer,[voice note: path], …) through the shared_whatsapp_observe_media_referenceshelper — no placeholders. Failed downloads degrade to an unavailable note.gateway.observed_context_max_chars/observed_context_max_rows(config.yaml) cap the API-only observed block; compression can never shrink it, so the bound is structural.Cache/alternation invariants
The observed-context wrapper is API-only on the current turn; the transcript persists the unwrapped message. The formatter changes message content only (system prompt untouched; transcript byte-stable per conversation). No new env vars for behavior config — WAHA's settings are config.yaml keys bridged in
apply_yaml_config_fn.Tests
21 WAHA tests (payload mapping, gating through the real shared mixin, observed media download + reference, outbound REST shapes, webhook auth). Full
tests/gateway/green on this branch: 7834 passed, 0 failed.