fix(whatsapp): bundle bridge.js with the gateway package so pip/Nix installs find it (#15336) - #15460
Closed
briandevans wants to merge 2 commits into
Closed
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a packaging bug where the WhatsApp Node.js bridge wasn’t included in built wheels (breaking pip/Nix installs), by moving the bridge into the gateway Python package and updating all resolvers to use package-relative paths.
Changes:
- Move/ship the WhatsApp bridge under
gateway/whatsapp_bridge/via setuptoolspackage-dataso wheels includebridge.jsand its Node metadata. - Update WhatsApp bridge path resolution in the gateway adapter and CLI (
hermes whatsapp,hermes doctor) to work from installed wheels. - Add regression tests to ensure the bridge directory and required files remain packaged; update docs to reflect the new path.
Reviewed changes
Copilot reviewed 8 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
website/docs/user-guide/docker.md |
Updates Docker docs to reference the new bridge location under gateway/whatsapp_bridge/. |
tests/gateway/test_whatsapp_bridge_packaging.py |
Adds regression tests ensuring bridge files are discoverable and covered by pyproject.toml package-data. |
pyproject.toml |
Adds gateway.whatsapp_bridge package-data patterns so wheels include bridge assets. |
hermes_cli/main.py |
Updates hermes whatsapp setup flow to locate the bridge via gateway.__file__. |
hermes_cli/doctor.py |
Updates npm-audit path resolution to locate the bridge from the installed gateway package. |
gateway/whatsapp_bridge/package.json |
Declares Node dependencies for the embedded WhatsApp bridge. |
gateway/whatsapp_bridge/package-lock.json |
Locks Node dependency tree for the embedded WhatsApp bridge. |
gateway/whatsapp_bridge/bridge.js |
Node.js WhatsApp bridge daemon (HTTP API for the Python adapter). |
gateway/whatsapp_bridge/allowlist.test.mjs |
Node-level tests for allowlist parsing/mapping helpers. |
gateway/whatsapp_bridge/allowlist.js |
Allowlist and identifier mapping utilities used by the bridge. |
gateway/whatsapp_bridge/__init__.py |
Marks the directory as a Python package to ensure setuptools includes package-data reliably. |
gateway/platforms/whatsapp.py |
Updates the default bridge directory to gateway/whatsapp_bridge/ (package-relative). |
CONTRIBUTING.md |
Updates repository layout documentation to reflect the new bridge location. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,15 @@ | |||
| """Embedded WhatsApp bridge (Node.js). | |||
|
|
|||
| This package vendors the small Node.js daemon that talks to whatsapp-web.js. | |||
There was a problem hiding this comment.
The docstring says the bridge talks to whatsapp-web.js, but gateway/whatsapp_bridge/bridge.js (and package.json) use Baileys (@whiskeysockets/baileys). Please update this documentation to match the actual implementation to avoid confusing contributors and downstream packagers.
Suggested change
| This package vendors the small Node.js daemon that talks to whatsapp-web.js. | |
| This package vendors the small Node.js daemon that talks to WhatsApp via | |
| Baileys (``@whiskeysockets/baileys``). |
Comment on lines
+162
to
+167
| # Default bridge location. The bridge files live inside the | ||
| # ``gateway`` package (``gateway/whatsapp_bridge/``) so setuptools / | ||
| # pip / Nix all ship them as package-data. Resolving via | ||
| # ``__file__.parents[1]`` (gateway/) keeps the path correct under | ||
| # both source-tree runs and installed wheels (#15336). | ||
| _DEFAULT_BRIDGE_DIR = Path(__file__).resolve().parents[1] / "whatsapp_bridge" |
There was a problem hiding this comment.
Setting _DEFAULT_BRIDGE_DIR to a path inside site-packages means the adapter’s existing auto-install step (npm install when node_modules/ is missing) will attempt to write into the installed package directory. That commonly fails for read-only installs (notably Nix store / system site-packages). Consider copying the bridge to a writable runtime directory (e.g., under the Hermes home/session dir) before running npm install, or otherwise directing npm’s install prefix/cache to a writable path.
| _whatsapp_bridge_dir = ( | ||
| Path(_gateway_pkg.__file__).resolve().parent / "whatsapp_bridge" | ||
| ) | ||
| except Exception: |
There was a problem hiding this comment.
The except Exception around importing gateway will also swallow unexpected runtime errors inside gateway (not just missing modules), making doctor silently fall back and potentially hiding real issues. Narrow this to ImportError (and, if needed, AttributeError for missing __file__) or at least surface/log the exception when falling back.
Suggested change
| except Exception: | |
| except (ImportError, AttributeError): |
Contributor
Author
|
Heads-up on CI: the Actual files this PR changes (verbatim from None of these match the scan's actual patterns:
The one-character fix for the scan to use three-dot merge-base diff ( Other lanes are pending or green: |
alt-glitch
added
type/bug
briandevans
added a commit
to briandevans/hermes-agent
that referenced
this pull request
Apr 25, 2026
…m install (Copilot NousResearch#15460) Copilot's second review on NousResearch#15460 flagged three issues; the third is critical and changes the bridge-install design: 1. ``gateway/whatsapp_bridge/__init__.py`` docstring called the bundled bridge ``whatsapp-web.js``-based. It's actually Baileys (``@whiskeysockets/baileys`` in ``package.json``). Corrected. 2. ``hermes_cli/doctor.py`` swallowed every exception when importing ``gateway`` — narrowed to ``(ImportError, AttributeError)`` so a genuine runtime bug inside the gateway module surfaces loudly. 3. **Critical**: shipping the bridge to ``site-packages/gateway/ whatsapp_bridge/`` means ``npm install`` tries to write into site-packages. On Nix store / system pip installs that's read- only, so every user on those install paths would hit ``EROFS`` / ``EACCES`` the first time they ran the WhatsApp gateway. ### Fix for (3): template + runtime dir split The site-packages location is now a read-only **template**. The adapter copies the JS sources + package.json to a writable runtime directory (``HERMES_HOME / 'whatsapp-bridge/'``) on first ``start()``, then runs ``npm install`` there. New module-level helpers in ``gateway/platforms/whatsapp.py``: * ``_BRIDGE_TEMPLATE_FILES`` — tuple of filenames that must be copied; excludes ``node_modules`` (npm manages it) and ``__init__.py`` (a Python artefact, not part of the Node app). * ``_resolve_runtime_bridge_dir()`` — returns ``get_hermes_home() / 'whatsapp-bridge'``. Honouring ``HERMES_HOME`` means profile- isolated installs and Docker volumes work without extra wiring. * ``_ensure_runtime_bridge_files(template_dir, runtime_dir)`` — copy-if-newer semantics so Hermes upgrades propagate new bridge code on first start after update. ``shutil.copy2`` preserves content; we then ``chmod 0o644`` explicitly because pip may ship package-data at ``0o444`` and npm needs write access to update ``package-lock.json``. No-ops cleanly when the template is missing (dev checkouts without ``pip install -e .``). Adapter: * New class constant ``_DEFAULT_BRIDGE_TEMPLATE_DIR`` pointing at the site-packages read-only location. * ``_DEFAULT_BRIDGE_DIR`` keeps pointing at the same template for backward compat (several tests read that attribute). * ``__init__`` stores a ``_runtime_bridge_dir`` on the instance and defaults ``_bridge_script`` to ``<runtime>/bridge.js``. * ``start()`` calls ``_ensure_runtime_bridge_files`` before the existing npm-install step when the script lives under the runtime dir — operator overrides via ``config.extra['bridge_script']`` are untouched. Setup wizard (``hermes_cli/main.py``) uses the same two helpers so the interactive ``hermes setup --whatsapp`` flow runs npm in the writable dir too. ``doctor.py``'s npm-audit step now points at the runtime location — that's where ``node_modules`` actually lives once any user has run the gateway — and narrows its import fallback exception set. ### Tests (6 new, all passing; total 54 whatsapp tests green) * ``test_runtime_bridge_dir_lives_under_hermes_home`` — runtime dir always resolves under ``HERMES_HOME``, never site-packages. * ``test_ensure_runtime_bridge_files_copies_template`` — first-boot copies the expected files, skips ``node_modules`` and ``__init__.py``. * ``test_ensure_runtime_bridge_files_chmods_writable`` — template at ``0o444`` ends up as ``0o644`` in runtime so npm can write. * ``test_ensure_runtime_bridge_files_is_mtime_aware`` — idempotent when template unchanged, re-copies when template gets newer (Hermes upgrade pulls in new bridge.js). * ``test_ensure_runtime_bridge_files_handles_missing_template`` — dev checkout with no template dir: no-ops without crashing. * ``test_adapter_default_bridge_script_points_at_runtime`` — E2E: ``WhatsAppAdapter(...)._bridge_script`` parent equals the runtime dir, not site-packages. Updated the ``_make_adapter`` fixtures in ``test_whatsapp_connect.py`` and ``test_whatsapp_formatting.py`` to set ``_runtime_bridge_dir`` on the ``__new__``-built instances (they bypass ``__init__`` so the new attribute needs explicit setup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
briandevans
force-pushed
the
fix/whatsapp-bridge-package-data-nixos
branch
from
April 25, 2026 01:42
c542b52 to
2c0a4ff
Compare
Contributor
Author
|
Thanks @copilot — all three findings addressed in 1. Docstring 2. 3. site-packages is read-only on Nix —
Six new tests cover the new seam: runtime dir location, file copy semantics, 54/54 whatsapp tests still pass. |
…nstalls find it (NousResearch#15336) The WhatsApp bridge (Node.js daemon launched by ``WhatsAppAdapter``) lived at ``scripts/whatsapp-bridge/`` — outside any Python package. Setuptools' ``find_packages`` only ships files that live INSIDE a package directory, so ``pip install`` (and downstream Nix / Docker / Homebrew installs that build from the wheel) silently dropped the bridge from the artifact. When users on those install paths tried to start the WhatsApp gateway: ✗ Bridge script not found at /nix/store/.../site-packages/scripts/whatsapp-bridge/bridge.js Even though the source tree had ``scripts/whatsapp-bridge/bridge.js``, the installed wheel did not. Move the bridge inside the ``gateway`` package — its only consumer — and register it as setuptools package-data so wheels actually contain it. The directory now lives at ``gateway/whatsapp_bridge/`` and resolves cleanly under both source-tree runs and installed wheels via ``Path(gateway.__file__).parent / "whatsapp_bridge"``. Files / changes: - ``git mv scripts/whatsapp-bridge/ → gateway/whatsapp_bridge/`` (5 files) - New ``gateway/whatsapp_bridge/__init__.py`` marker — required so ``find_packages`` treats it as a regular setuptools package and the package-data globs reliably include the JS files (PEP 420 namespace packages have spotty package-data support across setuptools versions). - ``pyproject.toml``: add ``"gateway.whatsapp_bridge" = ["*.js", "*.mjs", "package.json", "package-lock.json"]`` to ``[tool.setuptools.package-data]``. Globs deliberately exclude ``node_modules`` — the ``hermes setup`` flow re-creates that tree at runtime via ``npm install`` so we don't bloat the wheel with a resolved dependency tree. - Update three resolvers: - ``gateway/platforms/whatsapp.py::WhatsAppAdapter._DEFAULT_BRIDGE_DIR`` → uses ``Path(__file__).parents[1] / "whatsapp_bridge"`` (i.e. the package's own bridge subdir). - ``hermes_cli/main.py``: bridge-deps install step now resolves via ``gateway.__file__`` so it works under wheel installs too. - ``hermes_cli/doctor.py``: same — npm-audit step also uses the package-relative path. - Doc updates in ``CONTRIBUTING.md`` and ``website/docs/user-guide/ docker.md`` to point at the new path. - ``_DEFAULT_BRIDGE_DIR`` resolves to a real existing directory - That directory is a child of the ``gateway`` package's __file__ parent (so the package-data globs target it) - Required files (``bridge.js``, ``allowlist.js``, ``package.json``) are present - ``pyproject.toml`` has the ``gateway.whatsapp_bridge`` package-data entry with at least ``*.js`` and ``package.json`` patterns — parsed via ``tomllib`` so this regression is caught even outside an installed wheel - ``__init__.py`` marker exists (the thing that makes the directory a real setuptools package on every version) **Verified regression guards**: temporarily removed the ``[tool.setuptools.package-data] "gateway.whatsapp_bridge"`` entry from ``pyproject.toml``; ``test_pyproject_package_data_covers_bridge_files`` correctly failed with the exact assertion message ("wheel installs will not contain bridge.js (NousResearch#15336)"). Restored → all 5 pass. 48 total tests pass (43 existing whatsapp tests + 5 new packaging tests). Closes NousResearch#15336 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…m install (Copilot NousResearch#15460) Copilot's second review on NousResearch#15460 flagged three issues; the third is critical and changes the bridge-install design: 1. ``gateway/whatsapp_bridge/__init__.py`` docstring called the bundled bridge ``whatsapp-web.js``-based. It's actually Baileys (``@whiskeysockets/baileys`` in ``package.json``). Corrected. 2. ``hermes_cli/doctor.py`` swallowed every exception when importing ``gateway`` — narrowed to ``(ImportError, AttributeError)`` so a genuine runtime bug inside the gateway module surfaces loudly. 3. **Critical**: shipping the bridge to ``site-packages/gateway/ whatsapp_bridge/`` means ``npm install`` tries to write into site-packages. On Nix store / system pip installs that's read- only, so every user on those install paths would hit ``EROFS`` / ``EACCES`` the first time they ran the WhatsApp gateway. The site-packages location is now a read-only **template**. The adapter copies the JS sources + package.json to a writable runtime directory (``HERMES_HOME / 'whatsapp-bridge/'``) on first ``start()``, then runs ``npm install`` there. New module-level helpers in ``gateway/platforms/whatsapp.py``: * ``_BRIDGE_TEMPLATE_FILES`` — tuple of filenames that must be copied; excludes ``node_modules`` (npm manages it) and ``__init__.py`` (a Python artefact, not part of the Node app). * ``_resolve_runtime_bridge_dir()`` — returns ``get_hermes_home() / 'whatsapp-bridge'``. Honouring ``HERMES_HOME`` means profile- isolated installs and Docker volumes work without extra wiring. * ``_ensure_runtime_bridge_files(template_dir, runtime_dir)`` — copy-if-newer semantics so Hermes upgrades propagate new bridge code on first start after update. ``shutil.copy2`` preserves content; we then ``chmod 0o644`` explicitly because pip may ship package-data at ``0o444`` and npm needs write access to update ``package-lock.json``. No-ops cleanly when the template is missing (dev checkouts without ``pip install -e .``). Adapter: * New class constant ``_DEFAULT_BRIDGE_TEMPLATE_DIR`` pointing at the site-packages read-only location. * ``_DEFAULT_BRIDGE_DIR`` keeps pointing at the same template for backward compat (several tests read that attribute). * ``__init__`` stores a ``_runtime_bridge_dir`` on the instance and defaults ``_bridge_script`` to ``<runtime>/bridge.js``. * ``start()`` calls ``_ensure_runtime_bridge_files`` before the existing npm-install step when the script lives under the runtime dir — operator overrides via ``config.extra['bridge_script']`` are untouched. Setup wizard (``hermes_cli/main.py``) uses the same two helpers so the interactive ``hermes setup --whatsapp`` flow runs npm in the writable dir too. ``doctor.py``'s npm-audit step now points at the runtime location — that's where ``node_modules`` actually lives once any user has run the gateway — and narrows its import fallback exception set. * ``test_runtime_bridge_dir_lives_under_hermes_home`` — runtime dir always resolves under ``HERMES_HOME``, never site-packages. * ``test_ensure_runtime_bridge_files_copies_template`` — first-boot copies the expected files, skips ``node_modules`` and ``__init__.py``. * ``test_ensure_runtime_bridge_files_chmods_writable`` — template at ``0o444`` ends up as ``0o644`` in runtime so npm can write. * ``test_ensure_runtime_bridge_files_is_mtime_aware`` — idempotent when template unchanged, re-copies when template gets newer (Hermes upgrade pulls in new bridge.js). * ``test_ensure_runtime_bridge_files_handles_missing_template`` — dev checkout with no template dir: no-ops without crashing. * ``test_adapter_default_bridge_script_points_at_runtime`` — E2E: ``WhatsAppAdapter(...)._bridge_script`` parent equals the runtime dir, not site-packages. Updated the ``_make_adapter`` fixtures in ``test_whatsapp_connect.py`` and ``test_whatsapp_formatting.py`` to set ``_runtime_bridge_dir`` on the ``__new__``-built instances (they bypass ``__init__`` so the new attribute needs explicit setup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
briandevans
force-pushed
the
fix/whatsapp-bridge-package-data-nixos
branch
from
May 1, 2026 03:16
2c0a4ff to
b612330
Compare
Contributor
Author
|
Rebased onto current Re-verified focused tests on the rebased head (
Diff is otherwise unchanged: same source-of-truth move ( |
Contributor
Author
|
Closing to keep the queue clean — 17 days idle and now conflicting across pyproject.toml and gateway/whatsapp_bridge/ following recent main refactors. Happy to reopen if this is still useful. |
anoosa1
added a commit
to anoosa1/hermes-agent
that referenced
this pull request
May 25, 2026
7 tasks
|
Hello, |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes `#15336`. The WhatsApp bridge (Node.js daemon launched by `WhatsAppAdapter`) lived at `scripts/whatsapp-bridge/` — outside any Python package. Setuptools' `find_packages` only ships files that live INSIDE a package directory, so `pip install` (and downstream Nix / Docker / Homebrew installs that build from the wheel) silently dropped the bridge from the artifact. When users on those install paths tried to start the WhatsApp gateway:
```
✗ Bridge script not found at /nix/store/.../site-packages/scripts/whatsapp-bridge/bridge.js
```
Even though the source tree had `scripts/whatsapp-bridge/bridge.js`, the installed wheel did not.
Fix
Move the bridge inside the `gateway` package — its only consumer — and register it as setuptools package-data so wheels actually contain it. The directory now lives at `gateway/whatsapp_bridge/` and resolves cleanly under both source-tree runs and installed wheels.
Changes
```toml
"gateway.whatsapp_bridge" = [".js", ".mjs", "package.json", "package-lock.json"]
```
Globs deliberately exclude `node_modules` — the `hermes setup` flow re-creates that tree at runtime via `npm install`, so we don't bloat the wheel with a resolved dependency tree.
Related Issue
Fixes #15336
Type of Change
Test plan
Test coverage detail
`test_whatsapp_bridge_packaging.py` (5 tests):
Why move to `gateway/whatsapp_bridge/` and not just include `scripts/`?
Considered but rejected:
Moving inside `gateway/` is the smallest surface area that works on every setuptools version, keeps the bridge next to its only consumer, and matches the existing `hermes_cli = ["web_dist/**/*"]` pattern already in the codebase.
Out of scope