Skip to content

fix(gateway): register secondary profiles' shell hooks and outbound webhooks - #92682

Closed
chelsealong wants to merge 2 commits into
NousResearch:mainfrom
chelsealong:fix-secondary-profile-hooks-92672
Closed

chelsealong wants to merge 2 commits into
NousResearch:mainfrom
chelsealong:fix-secondary-profile-hooks-92672

Conversation

@chelsealong

Copy link
Copy Markdown
Contributor

Fixes #92672.

Root cause

With gateway.multiplex_profiles: true, GatewayRunner.start() registers declarative shell hooks and outbound webhooks exactly once, at process startup, against the root/default profile's config (gateway/run.py, the register_from_config call site right before self.hooks.discover_and_load()). That call runs before any profile scope exists.

Secondary profiles are started later, in _start_one_profile_adapters(), which runs each profile's turns under its own _profile_runtime_scope (its own HERMES_HOME override) and already calls discover_plugins() there so per-profile plugins work — but it never called agent.shell_hooks.register_from_config() / agent.outbound_webhooks.register_from_config() for that profile's own config.yaml. Because hermes_cli.plugins.get_plugin_manager() caches one PluginManager per resolved home, a secondary profile's turns dispatch through its own manager, which never received the root profile's (or its own) hook callbacks. Result: a secondary profile's hooks.pre_tool_call / hooks.outbound block is silently inert — no error, no webhook fires, and a security gate (e.g. a deny-writes hook) never runs.

Additional wrinkle noted in the issue: agent/shell_hooks.py's _registered idempotence set (and the analogous one in agent/outbound_webhooks.py) was keyed only by (event, matcher, command) / (event, url), with no home/profile scoping. Even after wiring the registration call into the secondary-profile startup path, two profiles configuring an identical hook would have the second call see the triple already in _registered and skip wiring it onto its own (different) plugin manager.

Fix

  • gateway/run.py: inside _start_one_profile_adapters(), within the profile's _profile_runtime_scope, load that profile's own config and call agent.shell_hooks.register_from_config() / agent.outbound_webhooks.register_from_config() against it (mirrors the exact call shape used at root startup, including accept_hooks=False so hooks_auto_accept in the profile's own config is honored). Failures are logged and never block startup, matching the root startup call site's behavior.
  • agent/shell_hooks.py / agent/outbound_webhooks.py: the module-global idempotence sets are now keyed by resolved Hermes home in addition to the existing fields, so identical hook/webhook configuration in two different profiles registers independently on each profile's own plugin manager instead of the second profile's attempt being dropped as a duplicate.

Test plan

Added TestSecondaryProfileHookRegistration to tests/gateway/test_multiplex_adapter_registry.py, asserting that _start_one_profile_adapters() calls both agent.shell_hooks.register_from_config and agent.outbound_webhooks.register_from_config with the secondary profile's own config.

Confirmed the new test fails without the fix:

$ git stash push -- gateway/run.py agent/shell_hooks.py agent/outbound_webhooks.py
$ python3 -m pytest tests/gateway/test_multiplex_adapter_registry.py -q -k TestSecondaryProfileHookRegistration
FAILED ...::test_registers_shell_hooks_and_webhooks_for_secondary_profile
AssertionError: assert ('shell', {...}) in []
1 failed, 18 deselected in 0.89s
$ git stash pop

And passes with it, alongside the full surrounding suites:

$ python3 -m pytest tests/gateway/test_multiplex_adapter_registry.py -q
19 passed in 1.89s

$ python3 -m pytest tests/gateway/ -q -k multiplex
159 passed, 2 skipped, 5983 deselected in 12.82s

$ python3 -m pytest tests/agent/test_shell_hooks.py tests/agent/test_shell_hooks_tree_kill.py tests/agent/test_shell_hooks_consent.py tests/agent/test_outbound_webhooks.py tests/hermes_cli/test_plugins.py -q
158 passed in 14.53s

$ python3 -m ruff check gateway/run.py agent/shell_hooks.py agent/outbound_webhooks.py tests/gateway/test_multiplex_adapter_registry.py
All checks passed!

Notes

Authored with AI assistance; the diff, reproduction, and test were verified manually (real test runs, real ruff run, and a stash-based before/after check of the new regression test) before pushing.

…ebhooks

Multiplex gateway startup only ever calls agent.shell_hooks/
outbound_webhooks register_from_config() once, against the
root/default profile's config, before any profile scope exists.
_start_one_profile_adapters() discovers Python plugins per profile
but never registered that profile's own declarative `hooks:` block,
so a secondary profile's shell hooks (e.g. a deny-writes gate) and
outbound webhooks silently never fire.

Load and register each profile's own config inside its
_profile_runtime_scope, and key the module-level idempotence sets in
shell_hooks.py/outbound_webhooks.py by resolved Hermes home so two
profiles configuring an identical hook/webhook both register on
their own plugin manager instead of the second being dropped as a
duplicate of the first.

Fixes NousResearch#92672
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins area/config Config system, migrations, profiles area/profiles Multi-profile isolation, HERMES_HOME scoping sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 23, 2026

@andrexibiza andrexibiza left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking review on exact head f828c27457bf8e1f2df296463e2104e9a2c48359.

The primary fix is pointed at the right boundary: secondary profiles now load their own config under _profile_runtime_scope, and adding the resolved home to the shell/outbound idempotence key is necessary so identical config in two profile-local PluginManagers does not shadow. Exact-head CI/Docker/Nix are green, and the branch is a clean one-commit child of current main fd760435c6688a2b6c6b7436dde30e267237baef.

There is still one lifecycle/ownership blocker before this can safely become the profile-scoped registration contract: the idempotence ledger is now per-home, but its reset/reload lifecycle is still process-global and asymmetric.

For shell hooks, agent.shell_hooks.re_register_config_hooks() still does _registered.clear() for the entire process and then re-registers only the current home. PluginManager.discover_and_load(force=True) runs under _plugin_home_scope(self.home_path), calls self.unload() (which clears only that manager's _hooks), then invokes _re_register_shell_hooks_after_force(). In a multiplex process with homes A and B:

  1. A and B each register the same shell hook once; both managers correctly hold one callback and _registered has A/B keys.
  2. A force-reloads plugins. A's manager clears its hooks; re_register_config_hooks() clears all A/B idempotence keys, then restores only A.
  3. B's existing callback was never removed, but B's idempotence key is now gone.
  4. The next B adapter restart/reconnect runs this PR's new register_from_config() call; it sees no B key and appends a second callback to B's still-live manager.

That makes a profile-local force reload in A mutate B's registration authority and can cause B's policy script to execute twice (including duplicate approval prompts / side effects). The new home dimension therefore is not actually owned end-to-end yet.

Outbound webhooks expose the mirror failure in the other direction. PluginManager.unload() clears the current manager's _hooks, but the force-reload path only re-registers shell hooks. agent.outbound_webhooks._registered is not cleared or replayed. After B force-reloads, its outbound callback is gone while (home_B,event,url) remains marked registered; a later _start_one_profile_adapters() call hits that stale key and skips re-wiring, so the webhook remains silently inert. That is the same symptom class #92672 is trying to close, just after a supported lifecycle transition instead of initial startup.

Please make config-owned hook registration have profile-local lifecycle ownership rather than patching only the startup key:

  • reset/re-register shell-hook idempotence only for the manager/home being force-reloaded, never all homes;
  • make outbound webhook callbacks participate in the same symmetric force-reload restoration, with stale idempotence state impossible;
  • preferably centralize current-home config-hook registration/teardown so startup, profile adapter recovery, and plugin force reload all use one authority instead of separate module-global ledgers;
  • add a real two-home regression: A+B identical shell hook, force-reload A, re-run B registration/reconnect, and assert B still has exactly one callback;
  • add B outbound webhook -> force-reload B -> callback still present and fires exactly once;
  • extend the current test beyond mocked register_from_config calls to invoke through the actual profile-scoped manager, so the per-home key and dispatch behavior are what is proven.

Topology/provenance:

  • #92672 by @vszgdcn8cj-ctrl owns the concrete multiplex-profile failure; #92682 by @chelsealong is the right delivery object for that axis.
  • #92655 by @LiTerBo is complementary, not duplicate: it fixes the separate hermes serve / Desktop startup surface where config shell hooks are never registered at all.
  • #60036 / open #60267 by @webtecnica is the prior force-reload shell-hook defect and the source of the current re_register_config_hooks() shape; current-main comments explicitly preserve that lineage via the #64188 salvage work. This PR now needs to generalize that older single-home repair to multi-home ownership rather than invalidating another profile's ledger.
  • #64178 / closed #64188 by @Bartok9 is the broader hook-delivery parity lineage and explicitly called force-reload symmetry only partial. This is exactly the remaining other side of that shape.

No separate code finding beyond this lifecycle authority issue: the initial per-profile registration call and home-scoped dedupe direction are sound. Hosted evidence on this exact SHA is genuinely green: CI 32616082267, Docker 32616081794, Nix 32616081787. Those tests do not exercise force reload across two live profile managers, which is why they do not close the blocker.

GitHub does not permit this reviewer identity to submit formal REQUEST_CHANGES, so this blocking disposition is recorded as COMMENTED.

re_register_config_hooks() cleared the entire process-global idempotence
set on every force-reload, so a profile-local plugin force-reload dropped
another live profile's ledger key without touching its still-registered
callback — the next registration call for that profile then appended a
duplicate. Scope the clear to the reloading profile's own home, and give
outbound webhooks the same force-reload restoration shell hooks already
had, since unload() wipes both from the shared _hooks dict.
@chelsealong

Copy link
Copy Markdown
Contributor Author

Fixed the lifecycle-authority gap in c435c0a:

  • agent/shell_hooks.re_register_config_hooks() now clears only the current home's idempotence keys instead of the whole process-global set, so a force-reload in profile A no longer drops profile B's still-live registration key.
  • agent/outbound_webhooks.py gained the same re_register_config_hooks() restoration shell hooks already had, wired into PluginManager._re_register_config_hooks_after_force() alongside the shell-hook call, since unload() wipes both from the shared _hooks dict.
  • Added a real two-manager regression (tests/hermes_cli/test_plugins.py::test_force_reload_of_one_profile_does_not_orphan_another): A+B register an identical shell hook, force-reload A via PluginManager.discover_and_load(force=True), then re-run B's register_from_config() and assert B still has exactly one callback (not duplicated, not dropped). Mirrored for outbound webhooks in tests/agent/test_outbound_webhooks.py::test_force_reload_restores_webhook_and_fires_once, asserting the webhook fires exactly once after B's force-reload. Both fail without the fix (confirmed via git stash) and pass with it.
  • Did not centralize the two ledgers into one authority — the "preferably" in the review — since the home-scoped fix closes the reported defect on its own; happy to do that follow-up separately if you'd rather have it in this PR.

Full run: pytest tests/agent/test_shell_hooks.py tests/agent/test_shell_hooks_tree_kill.py tests/agent/test_shell_hooks_consent.py tests/agent/test_outbound_webhooks.py tests/hermes_cli/test_plugins.py tests/gateway/test_multiplex_adapter_registry.py → 180 passed. ruff check on all touched files → clean.

@andrexibiza andrexibiza left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head rereview — lifecycle blocker closed; acceptance now blocked only on fresh CI

Reviewed exact head c435c0ac809de13b8735167b31e6bc4b4d066c53 against my prior f828c27457bf8e1f2df296463e2104e9a2c48359 lifecycle-authority finding.

The code-level blocker is closed.

  • Shell-hook idempotence teardown/replay is now current-home scoped instead of clearing the process-global ledger.
  • Outbound webhooks now participate in the same force-reload restoration path, so callback state and idempotence state remain symmetric.
  • PluginManager.discover_and_load(force=True) executes under _plugin_home_scope(self.home_path), so the "current home" used by both re-registration helpers is the manager actually being reloaded.
  • get_plugin_manager() is already cached per resolved Hermes home, so manager ownership and the new home-qualified ledgers now align on the same identity dimension.
  • The new two-manager shell-hook regression exercises force-reload A followed by B recovery and proves B remains wired exactly once; the outbound-webhook regression proves the reloaded callback is restored and fires once.

I do not see a remaining code-level blocker in this seven-file shape.

Exact-head Docker 32617192952 and Nix 32617192947 are green. CI 32617193278 is red only in e2e job 97139723243, where tests/e2e/test_platform_commands.py::TestSlashCommands::test_plaintext_restart_gateway_in_group_stays_plain_text[telegram] fails because its mocked send was called zero times instead of once. This PR does not touch that test or its command-routing implementation; the normal Python suite, lints, OSV/supply-chain checks, Windows-only and macOS-only lanes are green. That makes the remaining boundary repository acceptance, not a demonstrated hook-lifecycle regression.

I attempted to rerun the exact failed e2e job, but GitHub rejected the mutation with 403 Resource not accessible by integration, so I cannot manufacture the required fresh green exact-head receipt from this identity.

Disposition: the previous multiplex hook/webhook lifecycle-authority blocker is resolved. Keep the PR blocked only on exact-head CI acceptance; no further code change is requested by this review.

@vszgdcn8cj-ctrl

Copy link
Copy Markdown

The red Python tests / e2e check on this PR is the known intermittent failure in test_plaintext_restart_gateway_in_group_stays_plain_text[telegram], not something this change introduced.

Same test, same assertion (Expected 'mock' to have been called once. Called 0 times.) on PRs touching none of this code:

#92130 explains why the message carries no information: BasePlatformAdapter.handle_message returns after spawning background tasks, so an exception in one surfaces nowhere — a crash, a declined reply and a poll timeout all print that identical line. That PR adds the surfacing but states explicitly that it does not fix the flake.

Noted on #92909 today: no other open PR could be found running Python e2e to compare against. This run is that comparison — same job, same test, same assertion, on a change confined to hook and webhook registration.

@teknium1

teknium1 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Thanks @chelsealong — this landed. Merged via #101255 (bd81bf0) on current main.

Your commits were cherry-picked onto the salvage branch with your git authorship preserved.

Closing this PR as merged-via-salvage.

@teknium1 teknium1 closed this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles area/profiles Multi-profile isolation, HERMES_HOME scoping comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Multiplex gateway never registers a secondary profile's hooks.* (shell hooks and outbound webhooks)

5 participants