Skip to content

feat(gateway): fire agent_loop_stopped plugin hook on interrupt - #27208

Closed
francip wants to merge 1 commit into
NousResearch:mainfrom
kortexa-ai:kortexa/agent-loop-stopped-hook
Closed

francip wants to merge 1 commit into
NousResearch:mainfrom
kortexa-ai:kortexa/agent-loop-stopped-hook

Conversation

@francip

@francip francip commented May 17, 2026 •

Copy link
Copy Markdown

What does this PR do?

When /stop or the running-agent fast path of /new interrupts a gateway turn, plugins need a prompt signal to release external work waiting for a tool result. This adds the observer hook agent_loop_stopped immediately after request_hard_interrupt() in gateway/run_agent_cache.py::_interrupt_and_clear_session.

Fixes #27206. The motivating consumer is hermes-livekit, which uses interruption signals to cancel remote tool requests and notify its client.

Behavior

  • Fires only for a real running agent; pending and missing-agent paths still perform normal cleanup without emitting the hook.
  • Supplies session_key, platform, reason and invalidation_reason; callback return values are ignored.
  • A failing plugin cannot prevent adapter cleanup, generation invalidation or session-state release.
  • Documents the gateway-only hook and its relationship to the later on_session_finalize event.

The rebase onto upstream 3a7bf7455f moves dispatch into the current gateway sibling and preserves the existing hard-interrupt, process-reaping and session-state paths.

Validation

Two behavior tests cover seven cases through real temporary-plugin discovery, real hook dispatch and real session state: running/pending/missing turns for stop/new, plus a raising plugin. The three positive-observation cases fail against upstream source at 520e63661c; all seven pass with this change. The subsequent upstream commit changes desktop files only.

HERMES_TEST_FILE_RETRIES=0 scripts/run_tests.sh -j 4 \
  tests/gateway/test_agent_loop_stopped_hook.py \
  tests/gateway/test_session_state_cleanup.py \
  tests/hermes_cli/test_plugins.py

87 passed, 1 existing environment-dependent failure, macOS / Python 3.11. The failure is test_failed_discovery_is_not_cached: installed third-party plugin entry points make its isolated count assertion fail. The identical failure was reproduced with unchanged upstream source. The hook and session-cleanup files pass completely.

Changed-file Ruff, compatibility-pointer checks, contributor attribution and git diff --check pass. The full repository suite was not run for this rebase.

Related proposal

#99930 contains this gateway patch with its original authorship and adds TUI/desktop coverage. It is a broader successor that remains open; this PR retains the narrower gateway scope for review.

Checklist

  • Contribution guidance and existing review feedback addressed.
  • Real-path invariant tests verified against upstream and the fix.
  • User-facing hook documentation updated.
  • No new configuration, dependency or model-tool schema.

francip added a commit to kortexa-ai/hermes-livekit that referenced this pull request May 17, 2026
Connected clients can register tools the agent's LLM can invoke. The
agent calls them with a targeted agent:tool-call JSON message; the
client runs the tool locally and replies with client:tool-result. New
inbound types client:tool-register / -unregister / -result on the
hermes-control topic; matching outbound agent:tool-registered /
-unregistered / agent:tool-call / -cancelled / -timeout events. Flat
JSON envelope (no payload wrapper) to keep the protocol terse.

Tools register into the hermes registry under toolset
'hermes-livekit-tools'. Operators must add that toolset to their
livekit platform_toolsets list in ~/.hermes/config.yaml -- the plugin
does not auto-activate it.

Cleanup paths:
- participant_disconnected: deregister that client's tools, fail their
  pending calls
- full adapter teardown / room rejoin: wipe all tool state
- on_session_finalize (existing hook, fires on /new): cancel pending
  remote tool calls, notify owners with agent:tool-call-cancelled
- agent_loop_stopped (proposed hook, fires on /stop):
  no-op until upstream PR NousResearch/hermes-agent#27208 lands; once
  merged, /stop mid-call cancellation comes online with no plugin
  change
- per-call timeout (default 30s, override via
  HERMES_LIVEKIT_TOOL_TIMEOUT_SEC)

Full design including v0.4 RPC pivot and v0.5 byte-streams roadmap in
docs/remote-tools-design.md. Execution status + cross-session
breadcrumbs (notably the pending upstream PR) in PLAN.md.

Adds examples/test_client.py: an interactive + oneshot Python client
that registers desktop_notify (pops a macOS notification via osascript)
and lets you drive the agent over the data channel without the voice
path. Single file, no extra deps beyond what the plugin already pins.
Verified end-to-end against Avery (the hermes-agent gateway).
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins labels May 17, 2026
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 01686e2 to 7202a84 Compare May 21, 2026 15:08
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 7202a84 to e26d927 Compare May 29, 2026 22:27
@teknium1

teknium1 commented Jun 13, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks for the focused hook and the concrete external-RPC cancellation use case. The premise remains valid on current main: gateway/run.py:16164 interrupts the agent without a plugin observer signal, while active-session /stop and /new call the shared helper at gateway/run.py:9258 and gateway/run.py:9276.

Problems

  • GitHub currently marks this branch CONFLICTING. The target helper has since gained multiplex-aware adapter resolution at gateway/run.py:16166 and cached-agent eviction at gateway/run.py:16183; those current-main behaviors must remain intact when applying the hook.

Suggested changes

  • Salvage the real-agent-gated agent_loop_stopped dispatch immediately after running_agent.interrupt() in the current helper, preserving the current _adapter_for_source() and eviction logic. The PR's sentinel gating, fault tolerance, documentation, and direct-helper regression coverage fit the existing hook model (hermes_cli/plugins.py:1890).

This is an automated hermes-sweeper review.

@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from e26d927 to 681e061 Compare June 18, 2026 03:27
@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 681e061 to a102830 Compare July 15, 2026 12:57
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from a102830 to caed7c5 Compare July 28, 2026 22:21
@francip

francip commented Jul 28, 2026

Copy link
Copy Markdown
Author

Updated onto current main (ed5fd350) — now MERGEABLE.

The branch had drifted ~3348 commits and a trial merge produced 48 conflict markers, so I re-landed the change rather than rebasing the old history. Addressing the review points directly:

  • Current-main behaviour preserved. The hook is dispatched immediately after running_agent.interrupt() inside _interrupt_and_clear_session; the multiplex-aware _adapter_for_source() resolution and the cached-agent eviction below it are untouched. The diff is +223 −0 — purely additive, no existing lines modified.
  • Sentinel gating kept — the pending-sentinel /stop path has no in-flight work, so it doesn't fire.
  • Fault tolerance kept — dispatch failures are swallowed and logged at debug, so a misbehaving plugin can't break an interrupt.
  • Docs and the direct-helper regression coverage came across unchanged.

Verification: the 5 tests in tests/gateway/test_agent_loop_stopped_hook.py pass on current main, and 2 of them fail with the change stashed (hook registration and dispatch), so they pin the behaviour rather than the implementation. tests/gateway/test_session_boundary_hooks.py passes alongside. Three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery are pre-existing on this checkout and reproduce identically with the change stashed.

Motivation from a real consumer, for what it's worth: the hermes-livekit gateway plugin already registers agent_loop_stopped and currently logs Plugin 'livekit' registered unknown hook 'agent_loop_stopped' on every gateway start. It holds remote tool calls awaiting results from a connected voice client, which are exactly the per-turn resources that leak when a turn is interrupted.

@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from caed7c5 to 8dc80a1 Compare August 8, 2026 17:23
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 8dc80a1 to 1aa2ea2 Compare August 11, 2026 19:02
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 1aa2ea2 to f6c3716 Compare August 28, 2026 17:53
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from f6c3716 to d78cef6 Compare August 28, 2026 18:47
francip added a commit to kortexa-ai/hermes-agent that referenced this pull request Aug 31, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on NousResearch#27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from d78cef6 to 75928f9 Compare August 31, 2026 15:56
teknium1 pushed a commit that referenced this pull request Sep 1, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on #27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
@teknium1

teknium1 commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Salvaged into #99930 with your commit and authorship preserved (cherry-pick of 75928f9), plus a widening commit that fires the same hook from the TUI/desktop session.interrupt path (platform="tui") so both interruption surfaces are covered. Thanks for the focused hook and for re-landing it onto current main — that made the cherry-pick clean. Leaving both PRs open for maintainer review.

francip added a commit to kortexa-ai/hermes-agent that referenced this pull request Sep 8, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on NousResearch#27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 75928f9 to 8484437 Compare September 8, 2026 05:16
francip added a commit to kortexa-ai/hermes-agent that referenced this pull request Sep 9, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on NousResearch#27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.

(cherry picked from commit 8484437)
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on NousResearch#27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
@francip
francip force-pushed the kortexa/agent-loop-stopped-hook branch from 8484437 to 0f79b00 Compare September 9, 2026 22:49
teknium1 pushed a commit that referenced this pull request Sep 13, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on #27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
teknium1 pushed a commit that referenced this pull request Sep 13, 2026
Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on #27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.
@teknium1

Copy link
Copy Markdown
Collaborator

Landed via #99930 (merge commit d3202bbc8dba on main), which cherry-picked this PR's commits so @francip's authorship is preserved in history, rebased onto the current codebase layout. Closing as incorporated — thank you!

If anything from this PR did not survive the salvage, reopen with the specific hunk and I'll follow up.

@teknium1 teknium1 closed this Sep 13, 2026
acedatacloud-dev added a commit to AceDataCloud/hermes-agent that referenced this pull request Sep 13, 2026
* feat(gateway): fire agent_loop_stopped plugin hook on interrupt

Reapplied onto current main. The branch had drifted ~3348 commits and a trial
merge produced 48 conflict markers, so this is the same change re-landed rather
than a rebase of the old history.

_interrupt_and_clear_session interrupts the running agent without signalling
plugins, so a plugin holding a per-turn external resource — an outbound RPC
waiting on a tool result the loop will never consume — has no way to learn the
turn is gone. Dispatch agent_loop_stopped immediately after
running_agent.interrupt(), gated on a real running agent: the pending-sentinel
/stop path has no in-flight work, so firing there would be noise.

Per review on NousResearch#27208, the current helper's behaviour is preserved untouched —
multiplex-aware _adapter_for_source() resolution and cached-agent eviction both
still run; the hook is additive and its dispatch failures are swallowed so a
misbehaving plugin cannot break an interrupt.

Tests fail without the change (hook registration and dispatch) and pass with
it. The three failures in tests/hermes_cli/test_plugins.py::TestPluginDiscovery
are pre-existing on this checkout and reproduce with the change stashed.

* feat(tui_gateway): fire agent_loop_stopped on session.interrupt too

Widens the new hook to the sibling interrupt surface: the TUI/desktop
session.interrupt path stops a live turn exactly like the gateway's
/stop, so plugins holding per-turn external resources get the same
signal there (platform='tui'). Gated on a genuinely running turn;
dispatch failures are swallowed so a plugin can never break the
interrupt. Docs updated to describe both surfaces.

Inspired by ChatGPT Work / Codex CLI 0.150.0 'Interrupt' hooks
(hooks that run when an active top-level turn is interrupted).

* docs: document OpenRouter @preset references in /model (follow-up to NousResearch#99633)

* feat: resume x402 MCP tool calls

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci: publish AceDataCloud Hermes image

* chore: map AceDataCloud contributor

* ci: use hosted runners in fork

* feat: allow managed x402 payment enablement

* ci: preserve full tests on hosted runners

* ci: bound test workers on hosted runner

* fix: release quickstart lock before completion

---------

Co-authored-by: Franci Penov <francip@gmail.com>
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@francip
francip deleted the kortexa/agent-loop-stopped-hook branch September 22, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(gateway): plugin hook for /stop interrupt — agent_loop_stopped

3 participants