Skip to content

fix(agent): skip completion notification for killed processes (#36184) - #36210

Open
annguyenNous wants to merge 1 commit into
NousResearch:mainfrom
annguyenNous:fix/process-kill-notification
Open

fix(agent): skip completion notification for killed processes (#36184)#36210
annguyenNous wants to merge 1 commit into
NousResearch:mainfrom
annguyenNous:fix/process-kill-notification

Conversation

@annguyenNous

Copy link
Copy Markdown
Contributor

Problem

When a user cancels a background process (says "no need" and the agent kills it via process(action='kill')), the process may still complete before the SIGTERM lands. The completion watcher then injects a [IMPORTANT: Background process proc_xxx completed] notification, causing the agent to present results the user explicitly said they did not want.

Root cause: The watcher checks _completion_consumed (set by wait/poll/log) but has no concept of "killed" sessions. When the agent kills a process, it doesn't poll the result, so _completion_consumed stays empty and the notification is still injected.

Fix

Add a _killed_sessions set to ProcessRegistry that tracks sessions explicitly killed via kill_process(). The completion watcher checks this set and skips the notification for killed sessions.

Changes

tools/process_registry.py:

  • Add self._killed_sessions: set = set() in __init__
  • Add mark_killed(session_id) and is_killed(session_id) methods
  • Call mark_killed() in kill_process() after successful kill
  • Add cleanup for _killed_sessions in _gc_finished() alongside _completion_consumed

gateway/run.py:

  • Add not _pr_check.is_killed(session_id) check to the completion notification guard

Before vs After

Scenario Before After
User says "cancel", agent kills process, process exits before SIGTERM Agent presents full results Notification silently skipped
User says "cancel", agent kills process, process already exited Agent presents full results Notification silently skipped
Normal background process completion Notification injected Notification injected (unchanged)
Agent polls result via wait/poll Notification skipped Notification skipped (unchanged)

Tests

The fix is minimal and self-contained:

  • _killed_sessions follows the same pattern as _completion_consumed (add on event, check before injection, cleanup on GC)
  • No new external dependencies
  • Existing is_completion_consumed behavior is unchanged

Fixes #36184

When a user cancels a background process (says "no need" and the agent
kills it), the process may still complete before the SIGTERM lands.
The watcher then injects a completion notification, causing the agent
to present results the user explicitly said they did not want.

Add a _killed_sessions set to ProcessRegistry that tracks sessions
explicitly killed via kill_process(). The completion watcher checks
this set and skips the notification for killed sessions.

Fixes NousResearch#36184
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets labels Jun 1, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting a real cancellation race. The current gateway still injects an unconsumed agent-notify completion at gateway/run.py:15600, even though kill_process() records completion_reason = "killed" at tools/process_registry.py:1503-1507.

Problems

  • The proposed post-kill marker does not cover the stated already-exited case: current kill_process() returns at tools/process_registry.py:1458-1462 before the proposed marker would run.
  • The change only gates gateway/run.py; the TUI completion poller independently delivers completion events while checking only is_completion_consumed at tui_gateway/server.py:8745 and 8810.
  • Please add a regression test covering cancellation suppression. Existing tests cover kill metadata (tests/tools/test_notify_on_complete.py:143-170) but not watcher suppression.

Suggested changes

  • Salvage onto current kill metadata, cover the already-exited race, and use one shared suppression predicate in gateway and TUI delivery paths.

Automated hermes-sweeper review.

Comment thread tools/process_registry.py
@@ -1192,6 +1205,7 @@ def kill_process(self, session_id: str) -> dict:
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This marker is reached only after a non-exited session is successfully terminated. The current kill_process() returns already_exited before this point, so it does not cover the PR description's "process already exited" race; record explicit cancellation before that return as well.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/tools Tool registry, model_tools, toolsets P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Agent acts on background process completions without checking conversation cancellation state (Telegram)

3 participants