fix: coalesce multiple background process completion notifications by session_key

[handnewb]

Problem

When multiple background processes finish in the same gateway tick, each one injects a separate notification into the agent session via _inject_watch_notification. This floods the context with redundant messages, causes session lag (Persisted transcript lagged), and leaves the agent processing notification-after-notification instead of responding to the user.

Evidence from production (Discord gateway)

17:24:53.057 Process proc_99a794b9b5e9 finished — injecting agent notification
17:24:53.060 Process proc_ce2c0b31a186 finished — injecting agent notification
17:24:53.063 Process proc_12944841da9c finished — injecting agent notification
17:24:53.067 Process proc_7833aa6465b5 finished — injecting agent notification
17:24:53.070 Process proc_0f220e4e1655 finished — injecting agent notification
17:24:53.074 Process proc_d2824952ce08 finished — injecting agent notification

Then immediately:

WARNING Persisted transcript lagged live cached history (disk=154, memory=157)

6 processes → 6 injections → flooded session → unresponsive agent.

Fix

Added GatewayRunner._coalesce_and_inject_watch_events() which groups standard completion events by session_key before injection. When N processes complete for the same session in one drain tick, a single batched message is delivered:

[IMPORTANT: 6 background processes finished — results batched to avoid session flood. Use process(id=N, action='log') to inspect individual outputs.]

Key design decisions:

• Only standard completions are coalesced — watch_match events pass through individually since each pattern match is independently important
• Grouping by session_key, not globally — completions for different sessions remain separate
• Single-event pass-through is unchanged — zero overhead when only one process finishes

Changes

• gateway/run.py:

  • New method _coalesce_and_inject_watch_events() with coalescing logic
  • Updated post-turn drain loop to use new coalesced method
  • Extended _format_gateway_process_notification() to handle standard completion event type (previously returned None)

• tests/gateway/test_background_process_notifications.py:

  • 7 new tests covering: single event passthrough, coalescing same session_key, separate session_keys, watch_match non-coalescing, mixed event types, empty list noop

Testing

41 passed in 5.18s (all existing + 7 new)

Closes #70300

[yuzilongleif-collab]

I reproduced #70300 against current main and traced standard completions through the production call path.

One important ownership detail: standard completion events are delivered directly by each _run_process_watcher() through _deliver_completion_notification(). _drain_gateway_watch_events() returns watch_match events but does not return standard completions, so _coalesce_and_inject_watch_events() is not reached for the reported production path. The current unit tests call that helper directly and therefore do not detect the disconnect.

I opened #71898 with batching at the _run_process_watcher() delivery seam plus an end-to-end test that starts three concurrent watchers and asserts one adapter injection. It also preserves per-process retry/dedupe bookkeeping and covers route isolation, in-flight flush races, duplicate-primary handling, and formatter failure.

Sharing this here to connect the two approaches and avoid duplicated debugging; maintainers can choose whichever shape best fits the gateway ownership model.

[handnewb]

@yuzilongleif-collab — great catch on the architectural issue. You're absolutely right that standard completions flow through _run_process_watcher() → _deliver_completion_notification() and never reach the post-turn drain where my original _coalesce_and_inject_watch_events() sat.

I've opened a new approach in #72675 that:

1. Bathes standard completions at _run_process_watcher() (the correct seam, following your insight) with a Future-based pattern similar to fix(gateway): coalesce concurrent process completions #71898
2. PLUS coalesces watch_match/watch_disabled events at the post-turn drain — which neither PR covered
3. Adds threshold-based early flush (5+ entries → immediate), visual status indicators (✅/❌), aggregate summary, elapsed time, and zero-latency single completions

89 tests passing. Closing this in favor of the combined approach. Thanks for the thorough review!