feat(plugins): add streaming output observer hooks

[deaneeth]

Summary

Closes #64161 as part of #64182.

This adds observer-only plugin hooks for streaming LLM output without putting plugin callbacks on the token path:

• on_stream_start when a streaming response begins
• on_stream_delta for normalized text deltas
• on_stream_end when streaming finishes or errors
• on_interim_message for mid-loop assistant messages surfaced before the final answer

The hook delivery path uses host-owned bounded queues with one background worker per registered callback. If one plugin stalls, only that callback's queue can fill and drop its oldest pending observer event; other observers continue receiving events independently. Hook return values are ignored and callback exceptions are isolated from the stream.

Reasoning deltas remain private by default and require explicit opt-in:

plugins:
  stream_reasoning_deltas: true

Implementation notes

• Registers the four new hook names in hermes_cli.plugins.VALID_HOOKS.
• Adds agent.plugin_stream_hooks as the async per-consumer dispatcher for stream observer hooks.
• Emits lifecycle hooks from Codex, Bedrock, Anthropic Messages, and OpenAI-compatible streaming paths.
• Emits text/reasoning deltas from the existing stream delta helpers.
• Counts on_stream_start, on_stream_delta, and on_stream_end as stream consumers so plugin-only streaming observers can activate streaming transport.
• Keeps on_interim_message observable without letting an interim-only plugin force streaming transport.
• Removes the HERMES_PLUGIN_STREAM_HOOK_QUEUE_SIZE environment variable path; the queue bound is internal.
• Includes Bedrock plugin-only reasoning observers when plugins.stream_reasoning_deltas is enabled.
• Documents payload fields, per-consumer queue behavior, non-mutating semantics, interim-message transport behavior, and reasoning opt-in.
• Keeps fake/minimal agents safe by treating missing stream lifecycle emitters as no-ops.

Non-goals

• No mutation/transform support for streaming tokens.
• No raw provider event exposure.
• No Desktop GUI plugin surface changes.

Verification

E:\Projects\hermes-agent\.venv\Scripts\python.exe -m py_compile agent\plugin_stream_hooks.py agent\chat_completion_helpers.py hermes_cli\plugins.py run_agent.py
E:\Projects\hermes-agent\.venv\Scripts\python.exe -m pytest tests\run_agent\test_plugin_stream_hooks.py -q
E:\Projects\hermes-agent\.venv\Scripts\python.exe -m pytest tests\run_agent\test_streaming.py tests\agent\test_bedrock_adapter.py -q
E:\Projects\hermes-agent\.venv\Scripts\python.exe -m ruff check agent\plugin_stream_hooks.py agent\chat_completion_helpers.py hermes_cli\plugins.py tests\run_agent\test_plugin_stream_hooks.py
git diff --check upstream/main...HEAD

Results:

• tests/run_agent/test_plugin_stream_hooks.py: 12 passed
• tests/run_agent/test_streaming.py + tests/agent/test_bedrock_adapter.py: 183 passed
• py_compile: passed
• ruff check: passed
• git diff --check upstream/main...HEAD: passed

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[teknium1]

Thanks for implementing the off-token-path observer direction and documenting the payloads. The underlying streaming-hook gap remains present on current origin/main.

Problems

• agent/plugin_stream_hooks.py:42 drains one global queue via plugins.invoke_hook(). PluginManager.invoke_hook() runs registered callbacks serially (hermes_cli/plugins.py:1912-1927), so one slow plugin blocks every observer consumer. This does not meet the per-consumer queue + fan-out-worker contract stated by the maintainer on #64161.
• The Bedrock call still gates on_reasoning_delta on UI callbacks (agent/chat_completion_helpers.py:2106), so plugin-only reasoning observers enabled through plugins.stream_reasoning_deltas receive no Bedrock reasoning deltas.
• agent/plugin_stream_hooks.py:22 adds HERMES_PLUGIN_STREAM_HOOK_QUEUE_SIZE; behavioral configuration belongs in config.yaml, not a new user-facing HERMES_* variable.

Suggested changes

• Isolate each registered consumer with its own bounded queue/worker and test slow-consumer isolation.
• Include the reasoning opt-in in the Bedrock callback predicate and add a plugin-only Bedrock test.
• Remove the environment-variable tuning path or move it to supported config.

Automated hermes-sweeper review.

[deaneeth]

@teknium1 addressed the review on #64317:

• replaced the single global stream-hook queue with per-callback bounded queues/workers, so a slow plugin only backs up its own observer stream;
• removed HERMES_PLUGIN_STREAM_HOOK_QUEUE_SIZE; the queue bound is internal now;
• excluded on_interim_message from stream-consumer detection and added regression coverage for interim-only plugins not forcing streaming transport;
• included plugin-only Bedrock reasoning observers when plugins.stream_reasoning_deltas is enabled, with a Bedrock regression test;
• updated the hooks docs and PR body to describe the per-consumer queue behavior.

Local verification:

py_compile: passed
pytest tests/run_agent/test_plugin_stream_hooks.py -q: 12 passed
pytest tests/run_agent/test_streaming.py tests/agent/test_bedrock_adapter.py -q: 183 passed
ruff check touched Python files: passed
git diff --check upstream/main...HEAD: passed

Pushed at c35d504bd. The refreshed GitHub CI run is green, including All required checks pass.

[GottZ]

This was generated by AI during triage.

Summary

One PR addresses #64161. #64317 adds observer-only stream lifecycle, normalized delta, and interim-message hooks, with bounded off-token-path delivery, per-callback isolation, and opt-in reasoning deltas.

Related pull requests

• feat(plugins): add streaming output observer hooks #64317 best fix — (+771/-22) — n/a: The diff registers and emits on_stream_start, on_stream_delta, on_stream_end, and on_interim_message, using a bounded queue and worker per callback with drop-oldest overflow behavior and targeted regression coverage. The visible keep_open review identified global serialization, missing plugin-only Bedrock reasoning delivery, and an inappropriate environment variable; the current diff addresses those points with per-consumer dispatchers, opt-in Bedrock reasoning support, and an internal queue bound.

Suggested consolidation

Keep #64317 open with a salvage path: preserve its observer-only hook API, off-token-path per-consumer dispatcher, reasoning opt-in, documentation, and regression tests while obtaining contributor confirmation that the revised diff satisfies the corrected v1 design. There are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I64161(["issue #64161 (open)"])
    P64317["PR #64317 (open)"]
    P64317 -->|best fix| I64161
    class I64161 open
    class P64317 open
    class P64317 best
    class P64317 target
    click I64161 "https://github.com/NousResearch/hermes-agent/issues/64161"
    click P64317 "https://github.com/NousResearch/hermes-agent/pull/64317"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 40 kB of PR diffs, 7 kB of issue/PR text, 6 kB of discussion (9 comments), 2 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[teknium1]

Merged via #84924 with your commit intact (rebase-merge, authorship preserved) — the streaming observer hooks are on main with the bounded-queue contract and reasoning-delta opt-in exactly as you built them, adapted onto main's newer single-writer stream fencing. Thanks @deaneeth! Closing this original. (#64161)

[deaneeth]

Huge thanks to @teknium1 for salvaging this PR and getting it merged! I’m really glad to have contributed to Hermes Agent, and I truly appreciate you taking the time to bring it across the finish line.