Skip to content

fix(api-server): forward Codex commentary in streaming SSE endpoint - #67613

Open
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/67580-codex-commentary
Open

fix(api-server): forward Codex commentary in streaming SSE endpoint#67613
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/67580-codex-commentary

Conversation

@webtecnica

Copy link
Copy Markdown
Contributor

Summary

Wire interim_assistant_callback through APIServerAdapter._create_agent() and _run_agent() so Codex commentary (phase=commentary) is forwarded to API Server streaming clients.

Add assistant.commentary SSE event to the Hermes session stream endpoint (POST /api/sessions/{session_id}/chat/stream), gated behind display.show_commentary (defaults to true).

Fixes #67580

Problem

When Hermes uses the openai-codex backend through API Server, Codex can generate user-facing progress messages as assistant output items with phase="commentary". Hermes preserves those items in codex_message_items, but API Server streaming clients do not receive them.

Long-running, tool-heavy turns remain silent until the final answer. The user cannot tell whether the agent is making progress, stalled, or has misunderstood the task.

Root Cause

APIServerAdapter._create_agent() forwards stream_delta_callback, tool_progress_callback, tool_start_callback, and tool_complete_callback to AIAgent, but it does not accept or forward interim_assistant_callback. _run_agent() has the same omission. The Hermes session SSE handler wires assistant deltas and tool progress without a commentary path.

The core commentary behavior was added in #66115 and defaults to visible, but gateway/platforms/api_server.py was not part of that callback wiring.

Changes

  1. _create_agent(): Added interim_assistant_callback=None parameter and passes it to AIAgent()
  2. _run_agent(): Added interim_assistant_callback=None parameter and forwards it to _create_agent()
  3. _handle_session_chat_stream(): Added _commentary callback that emits assistant.commentary SSE events with message_id, text, and already_streamed fields. Gated behind display.show_commentary config.

Safety

  • Commentary only — phase="analysis" / raw chain-of-thought / reasoning summaries are NOT exposed through this path
  • Final content remains unchanged — commentary is emitted as a separate event type and is never concatenated into assistant.completed content
  • display.show_commentary: false preserves the quiet behavior (no commentary events emitted)
  • Non-streaming endpoints continue to return only the final answer (no change)
  • /v1/chat/completions delta content is unaffected (commentary does not go into choices[].delta.content)

Wire interim_assistant_callback through APIServerAdapter._create_agent()
and _run_agent() so Codex commentary (phase=commentary) is forwarded
to API Server streaming clients.

Add  SSE event to the Hermes session stream
endpoint, gated behind display.show_commentary (defaults to true).

Fixes NousResearch#67580
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery provider/openai OpenAI / Codex Responses API codex duplicate This issue or pull request already exists labels Jul 19, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #67593: both wire interim_assistant_callback into the API Server session SSE path to emit Codex commentary.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracing the existing callback boundary and keeping commentary separate from final response content.

Problems

  • The new callback ignores already_streamed (gateway/platforms/api_server.py:2572 in this diff). Core can call this callback with already_streamed=True when the same visible text already went through normal deltas (run_agent.py:4893-4900), so this would duplicate commentary in the session stream.
  • /v1/responses streaming still calls _run_agent without an interim callback on current main (gateway/platforms/api_server.py:3841-3853). The linked issue describes API Server streaming endpoints, and this supported stream would remain silent.
  • This adds a public SSE event but changes neither regression tests nor the documented session-stream event contract (website/docs/user-guide/features/api-server.md:346).

Suggested changes

  • Drop already_streamed=True events and add regression coverage for duplicate suppression, final-content separation, and the config gate.
  • Extend the Responses stream with a distinct phase="commentary" message item, then document both stream contracts.

Automated hermes-sweeper review.

@@ -2568,11 +2570,27 @@ def _tool_progress(event_type: str, tool_name: str = None, preview: str = None,
event_name = event_type.replace("tool.", "tool.")
_enqueue(event_name, {"message_id": message_id, "tool_name": tool_name, "preview": preview, "args": args})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honor already_streamed here. Core invokes this callback with already_streamed=True when the same visible text was already sent through the delta stream (run_agent.py:4893-4900); emitting it again as assistant.commentary duplicates client-visible narration.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Codex commentary is not forwarded by API Server streaming endpoints

3 participants