Skip to content

fix(codex): surface tool-progress + interim commentary on codex_app_server runtime (#33200) - #33294

Closed
xxxigm wants to merge 3 commits into
NousResearch:mainfrom
xxxigm:fix/33200-codex-app-server-gateway-progress
Closed

fix(codex): surface tool-progress + interim commentary on codex_app_server runtime (#33200)#33294
xxxigm wants to merge 3 commits into
NousResearch:mainfrom
xxxigm:fix/33200-codex-app-server-gateway-progress

Conversation

@xxxigm

@xxxigm xxxigm commented May 27, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Restores live tool-progress bubbles and interim assistant commentary on Discord / Telegram / TUI when the active provider runs on openai_runtime: codex_app_server. Fixes the silent-channel UX described in #33200.

The codex_app_server runtime hands the entire turn to a subprocess and short-circuits the normal Hermes tool loop, so tool_progress_callback, _fire_stream_delta and _emit_interim_assistant_message never fire while codex is working — only the final answer lands. CodexAppServerSession has always exposed a raw on_event hook, but run_codex_app_server_turn simply never supplied one.

This PR ships the missing bridge in three small commits:

  1. feat(codex) — mapping helpers + bridge factory. Four pure-dict helpers translate codex item/* payloads into the Hermes-shape (tool name, args, preview, result+is_error), then make_codex_app_server_event_bridge(agent) wraps them into a single on_event(note) callable. Tool names match CodexEventProjector so the progress bubble and the projected tool_calls entry agree on the identifier.

  2. fix(codex) — wire the bridge into the runtime. Pass on_event=make_codex_app_server_event_bridge(agent) when constructing the per-session CodexAppServerSession. ~7-line change, no other behaviour shift.

  3. test(codex) — 42 regression tests. Pin the mapping contract per type, the dispatch contract per Codex event, defensive paths (non-dict notifications, missing params, raising callbacks, agents without callbacks), and one integration guard that asserts run_codex_app_server_turn actually wires the bridge — so a future refactor can't silently regress this again.

Related Issue

Fixes #33200

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/codex_runtime.py — adds _codex_item_to_tool_name / _codex_item_to_args / _codex_item_to_preview / _codex_item_completion_payload plus make_codex_app_server_event_bridge(agent), then wires on_event= into CodexAppServerSession inside run_codex_app_server_turn. Exports the factory in __all__.
  • tests/agent/test_codex_app_server_event_bridge.py42 new tests across TestCodexItemToToolName, TestCodexItemToArgs, TestCodexItemToPreview, TestCodexItemCompletionPayload, TestStreamDeltaDispatch, TestToolProgressDispatch, TestAgentMessageInterimDispatch, TestBridgeRobustness, and TestBridgeWiredInRuntime.

Translation map

Codex notification Hermes callback
item/started for commandExecution / fileChange / mcpToolCall / dynamicToolCall tool_progress_callback("tool.started", name, preview, args)
item/completed for same tool_progress_callback("tool.completed", name, None, None, duration=…, is_error=…, result=…)
item/agentMessage/delta _fire_stream_delta(text)
item/reasoning/delta _fire_reasoning_delta(text)
item/completed for agentMessage _emit_interim_assistant_message({"role": "assistant", "content": text})

_emit_interim_assistant_message already calls _interim_content_was_streamed to set already_streamed=True when the stream-delta path showed the same text, so adapters stay dedup-safe.

Backwards compatible — CodexAppServerSession(on_event=...) has always been an optional kwarg; tests and cron / non-interactive contexts that never set the agent callbacks see no change (the bridge is a no-op when the agent has no callbacks registered).

How to Test

# New bridge suite (42 tests)
python3 -m pytest tests/agent/test_codex_app_server_event_bridge.py -v

# Full codex_app_server test pass (122 existing + 42 new = 164 tests)
python3 -m pytest \
    tests/run_agent/test_codex_app_server_integration.py \
    tests/agent/transports/test_codex_app_server_session.py \
    tests/agent/transports/test_codex_event_projector.py \
    tests/agent/transports/test_codex_app_server_runtime.py \
    tests/agent/test_codex_app_server_event_bridge.py
# expected: 164 passed

End-to-end behaviour after the fix, on a turn that runs a shell command:

> user: list /tmp
< Discord channel (live):
    🛠 exec_command  ls /tmp
    …
    ✅ exec_command (0.04s)
    [interim] I'll inspect the directory contents.
    [final] /tmp contains foo, bar, baz.

Before the fix, the same turn produced one final message and no live signal of any kind.

Checklist

  • Conventional Commits (feat(codex):, fix(codex):, test(codex):)
  • 3 focused commits, single author (xxxigm)
  • 42 new tests pass; 122 existing codex_app_server tests pass; full agent suite confirms no new failures
  • Tested on macOS 15.6 (darwin 24.6.0), Python 3.12
  • No new config keys, no schema change, no platform-specific calls
  • Bridge is a no-op when the agent has no callbacks registered (cron / gateway-less paths)

xxxigm added 3 commits May 27, 2026 21:51
Adds ``make_codex_app_server_event_bridge(agent)`` plus four small
mapping helpers (``_codex_item_to_tool_name`` / ``_codex_item_to_args``
/ ``_codex_item_to_preview`` / ``_codex_item_completion_payload``)
that translate codex JSON-RPC ``item/*`` notifications into the
exact shape Hermes' gateway UI callbacks expect — tool names match
``CodexEventProjector`` so the progress bubbles and the projected
``tool_calls`` entries use the same identifiers.

No behaviour change yet: the next commit wires the bridge into
``run_codex_app_server_turn`` (NousResearch#33200).
…time (NousResearch#33200)

Pass ``on_event=make_codex_app_server_event_bridge(agent)`` when
spawning the per-session ``CodexAppServerSession``. The session has
always had a raw event hook but ``run_codex_app_server_turn`` never
supplied one, so Discord / Telegram / TUI users saw nothing while
codex was working — only the final answer landed.

Now each ``item/started`` for a tool-shaped item fires
``tool_progress_callback("tool.started", ...)``, ``item/completed``
fires the matching ``"tool.completed"`` with duration + result,
``item/agentMessage/delta`` flows through ``_fire_stream_delta`` and
each completed ``agentMessage`` surfaces through
``_emit_interim_assistant_message`` so the gateway's
``already_streamed`` dedupe keeps interim commentary in the channel
without duplicating text the stream already showed.
…earch#33200)

42 tests across five suites:

* ``TestCodexItemToToolName`` / ``TestCodexItemToArgs`` /
  ``TestCodexItemToPreview`` / ``TestCodexItemCompletionPayload`` —
  pin the per-type mapping so the synthetic tool name + args the
  UI sees match what ``CodexEventProjector`` writes into messages.
* ``TestStreamDeltaDispatch`` / ``TestToolProgressDispatch`` /
  ``TestAgentMessageInterimDispatch`` — drive each Codex
  notification shape through the bridge and assert the right
  agent callback fires with the right arguments (including the
  duration / is_error / result kwargs the gateway renders).
* ``TestBridgeRobustness`` — defensive paths: non-dict
  notifications, missing params, raising callbacks (must not
  tear down the codex turn loop), and agents without callbacks
  registered (cron / gateway-less contexts).
* ``TestBridgeWiredInRuntime`` — integration guard that
  ``run_codex_app_server_turn`` actually constructs the session
  with ``on_event=<bridge>``, preventing a future refactor from
  silently regressing live progress visibility again.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API codex labels May 27, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for tracing the app-server event path and supplying focused callback coverage. The underlying issue is only partially superseded: current main now surfaces tool-start breadcrumbs, but it still lacks the completion and assistant/reasoning display paths proposed here.

Problems

  • Current main already installs _on_codex_event in agent/codex_runtime.py:376-400 and preserves approval routing through _ServerRequestRouting. The PR's older constructor change must be salvaged by composing with that hook, not replacing it; otherwise it would discard current routing behavior.
  • The existing start-event path is already covered by tests/run_agent/test_codex_app_server_integration.py:696-770 (commit 2f4f23fbf). The remaining tests should target the missing completed-tool and display-event behavior on the current wiring.

Suggested changes

  • Integrate only the missing event handling into the current _on_codex_event path and retain request_routing.
  • Add integration coverage that invokes the current session callback for item/completed and the intended assistant/reasoning events.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
teknium1 added a commit that referenced this pull request Jul 17, 2026
…on show_commentary

Follow-ups on top of @xxxigm's salvaged bridge (#33294):

- Remove the now-dead narrow item/started-only mapper from #38835
  (_codex_note_to_tool_progress) — the full bridge supersedes it and
  keeps the same tool-name contract; its tests are repointed at the
  bridge helpers.
- Preserve main's request_routing/approval-bypass wiring on the
  CodexAppServerSession constructor (landed after the PR was filed).
- Gate agentMessage interim delivery on display.show_commentary so the
  app-server runtime honors the same toggle as the codex_responses
  commentary channel (tool progress is unaffected).
- Add json import (bridge helpers use json.dumps) and modernize the
  wiring test's stub agent for main's usage-accounting attributes.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #66142 — your three commits were cherry-picked onto current main with your authorship preserved in git log (e840cca, 7b63c49, 68d5368).

Worth noting: you submitted the complete bridge on May 27, before the narrower item/started-only fix (#38835) landed — your version included the tool.completed and interim-commentary halves that fix never covered, plus 42 regression tests. During salvage we reconciled with the #38835 wiring, removed the now-superseded narrow mapper, and gated agentMessage interim delivery on the new display.show_commentary toggle (from #66115) so both codex runtimes honor the same contract.

Thanks for the thorough work — the test coverage made this an easy salvage.

Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
… commentary on show_commentary

Follow-ups on top of @xxxigm's salvaged bridge (NousResearch#33294):

- Remove the now-dead narrow item/started-only mapper from NousResearch#38835
  (_codex_note_to_tool_progress) — the full bridge supersedes it and
  keeps the same tool-name contract; its tests are repointed at the
  bridge helpers.
- Preserve main's request_routing/approval-bypass wiring on the
  CodexAppServerSession constructor (landed after the PR was filed).
- Gate agentMessage interim delivery on display.show_commentary so the
  app-server runtime honors the same toggle as the codex_responses
  commentary channel (tool progress is unaffected).
- Add json import (bridge helpers use json.dumps) and modernize the
  wiring test's stub agent for main's usage-accounting attributes.
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
… commentary on show_commentary

Follow-ups on top of @xxxigm's salvaged bridge (NousResearch#33294):

- Remove the now-dead narrow item/started-only mapper from NousResearch#38835
  (_codex_note_to_tool_progress) — the full bridge supersedes it and
  keeps the same tool-name contract; its tests are repointed at the
  bridge helpers.
- Preserve main's request_routing/approval-bypass wiring on the
  CodexAppServerSession constructor (landed after the PR was filed).
- Gate agentMessage interim delivery on display.show_commentary so the
  app-server runtime honors the same toggle as the codex_responses
  commentary channel (tool progress is unaffected).
- Add json import (bridge helpers use json.dumps) and modernize the
  wiring test's stub agent for main's usage-accounting attributes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Discord gateway shows no tool-progress or interim commentary when openai-codex uses codex_app_server runtime

3 participants