fix(agent): stream Codex commentary separately from reasoning

[100yenadmin]

What does this PR do?

This PR gives Codex phase="commentary" a first-class, user-visible streaming path without exposing private reasoning.

It supersedes #60453 while preserving David Robertson's two original commits and authorship. On top of that work, it adds the lifecycle, deduplication, fallback, and security fixes found during adversarial review against current main.

The user-visible contract is intentionally narrow:

Codex event / phase	Hermes destination	Visibility
phase="commentary"	existing interim_assistant_callback	visible progress narration
response.reasoning_text.* / reasoning summaries	existing reasoning callback	controlled by show_reasoning
phase="analysis"	reasoning-only path	never promoted to commentary
phase="final_answer"	normal assistant stream	final response

This means gateways can keep show_reasoning: false and still show polished Codex progress updates, matching the commentary/final channel split used by Codex clients.

Related issue and PRs

• Canonical commentary leak/routing family: [Bug]: Codex Responses commentary-phase tool planning leaks as visible Telegram text #24933
• Supersedes: fix(agent): surface Codex commentary items as interim messages #60453
• Related alternative implementations: fix(codex): surface commentary as interim assistant prose #59831, fix(codex): route commentary/analysis narration to a dedicated display lane #62396

Type of change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security hardening (redaction is preserved on the new visible route)
• 📝 Documentation update
• ✅ Tests (adds lifecycle, fallback, compatibility, and regression coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Why this supersedes #60453

#60453 found the correct user-facing seam and added the first safe extraction path. This PR retains those commits, including think-block stripping and secret redaction, then fixes cases that could otherwise duplicate, delay, omit, or misclassify commentary:

• emits completed commentary items immediately instead of waiting for the entire response;
• prefers final_answer content when a response contains both commentary and final text;
• keeps analysis and reasoning events off the visible commentary rail;
• supports delta, text-done, and output-item-done fallbacks;
• tracks multiple output items independently;
• deduplicates incomplete-to-tool transitions and transport retries;
• handles identical commentary items without collapsing distinct occurrences;
• prevents aggregate fallbacks from re-emitting items already delivered live;
• preserves the legacy CLI reasoning fallback when no interim callback exists;
• does not mutate phase-bearing replay items used for continuation/prompt-cache behavior.

Related design work:

• fix(codex): surface commentary as interim assistant prose #59831 explores moving this routing into the Codex runtime and raised the finalized-event and interleaved-item robustness cases covered here.
• fix(codex): route commentary/analysis narration to a dedicated display lane #62396 proposes a new persisted end-to-end Working UI protocol. That can still be evaluated independently; this PR deliberately uses Hermes' existing interim-assistant contract to keep the change backward-compatible and scoped.

Routing design

flowchart LR
    SSE["Codex Responses SSE"] --> Kind{"Event / output phase"}
    Kind -->|"phase=commentary"| Item["Assemble one output item"]
    Item --> Safe["Strip think blocks<br/>Redact sensitive text"]
    Safe --> Interim["interim_assistant_callback"]
    Kind -->|"reasoning events<br/>or phase=analysis"| Reasoning["reasoning_callback"]
    Kind -->|"phase=final_answer"| Final["normal stream_delta_callback"]
    Interim --> Gateway["Gateway progress message"]
    Reasoning --> Thinking["Shown only when reasoning is enabled"]
    Final --> Reply["Assistant final response"]

Loading

Commentary is emitted at the earliest authoritative lifecycle boundary:

sequenceDiagram
    participant C as Codex SSE
    participant H as Hermes runtime
    participant G as Gateway
    C->>H: commentary deltas
    C->>H: commentary output_item.done
    H->>H: strip think blocks + redact + deduplicate
    H-->>G: interim commentary
    C->>H: function call / tool transition
    H-->>G: existing tool progress
    C->>H: final_answer deltas
    H-->>G: normal final response stream

Loading

Implementation notes

• agent/codex_runtime.py exposes a commentary callback alongside the existing reasoning and final-stream callbacks.
• agent/conversation_loop.py and agent/agent_init.py wire that callback through the existing interim assistant delivery path.
• run_agent.py owns per-turn delivery state so retries and response fallbacks remain idempotent.
• The wire fixtures use the actual Responses API event shapes, including response.reasoning_text.delta.
• No configuration migration or new public gateway protocol is required.

Safety and compatibility

• Commentary is sanitized before any user-facing callback: <think> content is removed and redact_sensitive_text() is applied.
• phase="analysis" is never promoted to the visible path.
• show_reasoning: false continues to hide reasoning; it no longer hides commentary.
• Callers that do not provide the new commentary callback retain the previous CLI behavior.
• Final-answer streaming and tool execution remain on their existing rails.
• The patch is additive at callback boundaries and does not alter stored Codex message-item phases.

Verification

Rebased onto main at 53adb3fd9750376b520100b0f90b737da802d1e1; reviewed head: 48a48b85ca9a8a935d75590136fbd4cae51c1af6.

Focused local proof: 521 passed, 0 failed.

scripts/run_tests.sh tests/run_agent/test_run_agent_codex_responses.py
# 102 passed

scripts/run_tests.sh tests/run_agent/test_provider_parity.py -k 'codex or commentary or message_items'
# 38 passed

scripts/run_tests.sh tests/agent/test_redact.py
# 149 passed

scripts/run_tests.sh tests/agent/test_auxiliary_client.py tests/agent/test_codex_responses_adapter.py -k 'codex or commentary or analysis'
# 87 passed

scripts/run_tests.sh tests/gateway -k 'commentary or interim or progress'
# 145 passed across 479 discovered gateway files

Also passed:

• Ruff on all changed Python files (one pre-existing invalid-noqa warning at run_agent.py:107)
• compileall on all changed Python files
• git diff --check
• adversarial review of security, lifecycle, compatibility, and fallback/dedup behavior, with all >=95%-confidence findings fixed and regression-tested

Not claimed here: a live external Codex provider or messaging-platform smoke. The deterministic stream fixtures exercise the same event lifecycle without consuming credentials or sending messages.

Review guide

1. Start with the callback contract in agent/codex_runtime.py.
2. Check callback wiring in agent/agent_init.py and agent/conversation_loop.py.
3. Review per-turn delivery/dedup logic in run_agent.py.
4. Use the named regression tests in tests/run_agent/test_run_agent_codex_responses.py to verify each lifecycle edge case.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched existing PRs and documented the three overlapping approaches above
• This PR contains only changes related to Codex commentary routing and its tests
• I've run pytest tests/ -q locally — the 39k-test repository suite is left to canonical CI; 521 focused tests pass locally
• I've added tests for the behavior and regressions
• Tested on macOS (focused deterministic Python suites)

Documentation and housekeeping

• Relevant documentation is N/A; the callback and lifecycle contracts are documented in code and this PR
• cli-config.yaml.example is N/A; no config keys changed
• CONTRIBUTING.md / AGENTS.md updates are N/A; no contributor workflow changed
• Cross-platform impact considered; the change is platform-neutral Python callback/state logic
• Tool descriptions/schemas are N/A; no tool behavior or schema changed

Behavioral acceptance

• Original fix(agent): surface Codex commentary items as interim messages #60453 commits and authorship preserved
• Commentary visible without enabling reasoning
• Analysis/reasoning remains private
• Final-answer streaming unchanged
• Secret redaction covered by regression tests
• Multi-item, fallback, retry, and dedup cases covered
• Current main rebase and focused validation complete

Credit: David Robertson (@davidrobertson) for the original #60453 diagnosis and safe extraction path.

[null-runner]

Independently verified this PR on a live deployment (WSL2, systemd gateway + desktop/dashboard backends, brain gpt-5.6-sol via openai-codex).

Repro before the patch: bundled prompts like "write just 'ciao', then count the files in /tmp" returned only the tool result — the greeting landed in the hidden reasoning channel (phase=commentary demoted by ea125dd / 538173f). We also hit the nastier variant: the model asked a clarifying question whose rationale/options stayed in commentary, so the user saw a bare question with no context.

After cherry-picking these 4 commits onto current main:

• tests/run_agent/test_run_agent_codex_responses.py: 102 passed
• full tests/run_agent/ suite: 2059 passed, 4 skipped, 0 failures
• live CLI (hermes -z): the same bundled prompt now prints ciao before the tool result
• gateway surfaces deliver commentary as a separate interim message; no duplicate re-emission observed across tool continuations

Compared with the sibling PRs (#59831 streaming-only, #60453 superseded here with authorship preserved, #62396 much larger UI surface), this one covers both the streaming and non-streaming paths with dedup and secret redaction, which is what made it adoptable as-is. Would be great to see it land.

[teknium1]

Merged via PR #66115 with your commits cherry-picked onto current main — authorship preserved in git log (136ade2, a15397d, plus David Robertson's two originals from #60453).

Thanks for finding #60453, fixing it up, and getting it tested end-to-end for codex models — and thanks to @null-runner for the independent live verification. We added a display.show_commentary config toggle on top (default true) so users who find the extra narration noisy can route commentary back to the reasoning channel.