feat(agent): bridge Hermes context and tools into Codex App Server

[FPSUnleashed]

Related Issue

Fixes #36801

Type

• Feature

Summary

Adds a first-class Hermes ↔ Codex App Server runtime and makes Codex turns stream live through the existing Hermes TUI/Desktop/Web gateway instead of appearing only after completion.

The bridge preserves Codex-native tools and MCP behavior while giving Codex the same Hermes context and recall capabilities as the default runtime.

Changes Made

• Added opt-in codex_app_server runtime support and persistent Codex thread/session handling.
• Injected Hermes system context, project instructions, memory/Honcho prefetch, and session history without persisting ephemeral recall into user history.
• Exposed hermes_history, hermes_session_search, hermes_memory, and hermes_context as Codex dynamic tools.
• Streamed assistant text, reasoning deltas when emitted, tool start/progress/completion, and terminal turn events through ACP/TUI gateway events.
• Preserved stable tool IDs between live callbacks and persisted history.
• Added strict nonempty threadId + turnId correlation, post-terminal quarantine, stale approval rejection, replay/orphan suppression, and fail-closed malformed identity handling.
• Added bounded terminal requestUserInput draining: accepted snapshot requests may finish past the normal turn deadline, while overflow is explicitly rejected.
• Added forced live redaction for arguments, results, errors, arbitrary-scheme URI userinfo/query secrets, fragmented output, and structured/truncated results.
• Updated TUI turn handling so live deltas and tool lifecycle events render without duplicating canonical history.

Verification

Exact published commit: 26585ed4723f2e1b8dd34c1e4f1b75fbdf05e52b

Pre-commit staged artifact fingerprint:

5a0932018e652687bf9cd5ae2c1b8e60e668bfaffa8d9c34040381b0c1547f2e

Independent read-only release gate: approved, zero blockers. The reviewer independently exercised malformed/whitespace item types and IDs, MCP/dynamic discriminators, immutable start identities, forward-compatible opaque items, strict callback gating, and deterministic call-ID parity.

Automated results:

37 passed                         # event projector suite
569 passed, 1 deselected          # all changed Python tests; known baseline Chromium test deselected
396 passed                        # staged runtime bridge/protocol gate
80 passed                         # focused TUI gateway handler tests
1122 passed, 4 skipped            # broader TUI suite
ruff check .                      # pass
py_compile changed Python files   # pass
Windows footgun scan: 758 files   # pass
git diff checks + git fsck        # pass
TUI typecheck + build             # pass
Desktop typecheck + build         # pass; dist assertion succeeded

The repository-wide Python and Desktop test baselines are not fully green in this macOS worktree for unrelated existing environment/fixture failures; no bridge-focused or changed-file failure remained.

Real Codex App Server smokes passed locally and after atomic deployment:

assistant streaming: observed
final response: observed
native exec_command start/completion: observed
stable start/completion tool ID: true
tool result marker: observed
post-deployment stream deltas: 33
protocol error: none

The deployed candidate passed pre-switch and post-switch protocol gates, created a rollback snapshot, restarted cleanly, and passed the full active-runtime gate after activation.

Security Notes

• Live display data is redacted with force=True before truncation or UI delivery.
• Redaction failure returns empty display data; raw values are never restored by fallback.
• Missing, blank, stale, conflicting, or malformed protocol identities fail closed before callbacks, projection, history mutation, counters, or deduplication.
• Stale or incomplete command/file approvals are declined before policy callbacks.

Screenshots

Not applicable — protocol/runtime change with automated TUI coverage.

[teknium1]

Thanks for extending the opt-in Codex runtime with Hermes context, stateful tools, and live event projection.

Problems

• agent/codex_runtime.py:880 starts a Codex turn without resetting Hermes' per-response stream state, even though its event handler sends deltas through _fire_stream_delta() (agent/codex_runtime.py:622-625). The standard provider path resets that state before every request at agent/conversation_loop.py:1188. Without the reset, _current_streamed_assistant_text and the stateful think/context scrubbers can leak from one Codex turn into the next.

Suggested changes

• Call agent._reset_stream_delivery_tracking() immediately before codex_session.run_turn(...), and cover two sequential streamed Codex turns.

Automated hermes-sweeper review.

[teknium1]

Reset Hermes' stream delivery state immediately before this call. This path emits deltas through _fire_stream_delta(), but unlike the normal provider loop (agent/conversation_loop.py:1188) it never clears _current_streamed_assistant_text or the stateful think/context scrubbers between turns.

[FPSUnleashed]

Thank you, I am trying to make it so codex can be a seamless replacement of the main agent loop, there is still a lot of work but it will be done.

[GottZ]

This was generated by AI during triage.

Summary

Three PRs address or reference #36801: #37196 exposes Codex token usage, #48485 adds threshold-based retire-and-handoff, and #63798 substantially expands the Codex App Server bridge while hardening native compaction event handling. The issue's exact unbounded Codex-owned-thread failure was subsequently fixed on main through Codex-native compaction in #60114.

Related pull requests

• fix(codex): extract turn/completed token usage on app-server runtime #37196 [closed] partial — (+102/-6) — n/a: Extracts per-turn usage from turn/completed and exposes it as codex_usage, supplying an observability prerequisite rather than proactive compaction itself; it remains relevant as the closed, narrow usage-extraction approach.
• fix(codex): proactive context-pressure retire + handoff on app-server runtime (#36801) #48485 [closed] best fix — (+244/-0) — n/a: Implements proactive threshold-based retirement of the Codex-owned thread plus auxiliary-model handoff reseeding. Although recorded as the complex's best-fix candidate, it remains closed because the maintainer-bot verdict documents that Codex-native compaction superseded this approach on main via feat(compression): route Codex app-server compaction through thread/compact/start (#36801) #60114, commit 87b65e24a, with implementation commit d1c8c03416d7c6780d31ce2371c5b0777f10afaf and agent/conversation_compression.py:1018-1126.
• feat(agent): bridge Hermes context and tools into Codex App Server #63798 related — (+7528/-673) — n/a: Primarily adds a broad Hermes–Codex context, stateful-tool, live-event, correlation, redaction, and UI bridge, including hardening around the native compaction path rather than replacing the already-landed Codex app-server runtime: long sessions grow unbounded → hard context reset (no proactive compaction) #36801 fix. Keep-open salvage is blocked on explicitly addressing the contributor review: call agent._reset_stream_delivery_tracking() immediately before codex_session.run_turn(...) and test two sequential streamed Codex turns to prevent cross-turn stream-state leakage.

Suggested consolidation

Keep #63798 open with a salvage path focused on its distinct context/tool bridge and live event projection, contingent on resolving the contributor's stream-reset review and preferably splitting the reviewable bridge components from the 7,528-line aggregate. Leave #37196 and #48485 closed: #48485's recorded best-fix status is superseded by the documented Codex-native implementation on main, while #37196 is a narrower observability precursor rather than the implemented compaction solution.

Cross-PR triage: Reviewed 3 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 444 kB of PR diffs, 14 kB of issue/PR text, 6 kB of discussion (7 comments), 3 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.