feat(codex): one-shot runner, model registry, 4-way backend parity tests - #175
Merged
Conversation
…arity tests
Brings Codex up to parity with Claude/Kilo/OpenCode on the optional
`QueryBackend` surface that core features (heartbeat, dream, /status,
/model, /settings) depend on.
Added:
- `src/backend/codex/one-shot.ts` — `runOneShotAgent` for heartbeat
+ dream support. Drives `thread.runStreamed` directly, translates
Codex's full ThreadEvent surface (thread.started, turn.{started,
completed, failed}, item.completed for agent_message / reasoning /
mcp_tool_call / command_execution / file_change / web_search /
todo_list / error) into structured run-log lines.
- `src/backend/codex/models.ts` — model catalog + UnifiedModel
helpers. Codex ships a fixed set (gpt-5-codex, gpt-5, gpt-5-mini,
o4-mini) rather than a live provider catalog, so this is a
hand-maintained array — kept short because Codex's
reasoning-effort variants are configured via
`modelReasoningEffort` rather than separate model ids. Provides
resolveModel / getModelInfo / getSettingsPresentation /
getProviders / getProviderModels / formatModelError / listModels
with identical signatures to kilo/opencode/claude-sdk.
- `src/__tests__/codex-models.test.ts` — 19 tests covering all
catalog helpers (exact match, prefix ambiguity, missing,
presentation buttons, provider pagination, error formatting,
free-filter behaviour).
- `src/__tests__/codex-one-shot.test.ts` — 2 tests using a mocked
`@openai/codex-sdk` to drive a representative event sequence and
verify run-log translation (thread-started, turn lifecycle,
agent message, MCP tool call, abort path).
- `src/__tests__/backend-registry-parity.test.ts` — 6 tests proving
all four backends (claude, kilo, opencode, codex) register
themselves with non-empty labels + init functions, sorted output
from listBackends, duplicate-id rejection.
Factory now wires the full QueryBackend surface for Codex:
- resolveModel / getModelInfo / getSettingsPresentation /
getProviders / getProviderModels / formatModelError / listModels
- runOneShotAgent
- backendLabel
Heartbeat + dream now backend-agnostic across all 4 backends.
Full suite: 2301 passing (was 2274 + new tests), 12 skipped (live),
0 failing. tsc clean, lint 0 errors / 15 pre-existing warnings.
claudiusthebot
enabled auto-merge (squash)
May 15, 2026 23:50
This was referenced May 15, 2026
claudiusthebot
added a commit
that referenced
this pull request
May 16, 2026
… line (#177) Three small parity improvements building on the Codex backend foundation (#174, #175, #176): 1. **Cleanup hook in codex factory.** Kilo/OpenCode return a `cleanup: stopServer` callback from their factory `init` so the dispatcher can clean up on hot-reload or shutdown. Codex now returns a `cleanup` that calls `resetCodexState()` — clearing the cached Codex instance + frontend label + gateway port. Each `runStreamed` spawns a fresh subprocess so there's no long-lived server to stop, but resetting the module-scoped state matters for test isolation and future hot-reload paths. 2. **Backend conformance expanded to cover Codex.** Two of the four conformance tests (`routeDelivery` route parity + synthetic-error label prefix) now drive all three remote-route-using backends (Kilo / OpenCode / Codex) through the same shared `routeDelivery` helper and assert identical decisions + identical `⚠️ <Backend>: <msg>` formatting. The MCP-rotation + SSE event tests stay Kilo/OpenCode-only since Codex's MCP wiring is constructor-time TOML, not HTTP-API registration. 3. **Codex model-resolution log line.** Kilo emits `[chatId] Kilo model resolved: provider=X model=Y`; OpenCode emits the same. Codex now emits `[chatId] Codex model resolved: <model>` on every turn so the agent log has a uniform per-turn fingerprint across all four backends. Tests: 2305 passing, 12 skipped (live-tier), 0 failing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #174 (which added the Codex backend's basic
handleMessage). This PR brings Codex to parity with Claude/Kilo/OpenCode on the optional
QueryBackendsurface that core features(heartbeat, dream, /status, /model, /settings) rely on.
What's new
src/backend/codex/one-shot.ts—runOneShotAgentfor heartbeatthread.runStreameddirectly, translates Codex'sfull ThreadEvent surface into structured run-log lines:
thread.started→ records the thread idturn.started/turn.completed/turn.failed→ lifecycle markersitem.completedfor:agent_message→ "Assistant" block with the model's textreasoning→ "Reasoning" block (private scratchpad surfaced in logs)mcp_tool_call→ "MCP tool call" block with server.tool + argumentscommand_execution→ "Command" block with status + exit codefile_change→ "File changes" block with kind + path listweb_search→ "Web search" block with querytodo_list→ checkbox-style todo list renderingerror→ "Error item" blockHonors the heartbeat module's
AbortSignalso timeouts stop themodel and surface "Aborted" in the log.
src/backend/codex/models.ts— model catalog + UnifiedModelhelpers. Codex ships a fixed set rather than a live provider catalog,
so this is a hand-maintained array (gpt-5-codex, gpt-5, gpt-5-mini,
o4-mini — Codex's reasoning-effort variants flow through
modelReasoningEffortthread option rather than separate model ids).Provides
resolveModel/getModelInfo/getSettingsPresentation/getProviders/getProviderModels/formatModelError/listModelswith identical signatures to kilo / opencode / claude-sdk.src/backend/codex/factory.ts— wires the full QueryBackendsurface for Codex (resolveModel/getModelInfo/..., runOneShotAgent,
backendLabel).
Tests
19 new unit tests for
codex/models.ts:free(empty — no free Codex models) /all2 new tests for
codex/one-shot.tsusing a mocked@openai/codex-sdk:6 new tests for backend registry parity (
backend-registry-parity.test.ts):listBackends()returns them sorted by idFull suite: 2301 passing (was 2274), 12 skipped (live-tier), 0 failing.
Test plan
npx tsc --noEmitcleannpm test— 2301 passing, 12 skipped, 0 failingnpm run lint— 0 errors, 15 pre-existing warnings unchangednpm run format:checkclean🤖 Generated with Claude Code