Repository navigation
feat(backend): add Codex as a fourth backend provider - #174
Merged
Merged
Conversation
Wraps `@openai/codex-sdk` (which drives the `codex` CLI from `@openai/codex`) and registers it as `backend: "codex"`. Selectable alongside Claude SDK, Kilo, and OpenCode. Reference design from https://github.com/pingdotgg/t3code — t3code's `effect-codex-app-server` package writes its own JSON-RPC client against the raw Codex `app-server` protocol; Talon takes the lighter path and uses the official `@openai/codex-sdk` which already wraps that protocol cleanly. Architecture: - `constants.ts` — system-prompt suffix, default model (`gpt-5-codex`). - `state.ts` — per-process state container (config, cached Codex instance, gateway port, frontend label). - `mcp-config.ts` — flattens Talon's MCP plugin map into the `mcp_servers.<name>.{command,args,env}` TOML shape Codex's CLI accepts via `--config` overrides. Covers frontend-tools servers, plugin servers, and Brave Search. - `init.ts` — lazy `Codex` instance construction, keyed by active chat id so per-chat MCP isolation survives the CLI's "configure-at-startup" constraint. - `handler.ts` — main message handler. Subscribes to `thread.runStreamed`'s ThreadEvent generator, translates `agent_message` / `mcp_tool_call` items into the shared stream state, persists thread id for `resumeThread()` continuity. Hooks AbortController so terminator tools (end_turn / send / react) can cancel further model generation. - `factory.ts` — `registerBackend({ id: "codex", ... })`. - `index.ts` — barrel. Shared infrastructure reused: - `backend/shared/stream-state` for accumulator. - `backend/shared/delivery` (routeDelivery) for end-of-turn routing. - `backend/shared/{prompt-format,system-prompt,model-retry,session-name,usage}`. - `core/tools/index.isTurnTerminator` for tool-name detection (Codex emits MCP calls with bare tool names, no MCP prefix — `stripMcpPrefix` handles both shapes). Notes / caveats: - The Codex SDK doesn't expose `system` on `runStreamed`. Talon prepends the system prompt to the first user message on first turn; subsequent resumed turns inherit instructions from the persisted thread. - MCP servers are configured at thread-creation time (CLI constraint). Chat-id-keyed cache rebuilds the Codex instance on chat switch so each chat sees its own MCP environment. - Cache-write tokens always reported as 0 — Codex doesn't surface them in `Usage`. Config: - New `backend: "codex"` value accepted by `configSchema`. - New `openaiApiKey` config option (falls back to `OPENAI_API_KEY` env var). Prerequisites for the user: - `npm i -g @openai/codex` (provides the `codex` CLI). - `codex login` (ChatGPT auth) OR `OPENAI_API_KEY` env var OR `openaiApiKey` in `~/.talon/config.json`. Tests: 13 new unit tests in `codex-backend.test.ts` covering constants, state lifecycle, MCP-config flattening (frontend servers, plugin servers, brave-search opt-in, multi-frontend), and factory registration. Full suite 2274 passing (was 2261), 12 skipped (live-tier), 0 failing. README: - Backends badge updated to include Codex. - "Backends" section gains a Codex row. - Prerequisites + config-table rows updated. - Architecture tree adds `codex/`. Live verification: not exercised — requires `codex` CLI installed + OpenAI API key. A `docker/codex-test/` harness can be added in a follow-up similar to `docker/kilo-test/` once the SDK has been shaken out against real usage.
claudiusthebot
enabled auto-merge (squash)
May 15, 2026 23:17
…dex-sdk only
CI failed on the initial codex backend PR with:
npm error Missing: @emnapi/core@1.10.0 from lock file
npm error Missing: @emnapi/runtime@1.10.0 from lock file
Root cause: the first `npm install --include=optional` regenerated
the lockfile broadly, dropping main's `node_modules/@emnapi/core` and
`@emnapi/runtime` top-level entries even though three packages
(rolldown wasm bindings) still listed them as dependencies. Bare
`npm ci` (no flags) on a clean runner therefore couldn't resolve
the closure.
This commit uses a Python merge script to take main's lockfile as
base and add ONLY the codex-sdk-specific entries:
- node_modules/@openai/codex
- node_modules/@openai/codex-{darwin,linux,win32}-{arm64,x64}
(per-platform binaries — optional, only one installs per host)
- node_modules/@openai/codex-sdk
Everything else in main's lockfile (tsx@4.21.0, ajv@8.18.0,
@emnapi/core@1.10.0, etc) is preserved byte-for-byte. Result:
- bare `npm ci` passes
- tools/verify-lockfile.sh passes
- tsc clean
- 2274 tests passing, 0 failing
Reapplies the §-on-lockfile-bumps lesson from PR #105: when adding a
single new dep, prefer surgical merge over a broad `npm install` so
unrelated transitive versions don't shift.
This was referenced May 15, 2026
claudiusthebot
added a commit
that referenced
this pull request
May 16, 2026
… line (#177) Three small parity improvements building on the Codex backend foundation (#174, #175, #176): 1. **Cleanup hook in codex factory.** Kilo/OpenCode return a `cleanup: stopServer` callback from their factory `init` so the dispatcher can clean up on hot-reload or shutdown. Codex now returns a `cleanup` that calls `resetCodexState()` — clearing the cached Codex instance + frontend label + gateway port. Each `runStreamed` spawns a fresh subprocess so there's no long-lived server to stop, but resetting the module-scoped state matters for test isolation and future hot-reload paths. 2. **Backend conformance expanded to cover Codex.** Two of the four conformance tests (`routeDelivery` route parity + synthetic-error label prefix) now drive all three remote-route-using backends (Kilo / OpenCode / Codex) through the same shared `routeDelivery` helper and assert identical decisions + identical `⚠️ <Backend>: <msg>` formatting. The MCP-rotation + SSE event tests stay Kilo/OpenCode-only since Codex's MCP wiring is constructor-time TOML, not HTTP-API registration. 3. **Codex model-resolution log line.** Kilo emits `[chatId] Kilo model resolved: provider=X model=Y`; OpenCode emits the same. Codex now emits `[chatId] Codex model resolved: <model>` on every turn so the agent log has a uniform per-turn fingerprint across all four backends. Tests: 2305 passing, 12 skipped (live-tier), 0 failing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Codex (OpenAI's coding agent CLI, via
@openai/codex-sdk) asa fourth backend alongside Claude SDK / Kilo / OpenCode. Selectable
via
backend: \"codex\"in~/.talon/config.json.Reference design taken from pingdotgg/t3code as requested. t3code's
effect-codex-app-serverpackage writes its own JSON-RPC client against theraw Codex
app-serverprotocol. Talon takes the lighter path and usesthe official
@openai/codex-sdkwhich already wraps that protocolcleanly — fewer LOC and tracks upstream API changes for free.
Architecture
New
src/backend/codex/:constants.ts— system-prompt suffix + default model (gpt-5-codex).state.ts— per-process state container.mcp-config.ts— flattens Talon's MCP plugin map into themcp_servers.<name>.{command,args,env}TOML shape Codex's CLI accepts via--configoverrides. Covers frontend-tools servers (telegram/discord/teams), plugin servers (mempalace/github/playwright/etc), and Brave Search.init.ts— lazyCodexinstance construction, keyed by active chat id so per-chat MCP isolation survives the CLI's "configure-at-startup" constraint.handler.ts— main message handler. Subscribes tothread.runStreamed'sThreadEventgenerator, translatesagent_message/mcp_tool_callitems into the shared stream state, persists thread id forresumeThread()continuity. HooksAbortControllerso terminator tools (end_turn / send / react) can cancel further model generation.factory.ts—registerBackend({ id: \"codex\", ... }).index.ts— barrel.Shared infrastructure reused
backend/shared/stream-statefor accumulator.backend/shared/delivery(routeDelivery) for end-of-turn routing — Codex gets the same 4-route decision (tool / synthetic-error / text-part / empty) as Kilo and OpenCode.backend/shared/{prompt-format,system-prompt,model-retry,session-name,usage}.core/tools/index.isTurnTerminatorfor tool-name detection.Caveats / known limitations
systemonrunStreamed. Talon prepends the system prompt to the first user message on first turn; subsequent resumed turns inherit instructions from the persisted thread.Usage.Config
backend: \"codex\"accepted byconfigSchema.openaiApiKeyconfig option (falls back toOPENAI_API_KEYenv var).Prerequisites
npm i -g @openai/codex(provides thecodexCLI).codex login(ChatGPT auth) OROPENAI_API_KEYenv ORopenaiApiKeyin talon config.Tests
13 new unit tests in
codex-backend.test.ts:id: \"codex\"in the backend registry.Full suite: 2274 passing (was 2261), 12 skipped (live-tier), 0 failing.
Live verification: not exercised — requires the
codexCLI installed + OpenAI API key. Adocker/codex-test/harness can be added in a follow-up similar todocker/kilo-test/once the SDK has been shaken out against real usage.README
codex/.Test plan
npx tsc --noEmitcleannpm test— 2274 passing, 12 skipped, 0 failingnpm run lint— 0 errors, 15 pre-existing warnings unchangednpm run format:checkcleancodex login+backend: \"codex\"in talon.json + DM the bot🤖 Generated with Claude Code