feat(backend): add OpenAI Agents SDK as fifth backend

[claudiusthebot]

Summary

Adds the OpenAI Agents SDK (@openai/agents) as Talon's fifth backend, alongside claude / kilo / opencode / codex. Selectable via backend: "openai-agents" in talon.json.

Originally filed against OpenAI's API directly; updated 2026-05-18 per Dylan's feedback to support arbitrary OpenAI-compatible endpoints out of the box — OpenRouter, Azure OpenAI, Ollama, LiteLLM, vLLM, Portkey, Together, Groq's compat surface, anything that speaks the protocol. Supersedes #202 (which was a special-cased OpenRouter redirect; now folded in here with generic naming).

Endpoint configuration

Two new optional config fields (and matching env vars), both generic — not OpenRouter-named:

• openaiBaseUrl / OPENAI_BASE_URL — redirect target. Empty = OpenAI direct.
• openaiApiMode / OPENAI_API_MODE — "responses" (OpenAI native, default) or "chat_completions" (most third parties). Auto-defaults to "chat_completions" when openaiBaseUrl is set, because most non-OpenAI endpoints don't implement Responses. Set explicitly to "responses" only if your proxy supports it.

The @openai/agents SDK exposes setDefaultOpenAIClient(client) and setOpenAIAPI(mode) — init.ts constructs a custom OpenAI({apiKey, baseURL}) client whenever either field is set and calls both globally. All subsequent new Agent({...}) instances inherit the redirect; no per-call wiring needed.

Example configs (all written to talon.json):

// OpenAI direct (default)
{ "backend": "openai-agents", "openaiApiKey": "sk-...", "model": "gpt-5.5" }

// OpenRouter
{ "backend": "openai-agents",
  "openaiApiKey": "sk-or-...",
  "openaiBaseUrl": "https://openrouter.ai/api/v1",
  "model": "meta-llama/llama-3.3-70b-instruct" }

// Local Ollama
{ "backend": "openai-agents",
  "openaiBaseUrl": "http://localhost:11434/v1",
  "model": "llama3.2:3b" }

// Azure OpenAI
{ "backend": "openai-agents",
  "openaiApiKey": "...",
  "openaiBaseUrl": "https://<resource>.openai.azure.com/openai/v1",
  "model": "gpt-4o-mini" }

Model catalog passthrough

The hardcoded catalog (gpt-5.5 / gpt-5 / gpt-5-mini / o4-mini) still works for OpenAI-native users. When openaiBaseUrl is set, resolveModel() and getModelInfo() accept arbitrary model ids as synthetic passthrough entries — so users can target meta-llama/llama-3.3-70b-instruct, qwen/qwen3-coder, llama3.2:3b, etc. without us maintaining a separate catalog per provider. Catalog hits are still preferred for OpenAI-native ids even with a baseURL set, so gpt-5.5 on a proxy that mirrors OpenAI's namespace still resolves cleanly.

formatModelError() points users at direct model: config in talon.json when a custom baseURL is set, instead of nagging about an unrelated OpenAI-flagship catalog.

CLI wizard + doctor + status

• openai-agents branch of setup wizard prompts API key → base URL → (if base URL set) Chat Completions vs Responses.
• talon status displays configured openaiBaseUrl + openaiApiMode.
• talon doctor reports OpenAI Agents auth state and resolved endpoint (defaults to api.openai.com when no override).

What's in the module

src/backend/openai-agents/ (~900 LOC) — mirrors the codex backend shape with the same QueryBackend surface:

• constants.ts — system-prompt suffix, default model (gpt-5.5), OPENAI_AGENTS_MAX_TURNS=50, agent name surfaced in OpenAI traces.
• models.ts — catalog + passthrough resolver.
• init.ts — endpoint resolution + setDefaultOpenAIClient / setOpenAIAPI wiring + auth-state logging.
• mcp.ts — MCPServerStdio[] from plugin map (one server per non-terminal frontend + brave-search + every plugin).
• handler.ts — streaming agent run via run(agent, prompt, {stream: true}). Translates RunItemStreamEvent (tool_called, message_output_created) into shared stream-state. Hooks abort controller for terminator-driven cancel. Closes MCP servers in finally.
• factory.ts — registers as openai-agents with the standard QueryBackend shape.
• state.ts — singleton config + frontend handle.

Wiring: src/bootstrap.ts extra side-effect import, src/util/config.ts adds the enum entry + new fields, src/cli.ts new menu/wizard/doctor branches.

Tests (+35 cases total)

• openai-agents-models.test.ts — 27 catalog tests (resolve, getModelInfo, providers, presentation, error formatting, listModels) + 8 new custom-endpoint tests: getOpenAIBaseUrl env-vs-config priority, custom-baseURL passthrough resolution, catalog hits still preferred, synthetic getModelInfo entries, baseURL-aware formatModelError.
• openai-agents-backend.test.ts — constants, state lifecycle, factory registration.
• backend-registry-parity.test.ts — five-backends parity assertions.

Acknowledged debt (callouts in the handler header)

• No session persistence — the Agents SDK has a Session abstraction but Talon's session model is per-chat numeric ids; wiring is a future PR.
• No progress-text emission before tool calls — the SDK's raw-stream events don't cleanly split into pre-tool text segments the way Claude SDK's do.
• No flow-violation reminder retry — single-pass for now. Base retry ladder (session_expired / context_length / fallback_model) still runs.

Verified

• npx tsc --noEmit clean
• npm test — 2449 passing, 12 skipped (live), 2 pre-existing opencode-integration timeouts unrelated to this change
• npm run lint — 11 pre-existing warnings, 0 errors
• npx prettier --check clean

Live verification path with a custom endpoint: set OPENAI_API_KEY + OPENAI_BASE_URL in env (or the matching fields in talon.json), flip backend: "openai-agents", send a message.

Out of scope (future PRs)

• A Backend Live (openai-agents, *) CI matrix job. Needs an API key secret; SDK won't smoke-test without one.
• Session persistence wiring via the SDK's Session abstraction.
• Streaming onStreamDelta callbacks for typing-indicator UX.

🤖 Generated with Claude Code

[claudiusthebot]

CI fix pushed (ed9eb0e).

Root cause of the all-jobs failure: @emnapi/core@1.10.0 and @emnapi/runtime@1.10.0 were missing from the lockfile. These are optional dev transitive deps (via @openai/codex) that exist in main's lockfile, but the lockfile was generated on an arm64/platform that doesn't need them — so they were dropped when npm install ran locally. CI (ubuntu-latest x64) considers them required and fails npm ci with EUSAGE immediately, which is why every job failed in under 10 seconds.

Fix: manually restored the two entries from main's lockfile. npm ci --dry-run now exits 0. CI re-running on the new commit.

[claudiusthebot]

Iterated on this per your feedback. Closed #202 — it was over-specified as an "OpenRouter redirect" when the @openai/agents SDK already supports arbitrary custom endpoints via setDefaultOpenAIClient(new OpenAI({apiKey, baseURL})). Folded the support into this PR directly with generic naming so it works with any OpenAI-compatible service, not just OpenRouter.

New commit ceacb9e on top of the existing two:

1. openaiBaseUrl + openaiApiMode config fields (also OPENAI_BASE_URL / OPENAI_API_MODE env). Generic, not OR-specific.
2. init.ts constructs new OpenAI({apiKey, baseURL}), calls setDefaultOpenAIClient() and setOpenAIAPI(mode). Auto-defaults to chat_completions when baseURL is set since most non-OpenAI endpoints don't implement Responses.
3. Model resolver accepts arbitrary ids when a custom baseURL is set (passthrough) — so meta-llama/llama-3.3-70b-instruct / llama3.2:3b / qwen/qwen3-coder etc work without needing a per-provider catalog. Catalog hits are still preferred for OpenAI-native ids.
4. CLI: wizard prompts API key → base URL → Chat Completions vs Responses; talon status shows the new fields; talon doctor reports auth + endpoint state for the openai-agents backend.
5. 8 new tests in openai-agents-models.test.ts covering the passthrough path. 35 / 35 openai-agents tests green.

Rebased onto current main while I was here (PR was behind before). Force-pushed with --force-with-lease against the pre-rebase tip.

Examples in the updated PR description:

• OpenAI direct (status quo)
• OpenRouter (https://openrouter.ai/api/v1)
• Local Ollama (http://localhost:11434/v1)
• Azure OpenAI (https://<resource>.openai.azure.com/openai/v1)

After June 1 (Max end), this gives a clean path off Anthropic billing: free-tier OpenRouter models for heartbeat/dream, anything else you want for chat. Or stay on OpenAI direct with a paid key. Or local Ollama if you want zero marginal cost and don't mind the latency. Same backend either way.