Skip to content

feat(backend): add OpenRouter redirect to OpenAI Agents backend - #202

Closed
claudiusthebot wants to merge 1 commit into
feat/openai-agents-backendfrom
feat/openai-agents-openrouter
Closed

feat(backend): add OpenRouter redirect to OpenAI Agents backend#202
claudiusthebot wants to merge 1 commit into
feat/openai-agents-backendfrom
feat/openai-agents-openrouter

Conversation

@claudiusthebot

@claudiusthebot claudiusthebot commented May 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Depends on PR #199. Merge that first, then rebase + merge this one.

Adds OpenRouter support to the openai-agents backend (PR #199) via three small changes:

  • src/util/config.ts — two new optional config fields:
    • openrouterApiKey — OR API key (falls back to OPENROUTER_API_KEY env)
    • openrouterBaseUrl — optional base URL override (defaults to https://openrouter.ai/api/v1)
  • src/backend/openai-agents/init.ts — if an OpenRouter key is present, creates a custom OpenAI({apiKey, baseURL}) client and calls setDefaultOpenAIClient() before returning. The Agents SDK picks this up globally; no handler changes needed.
  • src/backend/openai-agents/constants.ts — adds OPENAI_AGENTS_OPENROUTER_BASE_URL constant + updated JSDoc.

Why

PR #199's backend hardcodes to OpenAI's API (no baseURL override). To route through OpenRouter for the June 1 migration (Max subscription end), we need setDefaultOpenAIClient() called at init with an OR-pointed client.

OpenRouter is OpenAI API-compatible, so the Agents SDK works without changes to handler.ts. Only the client's base URL and API key change.

Migration config (for June 1)

After merging #199 + this PR, set in talon.json:

{
  "backend": "openai-agents",
  "openrouterApiKey": "<your-key>",
  "model": "meta-llama/llama-3.3-70b-instruct"
}

meta-llama/llama-3.3-70b-instruct is the best free-tier general-purpose pick (70B always-active, well-validated, identified in heartbeat #308 free-tier audit). Other free options: deepseek/deepseek-v4-flash, qwen/qwen3-coder, openai/gpt-oss-120b.

Auth priority in init

  1. OPENROUTER_API_KEY env / config.openrouterApiKey → injects custom OR client
  2. OPENAI_API_KEY env / config.openaiApiKey → SDK default (direct OpenAI billing)
  3. Neither → startup warning, first turn fails

Tests

No new tests in this PR — the init path is a thin setDefaultOpenAIClient() call that's hard to unit-test without mocking the SDK global. Verified the logic is correct by reading the @openai/agents 0.11.4 source (confirmed in heartbeat #270 SDK audit). Dylan or a follow-up PR can add a spy test for the client injection.

Notes

  • getOpenAIApiKey() returns undefined when OR is configured — documented in JSDoc. This is correct; the OR client is injected globally, not per-call.
  • openrouterBaseUrl also serves non-OR deployments (Azure OpenAI, local Ollama + openai-compat proxy) — named openrouterBaseUrl because that's the primary use case but the field is general.

Filed by claudiusthebot heartbeat #311 (2026-05-17 ~10:05Z) — June 1 migration prep.

@claudiusthebot
feat(backend): add OpenRouter redirect to OpenAI Agents backend
ab01e1f 
When `openrouterApiKey` (or OPENROUTER_API_KEY env) is set, init injects
a custom OpenAI client via setDefaultOpenAIClient() pointed at OpenRouter's
OpenAI-compatible API. No handler changes needed — the SDK picks up the
global client automatically.

Config additions:
  - openrouterApiKey — the OR API key
  - openrouterBaseUrl — optional override (defaults to OR's v1 endpoint)

Recommended free-tier model for heartbeats (set as `model` in talon.json):
  meta-llama/llama-3.3-70b-instruct

Depends on PR #199 (openai-agents backend). Rebase onto main after that
PR merges before merging this one.
@claudiusthebot

claudiusthebot commented May 18, 2026

Copy link
Copy Markdown
Collaborator Author

Closing — Dylan said we don't need this as a separate PR. The right design is to put generic OpenAI-compatible endpoint support (custom baseURL + apiKey) directly into PR #199 rather than have an OpenRouter-named redirect on top.

Folding the actual support into #199 with proper generic naming (openaiBaseUrl instead of openrouterBaseUrl), plus an openaiApiMode toggle so non-OpenAI endpoints can fall back to Chat Completions (most don't implement the Responses API).

dylanneve1 pushed a commit that referenced this pull request May 18, 2026
@claudiusthebot
feat(openai-agents): generic OpenAI-compatible endpoint support
ceacb9e 
Dylan flagged that the original v0 of this backend hardcoded to OpenAI's
production API, and a follow-up PR (#202) tried to add OpenRouter as a
special-cased redirect. The @openai/agents SDK already supports arbitrary
custom OpenAI-compatible endpoints via setDefaultOpenAIClient() — so the
right design is generic baseURL config on this backend, not an OpenRouter-
named adapter. PR #202 is closed; folding the support in here properly.

## What this adds

Two new optional config fields (plus matching env vars), both generic:

  - `openaiBaseUrl` / `OPENAI_BASE_URL`
      Redirects the SDK at any OpenAI-compatible service. Works for
      OpenRouter, Azure OpenAI, local Ollama, LiteLLM, Portkey, vLLM,
      Together, Groq's OpenAI-compat surface — anything that speaks the
      protocol.
  - `openaiApiMode` / `OPENAI_API_MODE`
      Which OpenAI API surface to target: "responses" (OpenAI native) or
      "chat_completions" (most third parties). Defaults to "responses"
      with no baseURL, and "chat_completions" automatically when a
      custom baseURL is set — since most non-OpenAI endpoints don't
      implement the Responses API.

`init.ts` constructs a custom `OpenAI({apiKey, baseURL})` client and
calls `setDefaultOpenAIClient()` whenever either field is configured.
`setOpenAIAPI()` is called unconditionally with the resolved mode so the
startup log line accurately reflects which surface subsequent `run()`
calls hit.

## Model catalog passthrough

`resolveModel()` and `getModelInfo()` previously rejected anything not in
the hardcoded `gpt-5.5 / gpt-5 / gpt-5-mini / o4-mini` catalog. Now: when
`openaiBaseUrl` is set, any unknown model id is accepted as a synthetic
passthrough entry — so users can target `meta-llama/llama-3.3-70b-instruct`
on OpenRouter, `llama3.2:3b` on Ollama, etc. without us maintaining a
separate catalog per third-party provider. Exact catalog hits still win
for OpenAI-native models even with a baseURL set.

## CLI wizard + doctor

- `openai-agents` branch of setup wizard prompts for API key, then base
  URL, then (if a base URL is set) Chat Completions vs Responses.
- `talon status` now shows configured `openaiBaseUrl` + `openaiApiMode`.
- `talon doctor` adds an `openai-agents` branch reporting auth state and
  endpoint config (defaults to api.openai.com when none).

## Tests

8 new cases in openai-agents-models.test.ts covering the passthrough
path: env-vs-config priority, custom-baseURL resolution, catalog hits
still preferred, getModelInfo synthetic entries, formatModelError
pointing users at direct config. 35 / 35 openai-agents tests pass; full
suite at 2449 / 2463 (2 pre-existing opencode integration-test
timeouts unrelated to this change).

Verified:
  - npx tsc --noEmit clean
  - npm run lint 0 errors, 11 pre-existing warnings
  - npx prettier --check clean
  - vitest 35/35 backend tests pass
@claudiusthebot claudiusthebot mentioned this pull request May 18, 2026
Merged
dylanneve1 pushed a commit that referenced this pull request May 18, 2026
@claudiusthebot
feat(backend): add OpenAI Agents SDK as fifth backend (#199)
d9a6e90 
* feat(backend): add OpenAI Agents SDK as fifth backend

Dylan asked for OpenAI Agents SDK support as a possible new backend
(`https://developers.openai.com/api/docs/guides/agents`). This adds
it alongside claude / kilo / opencode / codex, registered as
`backend: "openai-agents"` in talon.json.

## What's in scope (v0)

The backend speaks to OpenAI's Responses API via
`@openai/agents@^0.11.4` (the official Node SDK). Talon uses it as
a single-agent backend (no handoffs, no guardrails) with MCP servers
wired from the plugin system.

  - **`src/backend/openai-agents/`** — new module, ~900 LOC, mirrors
    the `codex` backend's shape with the same `QueryBackend` surface.
    - `constants.ts` — system-prompt suffix + default model
      (`gpt-5.5`, the broadest-access flagship), max turns (50),
      agent name surfaced in OpenAI traces.
    - `models.ts` — hand-maintained catalog (`gpt-5.5`, `gpt-5`,
      `gpt-5-mini`, `o4-mini`). `gpt-5-codex` is intentionally NOT
      in this catalog — that model is Codex-CLI-only.
    - `auth` via `init.ts`: `OPENAI_API_KEY` env > `openaiApiKey`
      in config > startup warning. No ChatGPT-OAuth fallback —
      the Responses API requires a paid API key.
    - `mcp.ts` — builds `MCPServerStdio[]` from Talon's plugin map.
      One server per non-terminal frontend + brave-search + every
      configured plugin. Parallel `connect()`; close-all on error.
    - `handler.ts` — streaming agent run via `run(agent, prompt,
      {stream: true})`. Translates `RunItemStreamEvent` events
      (`tool_called`, `message_output_created`) into the shared
      stream-state. Hooks the abort controller for terminator-
      driven cancellation. Closes MCP servers in `finally` to avoid
      subprocess leaks.
    - `factory.ts` — registers as `openai-agents` with the standard
      `QueryBackend` shape.
    - `state.ts` — singleton per-process config + frontend handle.

  - **Bootstrap wiring** (`src/bootstrap.ts`): one extra
    side-effect import, additive only.
  - **Config schema** (`src/util/config.ts`): adds `"openai-agents"`
    to the `backend` enum.
  - **CLI setup wizard** (`src/cli.ts`): new menu entry +
    `OPENAI_API_KEY` prompt. The label note explicitly calls out
    "no ChatGPT-OAuth fallback" so users with only `codex login`
    auth don't pick this backend and then 401 on first turn.

## Tests (+27 cases, 2423 → 2435 total)

  - `openai-agents-models.test.ts` (19 tests) — catalog presence,
    `resolveModel` / `getModelInfo` / `getSettingsPresentation` /
    providers / `listModels` / error formatting.
  - `openai-agents-backend.test.ts` (8 tests) — constants, state
    lifecycle, factory registration.
  - `backend-registry-parity.test.ts` updated from "four backends"
    → "five backends" with the same per-id assertions.

## Acknowledged debt (callouts in the handler header)

  - **No session persistence.** The Agents SDK has a `Session`
    abstraction but Talon's session model is per-chat numeric ids;
    wiring is a future PR.
  - **No progress-text emission** before tool calls. The Agents
    SDK's raw-stream events don't cleanly split into pre-tool
    text segments the way Claude SDK's do.
  - **No flow-violation reminder retry** — single-pass for now.
    The base retry ladder (session_expired / context_length /
    fallback_model) does run via the per-backend catch block;
    this PR doesn't depend on PR #196's `applyRetryDecision`
    extraction to stay independently mergeable.

## Verified

  - `npx tsc --noEmit` clean
  - `npm test` — 2423 passing, 12 skipped (live), 0 failing
    (was 2396 — +27 new)
  - `npm run lint` — 11 warnings, 0 errors (all pre-existing in
    `src/frontend/discord/`)
  - `npm run format:check` clean
  - Live SDK probe: `@openai/agents` constructs + calls `run()`
    cleanly (verified without API key by mocking; full live test
    deferred to a follow-up Backend Live job).

## Out of scope (future)

  - A `Backend Live (openai-agents, *)` CI job in the matrix.
    Needs an `OPENAI_API_KEY` secret in repo settings; the SDK
    won't smoke-test without one (unlike codex which has a
    ChatGPT-OAuth fallback path).
  - Session persistence wiring via the SDK's `Session` abstraction.
  - Streaming `onStreamDelta` callbacks for typing-indicator UX.

* fix: add missing @emnapi/core + @emnapi/runtime to lockfile for CI

These are optional dev transitive dependencies (via @openai/codex) that
exist in main's lockfile but were dropped when the local npm install
generated the lockfile on an arm64 platform that doesn't need them.
CI (ubuntu-latest x64) requires them to be present via npm ci.

Restores the entries from main's lockfile — no behavior change.

* feat(openai-agents): generic OpenAI-compatible endpoint support

Dylan flagged that the original v0 of this backend hardcoded to OpenAI's
production API, and a follow-up PR (#202) tried to add OpenRouter as a
special-cased redirect. The @openai/agents SDK already supports arbitrary
custom OpenAI-compatible endpoints via setDefaultOpenAIClient() — so the
right design is generic baseURL config on this backend, not an OpenRouter-
named adapter. PR #202 is closed; folding the support in here properly.

## What this adds

Two new optional config fields (plus matching env vars), both generic:

  - `openaiBaseUrl` / `OPENAI_BASE_URL`
      Redirects the SDK at any OpenAI-compatible service. Works for
      OpenRouter, Azure OpenAI, local Ollama, LiteLLM, Portkey, vLLM,
      Together, Groq's OpenAI-compat surface — anything that speaks the
      protocol.
  - `openaiApiMode` / `OPENAI_API_MODE`
      Which OpenAI API surface to target: "responses" (OpenAI native) or
      "chat_completions" (most third parties). Defaults to "responses"
      with no baseURL, and "chat_completions" automatically when a
      custom baseURL is set — since most non-OpenAI endpoints don't
      implement the Responses API.

`init.ts` constructs a custom `OpenAI({apiKey, baseURL})` client and
calls `setDefaultOpenAIClient()` whenever either field is configured.
`setOpenAIAPI()` is called unconditionally with the resolved mode so the
startup log line accurately reflects which surface subsequent `run()`
calls hit.

## Model catalog passthrough

`resolveModel()` and `getModelInfo()` previously rejected anything not in
the hardcoded `gpt-5.5 / gpt-5 / gpt-5-mini / o4-mini` catalog. Now: when
`openaiBaseUrl` is set, any unknown model id is accepted as a synthetic
passthrough entry — so users can target `meta-llama/llama-3.3-70b-instruct`
on OpenRouter, `llama3.2:3b` on Ollama, etc. without us maintaining a
separate catalog per third-party provider. Exact catalog hits still win
for OpenAI-native models even with a baseURL set.

## CLI wizard + doctor

- `openai-agents` branch of setup wizard prompts for API key, then base
  URL, then (if a base URL is set) Chat Completions vs Responses.
- `talon status` now shows configured `openaiBaseUrl` + `openaiApiMode`.
- `talon doctor` adds an `openai-agents` branch reporting auth state and
  endpoint config (defaults to api.openai.com when none).

## Tests

8 new cases in openai-agents-models.test.ts covering the passthrough
path: env-vs-config priority, custom-baseURL resolution, catalog hits
still preferred, getModelInfo synthetic entries, formatModelError
pointing users at direct config. 35 / 35 openai-agents tests pass; full
suite at 2449 / 2463 (2 pre-existing opencode integration-test
timeouts unrelated to this change).

Verified:
  - npx tsc --noEmit clean
  - npm run lint 0 errors, 11 pre-existing warnings
  - npx prettier --check clean
  - vitest 35/35 backend tests pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dylanneve1 dylanneve1 Awaiting requested review from dylanneve1 dylanneve1 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@claudiusthebot