diff --git a/docs/features/codex-proxy-support.md b/docs/features/codex-proxy-support.md index 6cee25ce3..63f605988 100644 --- a/docs/features/codex-proxy-support.md +++ b/docs/features/codex-proxy-support.md @@ -155,3 +155,36 @@ The Anthropic engine in `claudeProxyRoutes.ts` is untouched. The Codex engine is - **Quota-aware ordering is unverified against the live backend.** The usage endpoint and the rate-limit header shape were reconstructed from a capture, not confirmed end to end. If either is wrong, `fetchCodexAccountUsage` returns no quota, `x-neurolink-quota-source` reads `none`, and ordering degenerates to insertion order while every 429 falls back to the 15-minute transient cooldown. Rotation still works; it is simply not quota-aware. Verify with `neurolink auth list --refresh` — a `codex usage …` error line per account means the quota path is not live. - **SSE usage-limit signals are not acted on.** Only an HTTP 429 triggers a cooldown and rotation. A `200` response whose SSE stream carries `usage_limit_reached` (or the workspace-credit variants) is relayed to the client untouched, so the account is neither cooled nor rotated away from. HTTP-level exhaustion is handled; in-stream exhaustion is not. - **Client fingerprint is forwarded verbatim.** The proxy replaces the caller's credentials but does not regenerate `originator`, `installation_id`, `session-id`, or turn metadata per account, so every pooled account shares the client's fingerprint. This is what makes the terms-of-service point above concrete. + +## Model discovery + +The Codex CLI refreshes its model list on every invocation: + +``` +GET /backend-api/codex/models?client_version= +``` + +The proxy relays that upstream to `chatgpt.com/backend-api/codex/models` using a +pooled account, forwarding the CLI's own query parameters. + +**It relays rather than synthesises**, unlike the Claude and OpenAI `/v1/models` +routes, which build their lists locally from the model router. Which Codex +models an account can reach is a property of that account — plan tier, rollout +state — not something this proxy knows, so a locally-built list would be a guess +that reads as authoritative. + +Two details matter to anyone touching it: + +- **`client_version` is required upstream.** Omit it and ChatGPT answers `400` + with a pydantic `Field required` on `('query', 'client_version')`. The query + is rebuilt from `ctx.query`; `ctx.path` carries no query string, and reading it + from there drops the parameter silently. +- **Discovery is side-effect free.** No cooldown is recorded and no quota is + consumed, so the once-per-invocation refresh cannot influence routing for real + traffic. A cooling account is still allowed to answer it — being rate-limited + for completions does not make an account unable to say which models exist. + +Before this route existed the request 404'd, and the CLI printed +`failed to refresh available models: unexpected status 404 Not Found` on every +run before silently falling back to a default model — quietly ignoring the model +the user had configured. diff --git a/src/lib/auth/codexOAuth.ts b/src/lib/auth/codexOAuth.ts index 8aca5640b..2f8e78e89 100644 --- a/src/lib/auth/codexOAuth.ts +++ b/src/lib/auth/codexOAuth.ts @@ -37,6 +37,7 @@ export const CODEX_DEFAULT_SCOPES = [ export const CODEX_BACKEND_BASE_URL = "https://chatgpt.com/backend-api/codex"; export const CODEX_RESPONSES_URL = `${CODEX_BACKEND_BASE_URL}/responses`; export const CODEX_USAGE_URL = `${CODEX_BACKEND_BASE_URL}/usage`; +export const CODEX_MODELS_URL = `${CODEX_BACKEND_BASE_URL}/models`; // Client fingerprint. The real Codex CLI sends these; the proxy defaults them // only when a non-Codex client omits them. diff --git a/src/lib/server/routes/codexProxyRoutes.ts b/src/lib/server/routes/codexProxyRoutes.ts index d51aa92a6..357dbdab8 100644 --- a/src/lib/server/routes/codexProxyRoutes.ts +++ b/src/lib/server/routes/codexProxyRoutes.ts @@ -21,6 +21,7 @@ import { tokenStore } from "../../auth/tokenStore.js"; import { CODEX_ORIGINATOR, + CODEX_MODELS_URL, CODEX_RESPONSES_URL, CODEX_USER_AGENT, codexTokenNeedsRefresh, @@ -601,6 +602,139 @@ async function handleCodexResponsesRequest( return buildCodexErrorResponse(lastErrorStatus, lastErrorMessage); } +/** + * Relay Codex model discovery upstream. + * + * The CLI refreshes its model list on every invocation. Only `/responses` was + * registered, so that GET 404'd and the CLI printed a refresh failure before + * falling back to a default model — quietly ignoring the model the user had + * configured. + * + * This relays rather than synthesises, unlike the Claude and OpenAI `/v1/models` + * routes, which build their lists locally from the model router. Codex model + * availability is a property of the upstream account (plan tier, rollout), not + * of anything this proxy knows, so a synthesised list would be a guess that + * looks authoritative. + * + * Read-only with respect to ROUTING: no cooldown is recorded and no quota is + * consumed, so a discovery call cannot influence which account real traffic + * lands on. It is not literally side-effect free — a token refreshed below is + * persisted, exactly as the proactive refresh in `loadCodexProxyAccounts` + * persists one. What discovery deliberately never does is *penalise* an + * account: it cannot cool one and cannot disable one. A read-only probe must + * not be able to cost the user a login. + */ +async function handleCodexModelsRequest(ctx: ServerContext): Promise { + const accounts = await loadCodexProxyAccounts(); + if (accounts.length === 0) { + return buildCodexErrorResponse( + 401, + "No Codex accounts configured. Run `neurolink auth login codex`.", + ); + } + + const now = Date.now(); + const ordered = orderCodexAccounts(accounts, now); + // A cooling account is rate-limited for completions, not barred from + // answering what models exist. Healthy accounts go first, but a cooling one + // is still a candidate rather than a reason to fail discovery outright. + const isCooling = (a: (typeof ordered)[number]): boolean => + a.coolingUntil !== undefined && a.coolingUntil > now; + const candidates = [ + ...ordered.filter((a) => !isCooling(a)), + ...ordered.filter(isCooling), + ]; + + // Forward the CLI's own query — it sends client_version, and upstream + // *requires* it: without it ChatGPT answers 400 with a pydantic + // "Field required" on ('query','client_version'). Rebuild from ctx.query, + // not ctx.path: path carries no query string, so reading it there silently + // dropped the parameter and produced exactly that 400. + const params = new URLSearchParams(ctx.query ?? {}); + const query = params.toString(); + const url = query ? `${CODEX_MODELS_URL}?${query}` : CODEX_MODELS_URL; + + let lastErrorStatus = 502; + let lastErrorMessage = "Codex model discovery upstream failed"; + + for (const account of candidates) { + // One forced refresh per account, then move on. A token can be rejected + // upstream while still inside its local expiry window, so relaying that + // 401 straight back left discovery broken until the token expired locally + // — the CLI would fall back to a default model on every invocation in the + // meantime. + let authRetried = false; + for (;;) { + let upstream: Response; + try { + upstream = await fetch(url, { + method: "GET", + headers: buildCodexUpstreamHeaders(ctx.headers ?? {}, account), + // Bound the upstream call, as the responses route does. Without a + // signal a stalled connection holds the proxy request open with no + // ceiling, and the CLI blocks on model discovery at startup. + signal: AbortSignal.timeout(CODEX_UPSTREAM_TIMEOUT_MS), + }); + } catch (error) { + lastErrorStatus = 502; + lastErrorMessage = `Codex model discovery upstream failed: ${ + error instanceof Error ? error.message : String(error) + }`; + break; // rotate to the next account + } + + if (upstream.status === 401 || upstream.status === 403) { + if (!authRetried && account.refreshToken) { + authRetried = true; + try { + const refreshed = await refreshCodexToken(account.refreshToken); + account.token = refreshed.accessToken; + account.refreshToken = + refreshed.refreshToken ?? account.refreshToken; + account.expiresAt = refreshed.expiresAt ?? account.expiresAt; + account.accountId = resolveCodexAccountId(refreshed.accessToken); + await tokenStore.saveTokens(account.key, { + accessToken: account.token, + refreshToken: account.refreshToken, + expiresAt: account.expiresAt ?? Date.now() + 3_600_000, + tokenType: "Bearer", + }); + continue; // retry this account with the fresh token + } catch { + // No cooldown and no disable, unlike the responses path: the + // verdict a completion draws from a failed refresh is earned by a + // request the user actually made. Discovery fires on every CLI + // invocation, so letting it disable an account would turn a + // background probe into a forced re-login. + lastErrorStatus = 401; + lastErrorMessage = "Codex token refresh failed; re-login required"; + break; // rotate + } + } + lastErrorStatus = upstream.status; + lastErrorMessage = "Codex model discovery rejected upstream"; + break; // rotate + } + + // Every other status — including a 400 — is upstream's real answer to a + // well-formed request and is relayed unchanged. A 400 here means the + // query was not forwarded correctly, and hiding it behind a retry would + // bury the exact regression this route was added to fix. + const body = await upstream.text(); + const contentType = + upstream.headers.get("content-type") ?? "application/json"; + return new Response(body, { + status: upstream.status, + headers: { "content-type": contentType }, + }); + } + } + + // A discovery failure must not look like a missing route, or the next + // person debugging it re-opens this same issue. + return buildCodexErrorResponse(lastErrorStatus, lastErrorMessage); +} + /** * Create Codex proxy routes. * @@ -617,6 +751,12 @@ export function createCodexProxyRoutes(basePath: string = ""): RouteGroup { description: "Codex ChatGPT-backend Responses API (account pool)", handler: (ctx: ServerContext) => handleCodexResponsesRequest(ctx), }, + { + method: "GET", + path: `${basePath}/backend-api/codex/models`, + description: "Codex model discovery, relayed upstream (account pool)", + handler: (ctx: ServerContext) => handleCodexModelsRequest(ctx), + }, ], }; } diff --git a/test/continuous-test-suite-proxy.ts b/test/continuous-test-suite-proxy.ts index 94e790690..3ba8dd0eb 100644 --- a/test/continuous-test-suite-proxy.ts +++ b/test/continuous-test-suite-proxy.ts @@ -592,6 +592,72 @@ async function testProxyModelsEndpoint(): Promise { } } +/** + * Codex CLI model discovery must not 404. + * + * The Codex CLI refreshes its model list on every invocation, hitting + * `GET /backend-api/codex/models?client_version=`. The proxy registered only + * the `/responses` route, so that request 404'd and the CLI printed + * `failed to refresh available models: unexpected status 404 Not Found` on each + * run before silently falling back to a default model — the user's configured + * model quietly ignored. + * + * Driven against the spawned proxy exactly as the CLI drives it, including the + * client_version query the CLI sends. Without Codex credentials the relay + * cannot reach upstream and answers 401/502/503; that is a correct answer from + * a route that exists. + * + * The tolerated set is an ALLOW-list, not a deny-list, and that distinction is + * the whole point. An earlier version accepted anything that was not 404 or + * 405 — which meant it also accepted 400, and 400 is precisely what upstream + * returns when `client_version` is dropped from the forwarded query. That is + * the bug this route was written to fix, so the test passed on the regression + * it existed to catch. Any status outside the allow-list now fails. + */ +async function testCodexModelsDiscovery(): Promise { + // 200 with credentials; 401 unauthenticated or rejected; 502/503 when + // upstream is unreachable or the refresh is transiently unavailable. + const ACCEPTED = new Set([200, 401, 502, 503]); + try { + const resp = await fetchProxy( + "/backend-api/codex/models?client_version=0.147.0", + ); + if (resp.status === 404) { + log( + "Codex model discovery is unroutable — the CLI cannot list models", + "red", + ); + return false; + } + if (resp.status === 405) { + log("Codex model discovery rejected the CLI's GET method", "red"); + return false; + } + if (resp.status === 400) { + log( + "Codex model discovery answered 400 — the client_version query is not reaching upstream", + "red", + ); + return false; + } + if (!ACCEPTED.has(resp.status)) { + log( + `Codex model discovery answered an unexpected status ${resp.status}`, + "red", + ); + return false; + } + log(`Codex /models answered with status ${resp.status}`, "green"); + return true; + } catch (err) { + log( + `Codex models endpoint error: ${err instanceof Error ? err.message : String(err)}`, + "red", + ); + return false; + } +} + async function testProxyCountTokens(): Promise { try { const resp = await fetchProxy("/v1/messages/count_tokens", { @@ -5429,6 +5495,11 @@ const tests: TestFunction[] = [ fn: testOpenCodeRestoreLeavesUserEditedBlock, category: "proxy-config", }, + { + name: "Codex: model discovery route answers the CLI", + fn: testCodexModelsDiscovery, + category: "proxy-config", + }, { name: "Proxy clients: uninstall restores every client config", fn: testUninstallRestoresClientConfigs,