fix(responses): count input tokens locally for Codex OAuth

[anhtran-ai]

Summary

Add a static POST /v1/responses/input_tokens route that counts locally and returns the standard OpenAI Responses contract without issuing an upstream request.

This complements #13161 / #13157. #13161 makes a Cloudflare managed challenge non-terminal; this PR removes the request that triggers that challenge in the first place.

Problem

The generic /v1/responses/[...path] handler currently forwards input_tokens as a native Responses suffix. For a Codex OAuth / ChatGPT-subscription connection that produces:

POST chatgpt.com/backend-api/codex/responses/input_tokens

Measured behavior for the same live account and token:

• native/undici transport: Cloudflare managed challenge, HTTP 403, cf-mitigated: challenge;
• bundled Chrome 124 wreq-js transport: HTTP 404 {"detail":"Not Found"};
• normal /backend-api/codex/responses inference: succeeds.

The subscription backend does not serve this OpenAI API subresource. Forwarding the preflight can therefore never produce a useful token count. It adds latency and fallback on every call and, before #13161, the managed-challenge 403 was persisted as terminal banned / isActive:false, taking the healthy Codex provider offline.

Fix

A static App Router segment shadows the catch-all:

src/app/api/v1/responses/input_tokens/route.ts

It:

• returns the exact public contract { "object": "response.input_tokens", "input_tokens": <int> };
• uses the existing offline countTextTokens() path (o200k_base for Codex / cx models);
• counts instructions, string/structured input, tool call/output payloads, tool definitions, tool_choice, text and reasoning configuration;
• keeps image/file parts non-textual instead of tokenizing base64 payloads;
• preserves API-key/model policy and process-wide body admission from the Responses route it shadows;
• never calls fetch, so the Cloudflare challenge cannot occur on this path.

The estimate is intentionally conservative. Live A/B showed a stable 9–10 token Responses envelope overhead for no-tool requests, while tool definitions already carry their own framing. The route adds that fixed overhead for no-tool requests and keeps a 5% margin for tool requests. Over-counting means slightly earlier client compaction; under-counting risks a context-window overflow.

Verification

Local

• node --import tsx/esm --test tests/unit/responses-input-tokens-local-route.test.ts: 9/9 pass.
• npm run typecheck:core: clean.
• ESLint on both changed files: clean.

Tests pin the exact two-field response schema, prove fetch is never called, and cover instructions, structured input, tools, non-text parts, server-held context IDs, invalid JSON, OPTIONS and the conservative lower bound.

Production A/B

Deployed to the affected gateway as llm-gateway/omniroute:3.8.51-113ddf9a-azox1 and compared the local result to the subsequent real Codex usage.input_tokens for eight request shapes:

Case	Local	Upstream usage	Difference
short	16	16	0%
medium	89	89	0%
long	817	817	0%
structured messages	46	41	+12.2%
one tool	66	65	+1.5%
two tools	98	75	+30.7%
Vietnamese/Unicode	33	33	0%
code	534	534	0%

Results: 8/8 valid, 0 under-counts, median absolute error 0%. Both over-counts are in the safe direction (earlier compaction).

Additional production evidence:

• 50/50 count soak HTTP 200.
• Isolated 100/100 count soak HTTP 200.
• The isolated 100-count soak changed proxy_logs by exactly 0 rows.
• Cloudflare challenge events: 0.
• Permanent-disable events: 0.
• The Codex connection remained is_active=1, test_status=active with clear error fields.
• All eight paired inference requests returned HTTP 200; a separate chat soak returned 6/6 HTTP 200.

[diegosouzapw]

Great root-cause fix — shadowing the catch-all instead of trying to patch around the Cloudflare
challenge is the right call, since the upstream subresource can never actually succeed for OAuth
accounts. I ran your test file at the PR head: 9/9 pass, including the "never calls fetch"
assertion which is really the core guarantee here. One ask before merge: could you add a
changelog.d/fixes/ fragment? I didn't find one in the diff.

[diegosouzapw]

Thanks @anhtran-ai — merging via the release merge-train. Validated in local merge-train (merge-train-20260918-120911-suite.log) on the devbox @ train tip 8c305709478b7052fb981a75852bbf88fe3a959d with the sibling PRs of this batch: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; changed-area node:test 310/311 (the single red, hard-lease inventory, reproduces on the pure release tip) + vitest green (fast parity mode — full suite ran today on the tip via the base-red and 3b trains). Merged --admin per merge-gates §4/§7.