Skip to content

fix(ai-gateway): always rewrite proxied model responses - #4851

Merged
chrarnoldus merged 1 commit into
mainfrom
fix/ai-gateway-always-rewrite-response
Jul 29, 2026
Merged

fix(ai-gateway): always rewrite proxied model responses#4851
chrarnoldus merged 1 commit into
mainfrom
fix/ai-gateway-always-rewrite-response

Conversation

@chrarnoldus

Copy link
Copy Markdown
Contributor

Summary

Removes the passthrough shortcut in rewriteModelResponse so every proxied model response goes through the processing pipeline (previously only Kilo-exclusive free models and request-logged traffic did).

Why

Stream errors (upstream timeouts/disconnects) are common, and for the traffic that skipped the rewrite — paid models without request logging — they were undebuggable: the raw stream was passed through untouched, so mid-stream failures surfaced only as connection errors with no captured context. Processing the full stream is the only way to surface and log them.

Performance is not a concern here: token generation is inherently slow, and ~50% of traffic (free models + internal traffic with logging enabled) already goes through the rewrite.

Changes

  • rewriteModelResponse now always returns a NextResponse (never null); the route returns its result directly instead of falling back to wrapInSafeNextResponse.
  • The unrecognized-API-kind branch now throws (fail loud on an implementation error) instead of silently passing the response through.
  • Tests updated for the new contract: the route mock passes responses through via wrapInSafeNextResponse, and the removed skip-behavior test now asserts paid-model traffic is rewritten with cost preserved.

Verification

  • rewriteModelResponse.test.ts: 58/58 pass
  • openrouter/[...path]/route.test.ts: 20/20 pass
  • tsgo --noEmit (web): clean
  • oxlint on the 4 changed files: 0 warnings, 0 errors
  • pnpm format:changed applied; git diff --check clean

Remove the passthrough shortcut in rewriteModelResponse so every
response is processed: stream errors are common and were undebuggable
for the traffic that skipped the rewrite (paid models without request
logging), and processing the full stream is the only way to surface
them. rewriteModelResponse now always returns a NextResponse and
throws on an unrecognized API kind instead of returning null.
@chrarnoldus chrarnoldus self-assigned this Jul 29, 2026
@kilo-code-bot

kilo-code-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Reviewed the removal of the passthrough shortcut in rewriteModelResponse and the corresponding route/test updates; the change is internally consistent, the single caller was updated correctly, and no security, correctness, or reliability issues were found in the changed lines.

Files Reviewed (4 files)
  • apps/web/src/app/api/openrouter/[...path]/route.ts
  • apps/web/src/app/api/openrouter/[...path]/route.test.ts
  • apps/web/src/lib/rewriteModelResponse.ts
  • apps/web/src/lib/rewriteModelResponse.test.ts

Reviewed by claude-sonnet-5 · Input: 14 · Output: 2.2K · Cached: 249.8K

Review guidance: REVIEW.md from base branch main

@chrarnoldus
chrarnoldus merged commit 2449380 into main Jul 29, 2026
16 checks passed
@chrarnoldus
chrarnoldus deleted the fix/ai-gateway-always-rewrite-response branch July 29, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants