fix(ai-gateway): fit generation routes within Vercel timeout

[chrarnoldus]

Summary

• cap the OpenRouter and Gateway chat/messages/responses routes at the non-extended 800s Vercel limit
• cap all four OpenRouter and Gateway transcription route aliases at the same 800s limit
• reduce the upstream generation and transcription request timeout from 900s to 600s, leaving 200s for post-stream work
• configure fetchGeneration with a 75s backoff budget, which yields roughly one minute of polling with the helper's next-delay reservation
• update the focused generation polling expectation, transcription route-duration expectation, and timeout comments

Slow-path concerns

• maxDuration covers the full invocation, including request processing and after() work. The 600s upstream timeout leaves 200s for usage persistence, metrics, generation polling, and other post-response work.
• the generation polling window is approximate rather than a hard wall-clock timeout because individual fetch attempts have no abort timeout.
• the shorter fetchGeneration budget is shared by transcription usage processing as requested; this reduces the five-minute polling window introduced in fix(ai-gateway): extend generation polling timeout #4818 and may miss unusually delayed OpenRouter generation records.
• post-response usage, metrics, auto-top-up, experiment attribution, request logging, and notification operations share the invocation budget, and some downstream database/network calls have no hard timeout.
• unrelated tRPC, chat webhook, and Discord route exports remain configured at 1,800s and may need separate changes for a strict non-extended 800s environment.

Verification

• formatted the changed files with oxfmt
• git diff --check
• tests and automated review left to CI

[kilo-code-bot]

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The new commit correctly caps all four transcription route aliases at 800s (their upstream fetch already aborts at 600s, so no regression), but the shared 75s fetchGeneration budget still narrows the transcription/generation polling window introduced in #4818, which can leave usage rows without upstream cost.

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
apps/web/src/lib/ai-gateway/providers/upstream-request.ts	29	GENERATION_FETCH_MAX_DELAY_MS (75s, ~66s effective) is shared with transcription usage processing, replacing the 5-minute polling window from #4818; transcription cost normally comes only from the OpenRouter generation record, so giving up early persists cost_mUsd of 0. The transcription routes now cap at 800s, so the earlier maxDuration = 1800 framing in the inline comment no longer applies, but the shortened window itself is unchanged at HEAD.

Files Reviewed (5 files, incremental vs 4d7367a)

• apps/web/src/app/api/openrouter/audio/transcriptions/route.ts - 0 issues
• apps/web/src/app/api/openrouter/v1/audio/transcriptions/route.ts - 0 issues
• apps/web/src/app/api/gateway/audio/transcriptions/route.ts - 0 issues
• apps/web/src/app/api/gateway/v1/audio/transcriptions/route.ts - 0 issues
• apps/web/src/app/api/openrouter/audio/transcriptions/route.test.ts - 0 issues

Resolved Since Previous Review

• All four transcription aliases and their route-duration test now use 800s, matching the chat/messages/responses routes; the pre-existing 600s upstream abort in transcriptionProxyRequest keeps 200s of post-response headroom, and no transcription path relied on the removed 1800s ceiling.

Assumptions

• The 800s non-extended Vercel ceiling comes from the PR description; apps/web/vercel.json has no functions.maxDuration entry, so it could not be verified from the repository.
• Other 1800s route exports (tRPC, chat webhook, Discord) are intentionally out of scope per the PR description.

Fix these issues in Kilo Cloud

Previous Review Summaries (2 snapshots, latest commit 4d7367a)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 4d7367a)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The new commit fixes the unreachable upstream timeout (600s abort now fits inside the 800s budget), but the shared 75s generation-polling budget still narrows the transcription routes' 5-minute window from #4818, risking under-billed usage rows.

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
apps/web/src/lib/ai-gateway/providers/upstream-request.ts	29	GENERATION_FETCH_MAX_DELAY_MS is shared with the audio transcription routes (still maxDuration = 1800), so their 5-minute polling window from #4818 drops to ~66s; transcription cost usually comes only from the generation record, so giving up early keeps the response's cost_mUsd (often 0)

Resolved Since Previous Review

• TIMEOUT_MS is now 600s with maxDuration = 800, so the graceful 503 upstream_disconnect path, timeout logging, and after() work are reachable again, and the stale 30-minute comment was replaced.
• The polling-window comment and test now reflect fetchWithBackoff's next-delay reservation (75s budget yields ~66s / 6 attempts).

Files Reviewed (4 files)

• apps/web/src/app/api/gateway/[...path]/route.ts - 0 issues
• apps/web/src/app/api/openrouter/[...path]/route.ts - 0 issues
• apps/web/src/lib/ai-gateway/providers/upstream-request.ts - 1 issue
• apps/web/src/lib/ai-gateway/providers/upstream-request.generation.test.ts - 0 issues

Assumptions

• The 800s ceiling was taken from the PR description; apps/web/vercel.json has no functions.maxDuration config, so the plan's non-extended limit could not be verified from the repository.
• Reserving 200s of headroom measures from the start of upstreamRequest, not from invocation start, so pre-fetch work (auth, rules engine, model resolution) reduces the actual post-stream margin slightly.

Fix these issues in Kilo Cloud

Previous review (commit 0bb550d)

Status: 2 Issues Found | Recommendation: Address before merge

Executive Summary

Lowering maxDuration to 600s leaves the AI gateway's 900s upstream abort unreachable, so slow-provider requests now die as platform terminations instead of structured 503s.

Overview

Severity	Count
CRITICAL	0
WARNING	2
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
apps/web/src/app/api/openrouter/[...path]/route.ts	110	TIMEOUT_MS (900s) in upstream-request.ts now exceeds the 600s function budget, so the graceful 503 upstream_disconnect path, timeout logging, and after() post-stream work are cut off by the platform instead; the upstream-request.ts:26 comment is stale
apps/web/src/lib/ai-gateway/providers/upstream-request.ts	28	Shared GENERATION_FETCH_MAX_DELAY_MS also shrinks generation polling for the transcription routes (still maxDuration = 1800), partially reverting #4818; effective window is ~41s/5 attempts, not 60s, due to fetchWithBackoff's maxDelayMs - nextDelay exit condition

Files Reviewed (4 files)

• apps/web/src/app/api/gateway/[...path]/route.ts - 0 issues
• apps/web/src/app/api/openrouter/[...path]/route.ts - 1 issue
• apps/web/src/lib/ai-gateway/providers/upstream-request.ts - 1 issue
• apps/web/src/lib/ai-gateway/providers/upstream-request.generation.test.ts - 0 issues

Fix these issues in Kilo Cloud

---

_{Reviewed by claude-opus-5 · Input: 32 · Output: 7.2K · Cached: 738.7K}

_{Review guidance: REVIEW.md from base branch main}