fix: map anthropic context window exceeded stop reason

[steebchen]

Problem

Production gateway pods log spurious errors like:

{"finishReason":"model_context_window_exceeded","msg":"Unknown finish reason encountered","provider":"anthropic","model":"anthropic/claude-sonnet-4-5",...}

Anthropic models emit the stop reason model_context_window_exceeded when generation hits the model's context window before reaching max_tokens. The gateway didn't recognize it, so:

• getUnifiedFinishReason classified it as UNKNOWN and logged an ERROR-level "Unknown finish reason encountered" alert on every occurrence.
• mapFinishReasonToOpenai silently converted it to "stop", so OpenAI-compatible clients saw a normal completion instead of "length".
• isLengthLimitFinishReason didn't treat these responses as length-limited.

Fix

Map model_context_window_exceeded as a length limit uniformly. Like the existing cross-provider refusal handling, this stop reason surfaces across the Anthropic direct API, Vertex, and Bedrock, so it's handled at the top level of both mappers:

• apps/gateway/src/lib/logs.ts — getUnifiedFinishReason returns LENGTH_LIMIT (also fixes isLengthLimitFinishReason, which derives from it), eliminating the spurious error log.
• apps/gateway/src/chat/tools/map-finish-reason-to-openai.ts — clients now receive the OpenAI-canonical "length" finish reason (streaming and non-streaming).
• apps/gateway/src/chat/tools/parse-provider-response.ts and transform-streaming-to-openai.ts — the Bedrock Converse branches map the raw stop reason to "length" alongside max_tokens.

Testing

• Added unit tests covering the new mapping for anthropic, vertex-anthropic, and aws-bedrock in logs.spec.ts (65 tests passing in affected files).
• turbo run build --filter=gateway passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LxoFcM6Btfm7x2MR56wDXs

---

Generated by Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of model context-window limits across supported providers.
  • Context-limit responses are now consistently reported as length-limit completions.
  • Streaming and non-streaming responses now use the correct OpenAI-compatible finish reason.
• Tests
  • Added coverage for context-window limit handling across Anthropic, Vertex, and AWS Bedrock responses.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3925f037-7ad3-42f7-bad9-aa560b871f58

📥 Commits

Reviewing files that changed from the base of the PR and between 27c8e7a and e475a3e.

📒 Files selected for processing (5)

• apps/gateway/src/chat/tools/map-finish-reason-to-openai.ts
• apps/gateway/src/chat/tools/parse-provider-response.ts
• apps/gateway/src/chat/tools/transform-streaming-to-openai.ts
• apps/gateway/src/lib/logs.spec.ts
• apps/gateway/src/lib/logs.ts

---

Walkthrough

The gateway now classifies model_context_window_exceeded as a length-limit finish reason and maps it to OpenAI-compatible "length" responses across Anthropic-family and AWS Bedrock paths, with expanded logging tests.

Changes

Finish Reason Normalization

Layer / File(s)	Summary
Unified length-limit classification apps/gateway/src/lib/logs.ts, apps/gateway/src/lib/logs.spec.ts	model_context_window_exceeded maps to UnifiedFinishReason.LENGTH_LIMIT for Anthropic, Vertex Anthropic, and AWS Bedrock cases, with coverage for unified and length-limit detection.
Provider response length mapping apps/gateway/src/chat/tools/map-finish-reason-to-openai.ts, apps/gateway/src/chat/tools/parse-provider-response.ts, apps/gateway/src/chat/tools/transform-streaming-to-openai.ts	Direct and streaming provider response handling maps model_context_window_exceeded to the OpenAI-compatible "length" finish reason, including AWS Bedrock stop events.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• theopenco/llmgateway#2875: Both changes update provider-specific finish-reason normalization in the streaming transform.

Suggested reviewers: copilot, analogpvt

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: mapping Anthropic context-window-exceeded stop reasons.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/fix-and-pr-9c935g

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}