Skip to content

fix: preserve max_output_tokens for Responses API targets in chatCore sanitization - #1313

Merged
diegosouzapw merged 2 commits into
cheaper-inference:release/v3.6.7from
Gi99lin:fix/chatcore-max-output-tokens-responses-passthrough
Apr 16, 2026
Merged

diegosouzapw merged 2 commits into
cheaper-inference:release/v3.6.7from
Gi99lin:fix/chatcore-max-output-tokens-responses-passthrough

Conversation

@Gi99lin

@Gi99lin Gi99lin commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Problem

The common input sanitization in chatCore.ts unconditionally normalizes max_output_tokens to max_tokens (#994). While this works for Chat Completions targets, it breaks OpenAI Responses API passthrough when sourceFormat === targetFormat === "openai-responses".

Reproduction

  1. Configure a client (e.g. Codex CLI, Claude Code) to use openai-responses API format
  2. Send a request with max_output_tokens to an OmniRoute instance that routes to a Responses API provider (e.g. Codex)
  3. The upstream rejects the request with: 400 Unsupported parameter: max_tokens

Root Cause

The flow is:

Client sends: { max_output_tokens: 4096, input: [...] }
  → chatCore.ts:914: max_output_tokens → max_tokens (normalization)
  → translateRequest(): sourceFormat === targetFormat → SKIP
  → Upstream receives: { max_tokens: 4096 } → ❌ "Unsupported parameter"

When source and target formats are both openai-responses, translateRequest() skips translation entirely (line 122-123 of translator/index.ts). The translator from PR #1245 — which correctly maps max_tokens back to max_output_tokens — is never invoked.

Fix

Make the normalization conditional: skip it when targetFormat is openai-responses, where max_output_tokens is the canonical field.

- if (body.max_output_tokens !== undefined) {
+ if (body.max_output_tokens !== undefined && targetFormat !== FORMATS.OPENAI_RESPONSES) {

This preserves the existing behavior for all non-Responses paths (Chat Completions, Claude, Gemini etc.) while fixing the Responses→Responses passthrough.

Why this is safe

  • Chat Completions targets: Normalization still runs → max_tokens is used ✅
  • openai → openai-responses path: The translator from PR fix: translate max_tokens/max_completion_tokens → max_output_tokens in Chat→Responses translator #1245 handles max_tokens → max_output_tokens conversion ✅
  • openai-responses → openai-responses path (FIXED): max_output_tokens is preserved → upstream receives the correct field ✅
  • Codex executor: Already deletes both max_tokens and max_output_tokens in transformRequest (lines 512-513), so no behavioral change ✅

Test

Added a test that verifies max_output_tokens is not normalized to max_tokens when routing to a Responses API target via the /v1/responses endpoint.

Related

… sanitization

The common input sanitization in chatCore unconditionally normalizes
max_output_tokens to max_tokens (cheaper-inference#994). This works for Chat Completions
targets but breaks Responses API passthrough (source and target both
openai-responses), because:

1. chatCore replaces max_output_tokens with max_tokens
2. Same-format requests skip the translator entirely
3. max_tokens reaches the upstream, which rejects it with
   'Unsupported parameter: max_tokens'

The fix makes the normalization conditional: skip it when the target
format is openai-responses, where max_output_tokens is the canonical
field. The existing openaiToOpenAIResponsesRequest translator (cheaper-inference#1245)
still handles the openai → openai-responses path correctly.

Adds a test that verifies max_output_tokens is not normalized to
max_tokens when routing to a Responses API target.
@Gi99lin
Gi99lin requested a review from diegosouzapw as a code owner April 16, 2026 10:27

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the input sanitization in chatCore.ts to skip the normalization of max_output_tokens to max_tokens when the target format is the OpenAI Responses API, preventing errors in passthrough requests. A unit test was added to verify this change. The review feedback suggests further improving the sanitization by also handling the reverse normalization—converting max_tokens or max_completion_tokens to max_output_tokens—for Responses API targets to ensure universal compatibility.

Comment thread open-sse/handlers/chatCore.ts Outdated
Comment on lines +913 to +918
// #994: Normalize max_output_tokens to max_tokens for universal compatibility.
// Skip for Responses API targets where max_output_tokens is the canonical field;
// normalizing it to max_tokens there causes "Unsupported parameter: max_tokens"
// errors because Responses→Responses passthrough skips the translator that would
// convert it back (see openaiToOpenAIResponsesRequest).
if (body.max_output_tokens !== undefined && targetFormat !== FORMATS.OPENAI_RESPONSES) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While preserving max_output_tokens for Responses API targets is correct, we should also handle the reverse normalization for passthrough requests. If a client sends max_tokens or max_completion_tokens to a Responses API endpoint, it will fail upstream because the translator is skipped for same-format routing. Adding this normalization here ensures "universal compatibility" for passthrough as well, which is the stated goal of this sanitization block.

Suggested change
// #994: Normalize max_output_tokens to max_tokens for universal compatibility.
// Skip for Responses API targets where max_output_tokens is the canonical field;
// normalizing it to max_tokens there causes "Unsupported parameter: max_tokens"
// errors because Responses→Responses passthrough skips the translator that would
// convert it back (see openaiToOpenAIResponsesRequest).
if (body.max_output_tokens !== undefined && targetFormat !== FORMATS.OPENAI_RESPONSES) {
// #994: Normalize between max_output_tokens and max_tokens for universal compatibility.
// For Responses API targets, max_output_tokens is the canonical field. For others,
// max_tokens is preferred. We handle normalization here to support passthrough
// paths where the translator is skipped.
if (targetFormat === FORMATS.OPENAI_RESPONSES) {
if (body.max_output_tokens === undefined) {
if (body.max_completion_tokens !== undefined) {
body.max_output_tokens = body.max_completion_tokens;
delete body.max_completion_tokens;
} else if (body.max_tokens !== undefined) {
body.max_output_tokens = body.max_tokens;
delete body.max_tokens;
}
}
} else if (body.max_output_tokens !== undefined) {

…_output_tokens for Responses targets

Per review feedback: when targeting openai-responses, also normalize
max_tokens and max_completion_tokens INTO max_output_tokens, not just
skip the outbound normalization. This covers the case where a Chat
Completions client sends max_tokens to a Responses endpoint via
passthrough.

Extends test to cover both reverse normalization paths.
@diegosouzapw
diegosouzapw changed the base branch from main to release/v3.6.7 April 16, 2026 11:18
@diegosouzapw
diegosouzapw merged commit ee024b3 into cheaper-inference:release/v3.6.7 Apr 16, 2026
2 checks passed
Poid-ZA pushed a commit to Poid-ZA/OmniRoute that referenced this pull request Aug 5, 2026
…ax-output-tokens-responses-passthrough

fix: preserve max_output_tokens for Responses API targets in chatCore sanitization
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ax-output-tokens-responses-passthrough

fix: preserve max_output_tokens for Responses API targets in chatCore sanitization
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants