Repository navigation
fix: preserve max_output_tokens for Responses API targets in chatCore sanitization - #1313
Conversation
… sanitization The common input sanitization in chatCore unconditionally normalizes max_output_tokens to max_tokens (cheaper-inference#994). This works for Chat Completions targets but breaks Responses API passthrough (source and target both openai-responses), because: 1. chatCore replaces max_output_tokens with max_tokens 2. Same-format requests skip the translator entirely 3. max_tokens reaches the upstream, which rejects it with 'Unsupported parameter: max_tokens' The fix makes the normalization conditional: skip it when the target format is openai-responses, where max_output_tokens is the canonical field. The existing openaiToOpenAIResponsesRequest translator (cheaper-inference#1245) still handles the openai → openai-responses path correctly. Adds a test that verifies max_output_tokens is not normalized to max_tokens when routing to a Responses API target.
There was a problem hiding this comment.
Code Review
This pull request modifies the input sanitization in chatCore.ts to skip the normalization of max_output_tokens to max_tokens when the target format is the OpenAI Responses API, preventing errors in passthrough requests. A unit test was added to verify this change. The review feedback suggests further improving the sanitization by also handling the reverse normalization—converting max_tokens or max_completion_tokens to max_output_tokens—for Responses API targets to ensure universal compatibility.
| // #994: Normalize max_output_tokens to max_tokens for universal compatibility. | ||
| // Skip for Responses API targets where max_output_tokens is the canonical field; | ||
| // normalizing it to max_tokens there causes "Unsupported parameter: max_tokens" | ||
| // errors because Responses→Responses passthrough skips the translator that would | ||
| // convert it back (see openaiToOpenAIResponsesRequest). | ||
| if (body.max_output_tokens !== undefined && targetFormat !== FORMATS.OPENAI_RESPONSES) { |
There was a problem hiding this comment.
While preserving max_output_tokens for Responses API targets is correct, we should also handle the reverse normalization for passthrough requests. If a client sends max_tokens or max_completion_tokens to a Responses API endpoint, it will fail upstream because the translator is skipped for same-format routing. Adding this normalization here ensures "universal compatibility" for passthrough as well, which is the stated goal of this sanitization block.
| // #994: Normalize max_output_tokens to max_tokens for universal compatibility. | |
| // Skip for Responses API targets where max_output_tokens is the canonical field; | |
| // normalizing it to max_tokens there causes "Unsupported parameter: max_tokens" | |
| // errors because Responses→Responses passthrough skips the translator that would | |
| // convert it back (see openaiToOpenAIResponsesRequest). | |
| if (body.max_output_tokens !== undefined && targetFormat !== FORMATS.OPENAI_RESPONSES) { | |
| // #994: Normalize between max_output_tokens and max_tokens for universal compatibility. | |
| // For Responses API targets, max_output_tokens is the canonical field. For others, | |
| // max_tokens is preferred. We handle normalization here to support passthrough | |
| // paths where the translator is skipped. | |
| if (targetFormat === FORMATS.OPENAI_RESPONSES) { | |
| if (body.max_output_tokens === undefined) { | |
| if (body.max_completion_tokens !== undefined) { | |
| body.max_output_tokens = body.max_completion_tokens; | |
| delete body.max_completion_tokens; | |
| } else if (body.max_tokens !== undefined) { | |
| body.max_output_tokens = body.max_tokens; | |
| delete body.max_tokens; | |
| } | |
| } | |
| } else if (body.max_output_tokens !== undefined) { |
…_output_tokens for Responses targets Per review feedback: when targeting openai-responses, also normalize max_tokens and max_completion_tokens INTO max_output_tokens, not just skip the outbound normalization. This covers the case where a Chat Completions client sends max_tokens to a Responses endpoint via passthrough. Extends test to cover both reverse normalization paths.
ee024b3
into
cheaper-inference:release/v3.6.7
…ax-output-tokens-responses-passthrough fix: preserve max_output_tokens for Responses API targets in chatCore sanitization
…ax-output-tokens-responses-passthrough fix: preserve max_output_tokens for Responses API targets in chatCore sanitization
Problem
The common input sanitization in
chatCore.tsunconditionally normalizesmax_output_tokenstomax_tokens(#994). While this works for Chat Completions targets, it breaks OpenAI Responses API passthrough whensourceFormat === targetFormat === "openai-responses".Reproduction
openai-responsesAPI formatmax_output_tokensto an OmniRoute instance that routes to a Responses API provider (e.g. Codex)400 Unsupported parameter: max_tokensRoot Cause
The flow is:
When source and target formats are both
openai-responses,translateRequest()skips translation entirely (line 122-123 oftranslator/index.ts). The translator from PR #1245 — which correctly mapsmax_tokensback tomax_output_tokens— is never invoked.Fix
Make the normalization conditional: skip it when
targetFormatisopenai-responses, wheremax_output_tokensis the canonical field.This preserves the existing behavior for all non-Responses paths (Chat Completions, Claude, Gemini etc.) while fixing the Responses→Responses passthrough.
Why this is safe
max_tokensis used ✅max_tokens → max_output_tokensconversion ✅max_output_tokensis preserved → upstream receives the correct field ✅max_tokensandmax_output_tokensintransformRequest(lines 512-513), so no behavioral change ✅Test
Added a test that verifies
max_output_tokensis not normalized tomax_tokenswhen routing to a Responses API target via the/v1/responsesendpoint.Related
max_tokens/max_completion_tokens→max_output_tokenstranslator fix (openai → openai-responses path)