Skip to content

fix(translator): preserve Responses Lite tools across Chat providers - #2983

Closed
nguyenha935 wants to merge 4 commits into
decolua:masterfrom
nguyenha935:fix/responses-nonstream-chat-projection
Closed

nguyenha935 wants to merge 4 commits into
decolua:masterfrom
nguyenha935:fix/responses-nonstream-chat-projection

Conversation

@nguyenha935

@nguyenha935 nguyenha935 commented Aug 1, 2026 •

Copy link
Copy Markdown
Contributor

Bug

A Responses-API client (notably Codex) routed to a chat-native OpenAI-compatible provider (openai-compatible-chat-*) lost tool use in three independent places:

  1. Non-streaming Chat responses leaked the raw chat.completion envelope instead of returning Responses output items, making upstream tool_calls invisible.
  2. The Responses→Chat translator's internal reasoning.encrypted_content continuity stash leaked into the outbound Chat body. Compatible upstreams reject that unknown assistant-message field and may return a literal "400" assistant response.
  3. Current Codex Responses Lite does not necessarily put tools in top-level tools. It sends a developer input item such as:
{
  "type": "additional_tools",
  "role": "developer",
  "tools": [{ "type": "custom", "name": "exec", "format": { "...": "..." } }]
}

The Responses→Chat request translator ignored additional_tools, custom_tool_call, and custom_tool_call_output. The outbound provider request therefore had tool_choice: "auto" but no tool declaration, and the model correctly reported that it had no terminal/SSH tool.

Root causes

  • translateNonStreamingResponse had no Chat-provider → Responses-client JSON projection.
  • Forced SSE→JSON chose its parser using the client format instead of the upstream response format.
  • Translator-only continuity fields were not stripped at the dispatch boundary.
  • Responses Lite input item types were recognized by the Codex executor but not by the Responses→Chat translator.
  • Chat responses have no native custom/freeform tool type, so request-side custom-tool identity has to survive as internal metadata for the return translation.

Fix

  • Project Chat JSON and forced Chat SSE into Responses output envelopes.
  • Strip encrypted_content and reasoning_encrypted_content immediately before upstream dispatch while retaining translator-level round-trip continuity.
  • Promote input[].type == "additional_tools" declarations into Chat tools.
  • Represent each Responses custom/freeform tool to Chat as a function with a required string input parameter.
  • Translate historical custom_tool_call and custom_tool_call_output items into Chat assistant tool_calls and tool messages.
  • Carry translator-only custom-tool names through streaming, true non-streaming, and forced-SSE response paths.
  • Map marked Chat tool calls back to Responses custom_tool_call items/events and unwrap the Chat { "input": "..." } argument wrapper into raw input.
  • Keep ordinary function tools as ordinary Responses function_call items.
  • Delay streaming response.output_item.added until both tool-call id and name are known, preventing split Chat deltas from being announced with the wrong Responses item type.

Additional fix: OpenAI-compatible apiType resolution

Editing an OpenAI-compatible node's API Type (chat ↔ responses) had no effect at runtime: the transport was derived from the immutable provider ID substring (openai-compatible-<chat|responses>-<uuid>) instead of the stored node config. The edit route updates node.apiType and propagates it into each connection's providerSpecificData without ever changing the ID, so a node edited after creation kept its original transport.

Fix — new resolveOpenAICompatibleApiType(provider, credentials) in open-sse/services/provider.js: the stored providerSpecificData.apiType is authoritative, falling back to the ID substring only for legacy nodes created before apiType was persisted. All three runtime call sites now use it, so format selection and endpoint path always agree:

  • open-sse/handlers/chatCore.js — target-format resolution via getTargetFormat(provider, credentials)
  • open-sse/executors/base.js — buildUrl endpoint path (/responses vs /chat/completions)
  • open-sse/executors/default.js — same for the default executor

Tests — tests/unit/openai-compatible-apitype-resolution.test.js (12 cases): stored-value precedence in both directions, ID fallback with absent/null credentials, invalid-value fallback, target-format selection, and endpoint path for both executors.

Verification

Automated

  • New tests/unit/openai-responses-custom-tools.test.js: 6 tests for additional_tools, merged tools, custom call/output history, streaming custom events, split id/name deltas, and normal-function regression.
  • Expanded tests/unit/openai-responses-nonstream.test.js: true JSON and forced-SSE custom-tool projections.
  • Targeted custom/non-stream/multi-turn suites: 19/19 passed.
  • Broader Responses translator/stream/continuity suites: 15 passed, with the repository's 2 remaining it.fails cases still expected.
  • ESLint clean on all modified files.

Live end-to-end

Tested through the exact routed model op-ericding/cx/gpt-5.6-sol and /v1/responses using a Responses Lite additional_tools custom exec declaration:

  • HTTP 200.
  • Persisted outbound provider request contains one Chat tool named exec, with required argument input.
  • Upstream returned a Chat tool call.
  • 9router emitted:
    • response.output_item.added with type: "custom_tool_call", name: "exec";
    • response.custom_tool_call_input.delta with raw input pwd;
    • response.custom_tool_call_input.done;
    • response.output_item.done with raw input: "pwd";
    • response.completed.
  • A second request carrying the returned custom_tool_call plus custom_tool_call_output: "/root" returned a normal final text response, confirming history translation.
  • The earlier encrypted-continuity live reproduction continues to return tool calls, and outbound provider requests contain no continuity-only fields.

This is a targeted fix for the openai ↔ openai-responses direction of the format-loss family described in #2347. #2348 proposes a broader unified projector.

nguyenha935 and others added 2 commits August 2, 2026 00:29
…PI clients

When a Responses-API client (e.g. Codex) is routed to a chat-native
OpenAI-compatible provider, non-streaming requests returned the raw
chat.completion body: object was "chat.completion" and tool_calls were
invisible to the client. Related to #2347 (openai -> openai-responses
direction of the leaked-format family).

Changes:
- nonStreamingHandler: add the openai(provider) -> openai-responses(client)
  case to translateNonStreamingResponse, projecting tool_calls to
  function_call items, content to output_text, reasoning to summary text;
  guard chat-specific post-processing so the Responses envelope is not
  mangled back into chat.completion
- sseToJsonHandler: branch the forced-SSE path on the upstream format
  (targetFormat) instead of the client format, so a Responses client behind
  a chat-native forced-streaming provider is parsed as chat SSE and then
  projected to Responses shape
- chatCore: pass targetFormat (providerResponseFormat) to
  handleForcedSSEToJson, consistent with the other two handlers

Tests: add tests/unit/openai-responses-nonstream.test.js (5 tests covering
tool_calls projection, text projection, chat->chat passthrough, forced-SSE
projection, and chat-client regression). The three projection tests fail on
the previous code and pass with this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…stream

The Responses->Chat request translator stashes reasoning encrypted_content
on assistant messages so an openai->responses round-trip can restore the
store=false continuity blob. That stash leaked into the body sent to
chat-native upstreams: OpenAI-compatible proxies reject the unknown
assistant-message field and answer every turn with a literal "400" body,
breaking multi-turn Codex sessions (the blob grows per turn, so every
retry failed too).

Strip encrypted_content / reasoning_encrypted_content from translated
messages at the dispatch boundary in handleChatCore, after translation.
The translator-level stash is preserved for round-trip symmetry (existing
multiturn tests untouched).

Tests: add tests/unit/continuity-strip.test.js — boundary strip removes
both keys while keeping reasoning_content and tool_calls; end-to-end
Responses multi-turn translation is clean after the strip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@nguyenha935 nguyenha935 changed the title fix(translator): convert non-streaming chat responses for Responses-API clients fix(translator): preserve Responses tools and strip continuity blobs upstream Aug 2, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@nguyenha935 nguyenha935 changed the title fix(translator): preserve Responses tools and strip continuity blobs upstream fix(translator): preserve Responses Lite tools across Chat providers Aug 2, 2026
…r ID

The runtime derived an OpenAI-compatible node transport from a substring
of the immutable provider ID instead of the stored apiType, so editing a
node API Type (Chat <-> Responses) in the dashboard had no effect.

Add resolveOpenAICompatibleApiType(provider, credentials): prefer the
stored providerSpecificData.apiType kept in sync with the node on
create/update, fall back to the ID substring for legacy nodes. Use it in
getProviderConfig/getTargetFormat and both executors buildUrl so format
selection and endpoint path agree. chatCore now passes credentials into
getTargetFormat.
@nguyenha935
nguyenha935 force-pushed the fix/responses-nonstream-chat-projection branch from 474ad86 to f0f6c91 Compare August 2, 2026 09:09
@decolua

decolua commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Thanks @nguyenha935 for the contribution! Reviewed and merged into master. 🙏

@decolua decolua closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants