Skip to content

fix(sse): anuncia role assistant no 1º delta do stream Responses→Chat - #3911

Merged
diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.26from
diego-anselmo:fix/responses-chat-assistant-role-first-chunk
Jun 15, 2026
Merged

diegosouzapw merged 3 commits into
cheaper-inference:release/v3.8.26from
diego-anselmo:fix/responses-chat-assistant-role-first-chunk

Conversation

@diego-anselmo

Copy link
Copy Markdown
Contributor

Summary

Tool calls em streaming de modelos codex (OpenAI Responses API, targetFormat: openai-responses)
são silenciosamente descartados por clients OpenAI estritos. O tradutor de stream Responses→Chat
nunca emite role: "assistant" no primeiro delta, então o _convertDeltaToMessageChunk do
@langchain/openai (usado pelo AI Agent do n8n) não tipa a mensagem como AIMessageChunk e nunca
acumula os deltas de tool_call. O agente executa a tool, mas retorna resposta final vazia
({"output": ""}).

Um stream Chat Completions normal abre com delta.role = "assistant"; a Responses API não tem
evento de anúncio de role, então o tradutor precisa sintetizá-lo. Este PR carimba
role: "assistant" no primeiro chunk não-nulo emitido (tanto no caminho de texto quanto no de
tool_call), seguindo o contrato de streaming do Chat Completions. Non-streaming e providers em
formato chat já não eram afetados; só a saída de streaming openai-responses muda.

Reprodução

curl -sN $OMNIROUTE/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model":"codex/gpt-5.5","stream":true,
  "tools":[{"type":"function","function":{"name":"t","parameters":{"type":"object","properties":{}}}}],
  "messages":[{"role":"system","content":"Always call tool t."},{"role":"user","content":"go"}]
}'

Antes: primeiro chunk delta:{ "tool_calls":[...] } (sem role) → o langchain descarta a tool call.
Depois: primeiro chunk delta:{ "role":"assistant", "tool_calls":[...] }.

Related Issues

  • Nenhuma (encontrado em debug de campo; posso abrir uma issue se preferir).

Validation

  • npm run lint — limpo nos arquivos alterados
  • Testes unitários — teste novo + suites do converter (claude-code-rendering-fixes,
    responses-translation-fixes): 34/34 verdes (suite completa + coverage via CI)
  • npm run test:coverage — via CI
  • SonarQube — via CI

Tests Added Or Updated

  • tests/unit/responses-chat-assistant-role-first-chunk.test.ts (novo) — verifica role=assistant
    no primeiro delta de tool_call e no primeiro delta de texto, e que ele não se repete nos deltas
    seguintes. Escrito em TDD (red→green).

Coverage Notes

  • Altera open-sse/translator/response/openai-responses.ts; o teste novo exercita o wrapper
    withAssistantRoleOnFirstDelta nos dois caminhos de emissão.

Reviewer Notes

  • Wrapper fino que carimba o role uma única vez, no primeiro chunk que carrega um objeto delta —
    sem mudar a lógica de tradução por evento.
  • Verificado ao vivo em codex/gpt-5.5 streaming com tools: o primeiro chunk tool_calls agora
    traz role:"assistant" e o AI Agent do n8n completa o loop tool→resposta.
  • Aditivo / baixo risco; caminhos non-streaming e em formato chat ficam intactos.

…elta

The Responses→Chat translator never emitted role=assistant on the first delta.
OpenAI Chat Completions streams open with delta.role=assistant; the Responses API
has no role-announcement event, so it must be synthesized on the first emitted
chunk. Strict streaming clients (e.g. @langchain/openai's
_convertDeltaToMessageChunk, used by n8n's AI Agent) key off the first chunk's
role to build an AIMessageChunk. Without it, streamed tool_call deltas are dropped
and the agent returns an empty response despite a well-formed upstream
function_call.

Adds a wrapper that stamps role=assistant on the first non-null emitted chunk,
covering every emission path (text + tool_call deltas).

TDD: tests/unit/responses-chat-assistant-role-first-chunk.test.ts (red->green).
Existing converter suites stay green (32/32).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to synthesize the assistant role on the first emitted delta when translating OpenAI Responses API chunks to OpenAI Chat Completions format. This ensures compatibility with strict streaming clients like LangChain that require a role announcement on the first chunk. A corresponding unit test suite has been added to verify this behavior for both tool calls and text deltas. There are no review comments, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@diegosouzapw
diegosouzapw changed the base branch from main to release/v3.8.26 June 15, 2026 22:38
diegosouzapw and others added 2 commits June 15, 2026 19:41
…e wrapper

PR cheaper-inference#3911 own growth: openai-responses.ts 878->903 (+25). Updates the
frozen file-size baseline so Fast Quality Gates pass.

Co-authored-by: diegosouzapw <diegosouza.pw@gmail.com>
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks, @diego-anselmo! 🙏 Spot-on fix — the Responses API has no role-announcement event, so translating Responses→Chat without synthesizing role: "assistant" on the first delta makes strict clients (langchain's _convertDeltaToMessageChunk, n8n's AI Agent) drop the streamed tool_call deltas and return an empty response. The withAssistantRoleOnFirstDelta wrapper is clean and your 2 TDD cases (tool_call + text first chunk, role only on the first) pin it well. Green locally. Added a small file-size baseline bump (+25). Merging into release/v3.8.26. 🚀

@diegosouzapw
diegosouzapw merged commit a5e704f into cheaper-inference:release/v3.8.26 Jun 15, 2026
1 check passed
@diegosouzapw diegosouzapw mentioned this pull request Jun 16, 2026
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants