fix(azure): preserve responses API streaming events

[steebchen]

Summary

Streaming requests to azure/gpt-5.5 (and any other Azure model with supportsResponsesApi: true) returned:

{"code":"upstream_error","message":"Response finished successfully but returned no content or tool calls"}

The Azure prompt-filter-only chunk guard at transform-streaming-to-openai.ts:687 was dropping every Responses API event because those events also lack top-level id/object/choices/usage (their data lives under data.response.* or alongside a type field). With every chunk transformed to null, no content/tokens accumulated and the empty-response detector at chat/chat.ts:7204 fired.

Fix: only drop the chunk when data.type is also absent — Responses API events always carry type (response.created, response.output_text.delta, response.completed, …); the chat-completions prompt-filter chunk does not.

Same guard for azure-ai-foundry got the same treatment defensively (no current model triggers it, but the shape is identical).

Test plan

• Unit tests added for prompt-filter drop, output_text delta passthrough, and response.completed usage extraction (11 tests total pass)
• Reproduced the original error against the local gateway with azure/gpt-5.5 streaming
• Verified fix with x-no-fallback: true — used_provider: "azure", content streams, usage reported correctly
• pnpm build clean
• pnpm format clean

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Resolved streaming transformation issue for Azure providers where valid response events were being incorrectly filtered and discarded.

• Tests

  • Added test cases for Azure streaming support, covering event filtering and response usage token mapping.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 3166ccee-14c8-4932-9c12-808fe11b38e7

📥 Commits

Reviewing files that changed from the base of the PR and between c8f5918 and fc3c0a9.

📒 Files selected for processing (2)

• apps/gateway/src/chat/tools/transform-streaming-to-openai.spec.ts
• apps/gateway/src/chat/tools/transform-streaming-to-openai.ts

---

Walkthrough

Azure provider streaming transformation guards refined to preserve Responses API events by checking the data.type field. Corresponding test cases added to verify prompt filter dropping, delta content preservation, and usage mapping behavior for Azure providers.

Changes

Azure Streaming Transformation & Coverage

Layer / File(s)	Summary
Guard Condition Refinement apps/gateway/src/chat/tools/transform-streaming-to-openai.ts	Guard conditions at lines 684–691 and 1246–1253 updated to check for data.type field, preventing discard of legitimate Azure Responses API events that lack top-level id/object/choices/usage fields.
Test Coverage apps/gateway/src/chat/tools/transform-streaming-to-openai.spec.ts	Added three test cases (lines 203–271): prompt-filter-only chunk drops to null; delta events preserve content; response.completed usage maps to tokens and finish_reason.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• theopenco/llmgateway#2089: Both PRs refine streaming event handling in transform-streaming-to-openai.ts by checking data.type to distinguish provider-specific chunk types (Responses API vs. Anthropic ping events).
• theopenco/llmgateway#2128: Both PRs enhance Azure Responses API and azure-ai-foundry provider streaming transformation in the same file.
• theopenco/llmgateway#2101: Both PRs modify Azure/OpenAI streaming guard logic in transform-streaming-to-openai.ts; one tightens prompt-filter drop conditions, the other refines guards for Responses API event preservation.

Suggested reviewers

• smakosh
• RATCHAW

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(azure): preserve responses API streaming events' is directly related to the main change: modifying Azure streaming transformation logic to prevent dropping valid Responses API events.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/azure-responses-streaming-filter

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Fixes Azure OpenAI streaming for models that use the Responses API (e.g., azure/gpt-5.5) by preventing the “prompt-filter-only leading chunk” guard from incorrectly dropping all Responses API events, which previously caused downstream “empty response” errors.

Changes:

• Update the Azure prompt-filter-only chunk drop guard to only trigger when the chunk also lacks data.type, ensuring Responses API events (which always include type) are preserved.
• Apply the same defensive data.type exemption to the analogous guard for azure-ai-foundry.
• Add unit tests covering: prompt-filter chunk dropping, Responses API output text delta passthrough, and response.completed usage extraction.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
apps/gateway/src/chat/tools/transform-streaming-to-openai.ts	Refines Azure/Azure-AI-Foundry streaming guard logic to avoid dropping Responses API events.
apps/gateway/src/chat/tools/transform-streaming-to-openai.spec.ts	Adds regression tests for Azure prompt-filter chunk handling and Responses API streaming/usage mapping.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.