fix(azure): force upstream SSE for streamed image gen - #2122
Conversation
Azure gpt-image-2 with stream=true never returned a final image because forceImageStreamUpstream was gated on !stream. For image generation the upstream request is always non-streaming (effectiveStream is forced false when faking streaming for the client), so without partial_images=1 the request hit Azure's 122s synchronous wall and timed out. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThe logic for forcing upstream SSE (Server-Sent Events) in image generation requests is modified to always apply when using OpenAI or Azure providers for image models, regardless of the client's streaming preference. This behavior is now consistently applied across the main execution path and retry/fallback paths. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 6/8 reviews remaining, refill in 8 minutes and 50 seconds.Comment |
There was a problem hiding this comment.
Pull request overview
Fixes Azure gpt-image-2 image-generation requests with stream=true by ensuring the gateway always forces upstream SSE (stream=true + partial_images=1) for OpenAI/Azure gpt-image-*, preventing Azure’s synchronous ~122s wall from causing timeouts and allowing the longer streaming timeout to apply.
Changes:
- Remove the
!streamgate soforceImageStreamUpstreamapplies regardless of client streaming preference. - Apply the same logic in both the initial request setup and the retry/provider-context re-resolution path.
- Update inline comments to reflect the corrected behavior and rationale.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Force upstream SSE for OpenAI/Azure gpt-image-* regardless of what the client | ||
| // requested. For image generation the upstream request is always non-streaming | ||
| // (effectiveStream is forced false above when faking streaming for the client), | ||
| // so partial_images=1 is needed in both cases to keep the connection alive past | ||
| // Azure's 122s synchronous wall and to use AI_STREAMING_TIMEOUT_MS (240s default) | ||
| // instead of AI_TIMEOUT_MS (180s). The SSE response is collapsed back into the | ||
| // regular non-streaming JSON shape before being returned (or re-wrapped as fake | ||
| // SSE for clients that requested streaming). | ||
| let forceImageStreamUpstream = | ||
| !stream && | ||
| isImageGeneration && | ||
| (usedProvider === "openai" || usedProvider === "azure"); |
Summary
gpt-image-2withstream=truenever returned a final image becauseforceImageStreamUpstreamwas gated on!stream.effectiveStreamis forced false when faking streaming for the client), so withoutpartial_images=1the request hit Azure's 122s synchronous wall and timed out.!streamgate in both call sites so upstream SSE injection (and the longer streaming timeout) applies whether or not the client requested streaming. The existingcollapseImageGenSse+fakeStreamingForImageGenwiring already handles re-wrapping the response for streaming clients.Test plan
gpt-image-2returns the final image instead of timing outgpt-image-2still works (regression check)gpt-image-*streaming and non-streaming still work🤖 Generated with Claude Code
Summary by CodeRabbit