Skip to content

fix(azure): force upstream SSE for streamed image gen #2122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
steebchen merged 1 commit into from
Apr 30, 2026
Merged

fix(azure): force upstream SSE for streamed image gen #2122

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions apps/gateway/src/chat/chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3719,13 +3719,15 @@ chat.openapi(completions, async (c) => {
// When the provider only supports streaming, force it even if the client didn't request it.
// The upstream request uses effectiveStream; the client response uses stream.
const forceStream = streamingSupport === "only" && !stream;
// Force upstream SSE for OpenAI/Azure gpt-image-* even when the client requested
// non-streaming. partial_images=1 keeps the connection alive past Azure's 122s
// synchronous wall and lets us use AI_STREAMING_TIMEOUT_MS (240s default) instead
// of AI_TIMEOUT_MS (180s). The SSE response is collapsed back into the regular
// non-streaming JSON shape before the client sees it.
// Force upstream SSE for OpenAI/Azure gpt-image-* regardless of what the client
// requested. For image generation the upstream request is always non-streaming
// (effectiveStream is forced false above when faking streaming for the client),
// so partial_images=1 is needed in both cases to keep the connection alive past
// Azure's 122s synchronous wall and to use AI_STREAMING_TIMEOUT_MS (240s default)
// instead of AI_TIMEOUT_MS (180s). The SSE response is collapsed back into the
// regular non-streaming JSON shape before being returned (or re-wrapped as fake
// SSE for clients that requested streaming).
let forceImageStreamUpstream =
!stream &&
isImageGeneration &&
(usedProvider === "openai" || usedProvider === "azure");
Comment on lines +3722 to 3732
const effectiveStream = fakeStreamingForImageGen
Expand Down Expand Up @@ -4036,7 +4038,6 @@ chat.openapi(completions, async (c) => {
requestCanBeCanceled = ctx.requestCanBeCanceled;
isImageGeneration = ctx.isImageGeneration;
forceImageStreamUpstream =
!stream &&
isImageGeneration &&
(usedProvider === "openai" || usedProvider === "azure");
if (forceImageStreamUpstream) {
Expand Down
Loading