Add SSE stream_format support for audio speech streaming

[syd520zy]

Summary

This PR adds stream_format="sse" support to /v1/audio/speech.

When stream_format="sse" is requested without stream=True, the speech endpoint now returns text/event-stream events:

• speech.audio.delta
• speech.audio.done
• speech.audio.error

Each speech.audio.delta contains a base64-encoded audio chunk in the delta field, plus the response_format.

The existing raw audio streaming behavior remains the default streaming path. If a request sets both stream=True and stream_format="sse", the endpoint still uses the existing non-SSE raw audio stream path and returns audio/pcm or audio/wav.

Changes

• Allow stream_format="sse" for OpenAICreateSpeechRequest.
• Add an SSE wrapper around the existing audio chunk generator.
• Keep stream=True raw audio streaming behavior higher priority than stream_format="sse".
• Treat stream_format="sse" as a streaming generation request internally so multimodal output is emitted as deltas.
• Add text/event-stream to /v1/audio/speech OpenAPI response metadata.
• Update tests for:
  • stream_format="sse" validation
  • SSE audio delta response shape
  • stream=True + stream_format="sse" preserving raw audio stream behavior
  • internal streaming parameter propagation for SSE
• Remove the old DFX invalid-param case that expected stream_format="sse" to be rejected for /v1/audio/speech.

Compatibility

This preserves the current streaming default:

• stream=True continues to return raw audio bytes.
• stream=True + stream_format="sse" continues to return raw audio bytes.
• SSE is opt-in via stream_format="sse" without stream=True.

Open Question

Should we consider changing the default streaming behavior to SSE in a follow-up, to better match OpenAI-style streaming semantics? That would be a breaking behavior change for existing raw-audio streaming clients, so this PR keeps the current default unchanged.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[linyueqian]

Thanks @syd520zy, this is the right approach for SSE and much more complete than #4185 (which accepts stream_format="sse" but falls through to a single non-streaming blob). A few items before merge:

CI blockers

• DCO is failing (action_required): commits need a sign-off. Fix and force-push:

  git rebase --signoff main        # or: git commit --amend --signoff for a single commit
  git push --force-with-lease

  Make sure your git user.email matches your GitHub-verified email.
• pre-commit is failing. Run locally and push the result: pre-commit run --all-files (ruff/format usually auto-fixes).
• FYI build and docs pass, but there's no unit-test job among the triggered checks, so the new tests aren't being exercised by CI yet. Worth confirming they pass locally / getting the full test CI triggered.

Event field name diverges from OpenAI (should be audio, not delta)
The payload is currently {"type":"speech.audio.delta","delta":"<b64>","response_format":"pcm"}. OpenAI's speech.audio.delta schema (required fields type + audio) puts the base64 chunk in audio:

{"type":"speech.audio.delta","audio":"<base64-encoded-audio-data>"}

delta is the Realtime API convention (response.audio.delta), which is a different event. A stock OpenAI client reading event.audio would get nothing here. Suggest renaming delta -> audio (keeping response_format as an extra field is fine). OpenAI's speech.audio.done also carries a usage object; optional to add.

Tests
Happy path is well covered. Consider adding the negative paths: speech.audio.error (exception inside the generator), sse + non-pcm/wav -> 4xx, and sse + speed != 1.0 -> 4xx. The existing test_sse_streaming assertion will also need updating once the field is renamed to audio.

Optional follow-up
#4185's one useful idea is treating stream_format="audio" as a raw-streaming trigger (OpenAI uses stream_format as the switch). Folding that in here would cover both audio and sse, and let us close #4185 to avoid the merge conflict (both touch create_speech + the stream_format default/validator).

[hsliuustc0106]

SSE streaming implementation looks correct. Tests cover the new behavior well.

[syd520zy]

Hi @linyueqian ,
I've addressed most of the review suggestions you left. Only the optional follow-up task is left unprocessed for now.
Could you help take another look when you have time? Thanks a lot for your feedback!
I will handle the optional follow-up in a separate commit later if needed.

[linyueqian]

Approving. Re-reviewed the latest revision and all earlier feedback is addressed, and I ran a live end-to-end smoke against a real Qwen3-TTS deploy.

Review items resolved

• Event field renamed delta to audio, matching OpenAI's speech.audio.delta schema. The docstring spelling out audio (Speech API) vs delta (Realtime API) is a nice touch.
• Negative-path tests added: mid-stream speech.audio.error, sse + non-pcm/wav returns 400, sse + speed != 1.0 returns 400.
• stream=True raw-audio precedence over sse is preserved and covered by a test.
• CI is green (DCO, pre-commit, build, docs).

E2E smoke (vLLM 0.23, this PR's head, Qwen3-TTS-1.7B-CustomVoice on an H20-class GPU)

• stream_format="sse" + pcm: returns text/event-stream, a sequence of speech.audio.delta events each carrying a base64 audio chunk plus response_format, terminated by speech.audio.done. Decoding and concatenating the deltas yields valid 24kHz mono PCM, and Whisper transcribes it back to the exact input text.
• stream_format="sse" + wav: same envelope, reassembles to a valid WAV.
• stream=True + stream_format="sse": still returns raw audio/pcm with no SSE framing (precedence intact).
• sse + mp3 returns 400, sse + speed=2.0 returns 400, both rejected before any stream begins.

Non-blocking nits (fine to defer)

• OpenAI's speech.audio.done also carries a usage object, which this omits.
• The stream_format="audio" raw-streaming trigger you deferred to a follow-up is fine to leave for now.

Nice work, thanks for the thorough iteration.

[syd520zy]

@linyueqian
I have folded in the useful idea from #4185: stream_format="audio" now works as an explicit raw audio streaming trigger, while keeping the existing legacy stream=True behavior.

The resulting request behavior is:

Request	Behavior
no stream, no stream_format	non-streaming
stream_format="audio"	raw streaming
stream=True	raw streaming, for backward compatibility
stream_format="sse"	SSE
stream=True + stream_format="sse"	raw streaming takes precedence, preserving the current #4490 behavior

I also fixed the two failing tests:

• test_sse_rejects_unsupported_response_format
• test_sse_rejects_speed_adjustment

The previous failures happened because the tests strictly expected an endpoint-level 400, while the latest request model can reject those invalid streaming combinations earlier during FastAPI/Pydantic request validation, which returns 422. The tests now assert the intended behavior instead: invalid SSE streaming requests must be rejected and must not return a successful 200 streaming response.