Skip to content

fix: stream audio transcription responses - #5394

Open
streamsc wants to merge 1 commit into
QuantumNous:mainfrom
streamsc:codex/audio-stt-streaming
Open

fix: stream audio transcription responses#5394
streamsc wants to merge 1 commit into
QuantumNous:mainfrom
streamsc:codex/audio-stt-streaming

Conversation

@streamsc

@streamsc streamsc commented Jun 9, 2026

Copy link
Copy Markdown

⚠️ 提交说明 / PR Notice

Important

  • 请提供人工撰写的简洁摘要,避免直接粘贴未经整理的 AI 输出。

📝 变更描述 / Description

修复 /v1/audio/transcriptions 接口在 stream=true 时无法通过网关流式返回的问题。
OPENAI相关接口文档:https://developers.openai.com/api/docs/guides/speech-to-text#streaming-the-transcription-of-a-completed-audio-recording

修复前:客户端通过 multipart 表单传入 stream=true 时,网关没有把该请求识别为流式请求;同时 OpenAI 兼容的 STT 响应处理会固定读取完整上游响应体后再返回。因此即使上游 vLLM服务/v1/audio/transcriptions接口 已经按 SSE 流式输出,经过网关后客户端仍只能在请求结束后一次性收到结果。

修复后:音频转写/翻译请求会正确识别 stream=true,并在上游响应确认为 text/event-stream 时边读边转发给客户端。普通 JSON 响应仍保持原有读取和 usage 解析逻辑。

实现上,新增了音频请求对 stream 字段的解析,并将流式响应处理限制在 STT/translation 且上游确认为 SSE 的场景;这样既能恢复转写流式透传,也不会改变非流式接口的行为。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

本地验证通过:

GOPROXY=https://goproxy.cn,direct go test -timeout=120s ./dto -count=1
ok  	github.com/QuantumNous/new-api/dto	0.272s
GOPROXY=https://goproxy.cn,direct go test -timeout=120s ./relay/channel/openai -count=1
ok  	github.com/QuantumNous/new-api/relay/channel/openai	0.606s
实际基于此分支构建的线上版本验证
curl -N https://myhost/v1/audio/transcriptions \
  -H 'Authorization: Bearer myapi-key' \
  -F model=qwen3-asr-1.7b \
  -F file=@test-asr.wav \
  -F stream=true

@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR adds streaming support to audio transcription and translation endpoints. AudioRequest.Stream field enables clients to request streaming, IsStream method detects it and restricts it to appropriate endpoints, OpenaiSTTHandler now conditionally handles SSE responses with usage parsing, and speech endpoints explicitly prevent streaming.

Changes

Audio Streaming Support

Layer / File(s) Summary
Audio request stream field and detection logic
dto/audio.go, dto/audio_test.go, relay/helper/valid_request.go
AudioRequest gains Stream field; IsStream returns true for stream_format="sse" or when Stream is set on transcription/translation endpoints. Speech endpoints clear the Stream field to prevent streaming there. Table-driven test covers all endpoint-specific and format-based scenarios.
OpenaiSTTHandler streaming and usage parsing
relay/channel/openai/audio.go
STT handler detects SSE responses via Content-Type header, initializes fallback usage, scans streamed events for "usage" chunks, parses and applies token usage when found, and forwards streamed data to client. Non-stream responses normalize usage through shared parseSTTUsage helper.
OpenaiSTTHandler test coverage
relay/channel/openai/audio_test.go
Three tests validate streaming SSE output with preserved usage, non-stream JSON pass-through with correct Content-Length, and streamed usage-chunk extraction. Test helpers construct Gin context and mock HTTP responses.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

  • QuantumNous/new-api#1721: Both PRs refactor OpenaiSTTHandler to extract or override STT token usage, including parsing upstream usage versus fallback estimation.
  • QuantumNous/new-api#2087: Both PRs share audio request streaming detection via AudioRequest.Stream/IsStream, enabling conditional streaming for protocol-specific adaptors.

Suggested reviewers

  • creamlike1024

Poem

🐰 A stream of audio now flows so free,
Transcriptions dance where SSE,
Usage parsed from chunks so small,
Speech stays silent through it all! 🎵

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'fix: stream audio transcription responses' accurately describes the main change—enabling streaming for audio transcription endpoints when stream=true is set.
Linked Issues check ✅ Passed The PR successfully addresses issue #5378 by recognizing stream=true on audio requests and forwarding upstream SSE responses as streams rather than buffering them.
Out of Scope Changes check ✅ Passed All changes are focused on enabling streaming for /v1/audio/transcriptions and /v1/audio/translations endpoints, directly addressing the linked issue without extraneous modifications.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/v1/audio/transcriptions接口不支持流式请求

1 participant