feat: vertex veo sora-compatible video output - #2102
Conversation
WalkthroughTwo files are modified to add OpenAI video format support for Vertex AI tasks. A new Changes
Sequence DiagramsequenceDiagram
participant Client
participant RelayTask as relay_task.go
participant TaskAdaptor as adaptor.go
participant Common
Client->>RelayTask: Request to /v1/videos/...
alt URL starts with /v1/videos/
RelayTask->>RelayTask: Skip standard response marshalling
RelayTask->>TaskAdaptor: ConvertToOpenAIVideo(task)
TaskAdaptor->>TaskAdaptor: Derive task name & infer model
TaskAdaptor->>TaskAdaptor: Build OpenAI video DTO
TaskAdaptor->>Common: Marshal(videoDTO)
Common-->>TaskAdaptor: []byte JSON
TaskAdaptor-->>RelayTask: []byte, error
RelayTask-->>Client: Video response
else Standard request
RelayTask->>RelayTask: Normal response flow
end
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes The changes are localized and straightforward: a new conversion method following standard patterns, and conditional URI-based routing logic. The implementation is self-contained and requires minimal context about existing video handling. Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
relay/channel/task/vertex/adaptor.go (1)
322-324: Remove the redundant length check and clarify backward compatibility intent.The observation about the redundant
len(task.FailReason) > 0check is valid—strings.HasPrefixreturnsfalsefor empty strings, making this condition unnecessary.More importantly, the codebase now prevents data URLs from being stored in
FailReason(confirmed in relay_task.go lines 354-357 and controller/task_video.go lines 140-142). This means the condition on line 322 should only trigger for legacy data. If this is intentional for backward compatibility, add a comment explaining it. Otherwise, this code may be unreachable.Apply this diff:
- if strings.HasPrefix(task.FailReason, "data:") && len(task.FailReason) > 0 { + if strings.HasPrefix(task.FailReason, "data:") { v.SetMetadata("url", task.FailReason) }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
relay/channel/task/vertex/adaptor.go(2 hunks)relay/relay_task.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-26T09:59:00.337Z
Learnt from: Sh1n3zZ
PR: QuantumNous/new-api#1659
File: relay/relay_task.go:285-305
Timestamp: 2025-08-26T09:59:00.337Z
Learning: In controller/task_video.go, data: URLs (containing base64 encoded video data) are prevented from being stored in task.FailReason by checking if the URL starts with "data:" before assignment. This same pattern should be applied consistently across the codebase.
Applied to files:
relay/relay_task.go
🧬 Code graph analysis (1)
relay/channel/task/vertex/adaptor.go (2)
dto/openai_video.go (1)
NewOpenAIVideo(43-47)common/json.go (1)
Marshal(21-23)
🔇 Additional comments (6)
relay/channel/task/vertex/adaptor.go (4)
13-13: LGTM!The import is necessary for
common.Marshalused in the newConvertToOpenAIVideomethod.
307-314: LGTM!The error handling for task ID decoding and model name extraction is reasonable, with appropriate fallback to a default model name that's consistent with line 91.
315-321: LGTM!The OpenAI video DTO is correctly populated with task metadata, and the status/progress conversions are appropriate.
326-327: LGTM!The marshaling and error handling are correct.
relay/relay_task.go (2)
385-398: Conditional response building looks correct.The logic to skip standard response marshalling for
/v1/videos/URIs is appropriate, as these are handled separately by the OpenAI format conversion below (lines 406-422). However, note the data URL issue flagged in the previous comment also affects this response path.
354-357: Verify whether data URLs should be returned in non-OpenAI format responses or discarded entirely.The data URL prevention logic is intentional (matching the pattern in controller/task_video.go), but it creates a real data loss issue for the non-OpenAI response path:
When
ti.Urlis a data URL (video content embedded in the URL):
- It's intentionally not stored in
originTask.FailReason(line 356 has an empty block)- But the response at line 392 returns
originTask.FailReasonas theurlfield- Result: The data URL is lost; clients get an empty or stale URL instead
The OpenAI format path (line 413+) bypasses this issue by reading from
originTask.Datarather thanFailReason.Options:
- If data URLs should be returned: Use the suggested local variable approach to preserve
ti.Urlfor the response without persisting it- If data URLs should be discarded: Add explicit handling to omit the
urlfield or return a placeholder when a data URL was receivedClarify the intended behavior before proceeding with the fix.
…ptor feat: vertex veo sora-compatible video output
2 participants
为 vertex 渠道 veo 模型添加
ConvertToOpenAIVideo方法支持以 OpenAI Sora 格式返回 vertex 渠道的 veo 模型生成的视频内容
Summary by CodeRabbit
New Features
Bug Fixes