新增: 豆包视频1.5pro - #2632
Conversation
WalkthroughIntroduces custom JSON serialization types (IntValue, BoolValue) in a new DTO file, expands the Doubao video task adaptor to support video content with OpenAI DTO conversion, adds a new model identifier to the Doubao model list, and extends channel type mapping to route VolcEngine requests through the Doubao adaptor. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Relay as Relay/TaskAdaptor
participant DoubaoAPI as Doubao API
participant Response as Response Handler
Client->>Relay: Send video task request
Relay->>Relay: BuildRequestBody (extract task via GetTaskRequest)
Relay->>Relay: convertToRequestPayload (build video request with ContentItem)
Relay->>DoubaoAPI: Submit video generation request (video + options)
DoubaoAPI-->>Relay: Return task status (processing/running)
Relay->>Relay: ParseTaskResult (check status, map "running" as in-progress)
DoubaoAPI-->>Relay: Task complete with video URL
Relay->>Relay: ConvertToOpenAIVideo (translate to OpenAI DTO)
Relay->>Response: DoResponse (serialize OpenAIVideo payload)
Response-->>Client: Return video task result (ID, model, URL, status)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
relay/channel/task/doubao/adaptor.go (1)
122-138:UpstreamModelNamecan be user-overridden via metadata unmarshal.
info.UpstreamModelName = body.Model(Line 132) reflects whateverconvertToRequestPayloadproduced; butconvertToRequestPayloadcurrently unmarshalsreq.Metadatainto the whole payload (Lines 235-243), which can overrideModel. If model override is not intended, block it (or explicitly allowlist it).Also applies to: 132-133
🤖 Fix all issues with AI agents
In @relay/channel/task/doubao/adaptor.go:
- Around line 286-311: In TaskAdaptor.ConvertToOpenAIVideo, stop ignoring
marshal errors, only set CompletedAt for terminal task states, and use the
normalized originTask.Status when marking errors: check the internal
originTask.Status (not dResp.Status) to decide if the task failed and to
determine terminal state (follow the same terminal-status check used elsewhere,
e.g., gemini/adaptor.go), set openAIVideo.CompletedAt = originTask.UpdatedAt
only when originTask.Status is a terminal state (SUCCESS/FAILURE), and
capture/return any error from common.Marshal(openAIVideo) instead of discarding
it.
🧹 Nitpick comments (3)
dto/values.go (2)
8-30:IntValueshould handlenulland avoidintrange surprises.Today
"duration": null(or other IntValue non-pointer fields) will fail unmarshal; alsoint/Atoiis arch-dependent. Suggest: acceptnullas “unset/zero”, and parse viaParseIntwith range checks (or switch toint64).Proposed direction (handles null + trims + ParseInt)
type IntValue int func (i *IntValue) UnmarshalJSON(b []byte) error { + bb := bytes.TrimSpace(b) + if bytes.Equal(bb, []byte("null")) { + *i = 0 + return nil + } var n int if err := json.Unmarshal(b, &n); err == nil { *i = IntValue(n) return nil } var s string if err := json.Unmarshal(b, &s); err != nil { return err } - v, err := strconv.Atoi(s) + v64, err := strconv.ParseInt(strings.TrimSpace(s), 10, 0) if err != nil { return err } - *i = IntValue(v) + *i = IntValue(int(v64)) return nil }
32-55:BoolValuestring parsing is case-sensitive andelsebranch is redundant.Consider accepting
"TRUE"/"False"viastrings.EqualFold(and optionally trimming). The finaljson.Unmarshal(data, &boolean)(Line 49) will just error again for non-bool inputs; returning a clearer error is nicer.relay/channel/task/doubao/adaptor.go (1)
28-59: Validate/normalizeContentItem.Type(now includes"video") to avoid upstream schema drift.Since
"type"is free-form (Line 29), consider centralizing allowed values (text,image_url,video) as constants and validating before sending upstream, especially if metadata can inject/overridecontent.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
dto/values.gorelay/channel/task/doubao/adaptor.gorelay/channel/task/doubao/constants.gorelay/relay_adaptor.go
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-26T09:59:00.337Z
Learnt from: Sh1n3zZ
Repo: QuantumNous/new-api PR: 1659
File: relay/relay_task.go:285-305
Timestamp: 2025-08-26T09:59:00.337Z
Learning: In controller/task_video.go, the redactVideoResponseBody function sanitizes video task responses by removing bytesBase64Encoded fields and truncating base64 strings to 256 characters to prevent large binary data from being stored in task.Data.
Applied to files:
relay/channel/task/doubao/adaptor.go
🧬 Code graph analysis (3)
relay/relay_adaptor.go (1)
constant/channel.go (2)
ChannelTypeDoubaoVideo(54-54)ChannelTypeVolcEngine(45-45)
relay/channel/task/doubao/adaptor.go (5)
relay/channel/task/sora/adaptor.go (1)
ImageURL(33-35)dto/values.go (2)
BoolValue(32-32)IntValue(8-8)relay/common/relay_utils.go (1)
GetTaskRequest(62-72)dto/openai_video.go (2)
NewOpenAIVideo(43-47)OpenAIVideoError(49-52)common/json.go (2)
Marshal(21-23)Unmarshal(9-11)
dto/values.go (1)
common/json.go (2)
Unmarshal(9-11)Marshal(21-23)
🔇 Additional comments (3)
relay/channel/task/doubao/adaptor.go (2)
258-266: Status mapping improvement (running=> in-progress) looks good.
145-174: The concerns raised in this review comment are incorrect. The code is functioning as intended.Non-2xx responses are already handled correctly: relay_task.go lines 205-210 validate the upstream status code before calling DoResponse. Non-2xx responses trigger an early return with proper error handling; they never reach the unmarshaling logic in DoResponse.
c.JSON has no conflict: All task adaptors (suno, ali, kling, jimeng, vertex, vidu, doubao, sora, gemini, hailuo) uniformly call
c.JSON(http.StatusOK, ...)to send the HTTP response to the client. The returnedtaskDatais used for database persistence (stored intask.Data), not HTTP response writing. There is no layering conflict—HTTP response and task data persistence serve separate purposes.relay/channel/task/doubao/constants.go (1)
3-8: Add new model to volcengine list if supported. The new modeldoubao-seedance-1-5-pro-251215is only intask/doubao/constants.go, but a separate doubao model list exists inrelay/channel/volcengine/constants.go(lines 3-17). Sincevolcenginealready includesdoubao-seedance-1-0-pro-250528(line 13), verify whethervolcengineshould also have the new1-5-promodel for consistency and to avoid incomplete feature support.
| metadata := req.Metadata | ||
| medaBytes, err := json.Marshal(metadata) | ||
| if err != nil { | ||
| return nil, errors.Wrap(err, "metadata marshal metadata failed") | ||
| } | ||
| err = json.Unmarshal(medaBytes, &r) | ||
| if err != nil { | ||
| return nil, errors.Wrap(err, "unmarshal metadata failed") | ||
| } |
There was a problem hiding this comment.
Security: metadata “unmarshal into payload” enables callback_url SSRF + overrides content/model.
json.Unmarshal(medaBytes, &r) (Line 240) allows user-supplied metadata to set callback_url, content, model, etc. This is risky (SSRF/callback abuse + bypassing server-side model/content validation). Prefer an explicit allowlist of metadata fields and map them onto requestPayload (and validate CallbackURL if it remains supported).
Safer pattern sketch (allowlist-only)
- err = json.Unmarshal(medaBytes, &r)
- if err != nil {
- return nil, errors.Wrap(err, "unmarshal metadata failed")
- }
+ type payloadMeta struct {
+ CallbackURL string `json:"callback_url,omitempty"`
+ ReturnLastFrame *dto.BoolValue `json:"return_last_frame,omitempty"`
+ ServiceTier string `json:"service_tier,omitempty"`
+ ExecutionExpiresAfter dto.IntValue `json:"execution_expires_after,omitempty"`
+ GenerateAudio *dto.BoolValue `json:"generate_audio,omitempty"`
+ Draft *dto.BoolValue `json:"draft,omitempty"`
+ Resolution string `json:"resolution,omitempty"`
+ Ratio string `json:"ratio,omitempty"`
+ Duration dto.IntValue `json:"duration,omitempty"`
+ Frames dto.IntValue `json:"frames,omitempty"`
+ Seed dto.IntValue `json:"seed,omitempty"`
+ CameraFixed *dto.BoolValue `json:"camera_fixed,omitempty"`
+ Watermark *dto.BoolValue `json:"watermark,omitempty"`
+ }
+ var m payloadMeta
+ if err := json.Unmarshal(medaBytes, &m); err != nil {
+ return nil, errors.Wrap(err, "unmarshal metadata failed")
+ }
+ r.CallbackURL = m.CallbackURL
+ r.ReturnLastFrame = m.ReturnLastFrame
+ r.ServiceTier = m.ServiceTier
+ r.ExecutionExpiresAfter = m.ExecutionExpiresAfter
+ r.GenerateAudio = m.GenerateAudio
+ r.Draft = m.Draft
+ r.Resolution = m.Resolution
+ r.Ratio = m.Ratio
+ r.Duration = m.Duration
+ r.Frames = m.Frames
+ r.Seed = m.Seed
+ r.CameraFixed = m.CameraFixed
+ r.Watermark = m.Watermark📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| metadata := req.Metadata | |
| medaBytes, err := json.Marshal(metadata) | |
| if err != nil { | |
| return nil, errors.Wrap(err, "metadata marshal metadata failed") | |
| } | |
| err = json.Unmarshal(medaBytes, &r) | |
| if err != nil { | |
| return nil, errors.Wrap(err, "unmarshal metadata failed") | |
| } | |
| metadata := req.Metadata | |
| medaBytes, err := json.Marshal(metadata) | |
| if err != nil { | |
| return nil, errors.Wrap(err, "metadata marshal metadata failed") | |
| } | |
| type payloadMeta struct { | |
| CallbackURL string `json:"callback_url,omitempty"` | |
| ReturnLastFrame *dto.BoolValue `json:"return_last_frame,omitempty"` | |
| ServiceTier string `json:"service_tier,omitempty"` | |
| ExecutionExpiresAfter dto.IntValue `json:"execution_expires_after,omitempty"` | |
| GenerateAudio *dto.BoolValue `json:"generate_audio,omitempty"` | |
| Draft *dto.BoolValue `json:"draft,omitempty"` | |
| Resolution string `json:"resolution,omitempty"` | |
| Ratio string `json:"ratio,omitempty"` | |
| Duration dto.IntValue `json:"duration,omitempty"` | |
| Frames dto.IntValue `json:"frames,omitempty"` | |
| Seed dto.IntValue `json:"seed,omitempty"` | |
| CameraFixed *dto.BoolValue `json:"camera_fixed,omitempty"` | |
| Watermark *dto.BoolValue `json:"watermark,omitempty"` | |
| } | |
| var m payloadMeta | |
| if err := json.Unmarshal(medaBytes, &m); err != nil { | |
| return nil, errors.Wrap(err, "unmarshal metadata failed") | |
| } | |
| r.CallbackURL = m.CallbackURL | |
| r.ReturnLastFrame = m.ReturnLastFrame | |
| r.ServiceTier = m.ServiceTier | |
| r.ExecutionExpiresAfter = m.ExecutionExpiresAfter | |
| r.GenerateAudio = m.GenerateAudio | |
| r.Draft = m.Draft | |
| r.Resolution = m.Resolution | |
| r.Ratio = m.Ratio | |
| r.Duration = m.Duration | |
| r.Frames = m.Frames | |
| r.Seed = m.Seed | |
| r.CameraFixed = m.CameraFixed | |
| r.Watermark = m.Watermark |
| func (a *TaskAdaptor) ConvertToOpenAIVideo(originTask *model.Task) ([]byte, error) { | ||
| var dResp responseTask | ||
| if err := json.Unmarshal(originTask.Data, &dResp); err != nil { | ||
| return nil, errors.Wrap(err, "unmarshal doubao task data failed") | ||
| } | ||
|
|
||
| openAIVideo := dto.NewOpenAIVideo() | ||
| openAIVideo.ID = originTask.TaskID | ||
| openAIVideo.TaskID = originTask.TaskID | ||
| openAIVideo.Status = originTask.Status.ToVideoStatus() | ||
| openAIVideo.SetProgressStr(originTask.Progress) | ||
| openAIVideo.SetMetadata("url", dResp.Content.VideoURL) | ||
| openAIVideo.CreatedAt = originTask.CreatedAt | ||
| openAIVideo.CompletedAt = originTask.UpdatedAt | ||
| openAIVideo.Model = originTask.Properties.OriginModelName | ||
|
|
||
| if dResp.Status == "failed" { | ||
| openAIVideo.Error = &dto.OpenAIVideoError{ | ||
| Message: "task failed", | ||
| Code: "failed", | ||
| } | ||
| } | ||
|
|
||
| jsonData, _ := common.Marshal(openAIVideo) | ||
| return jsonData, nil | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cd /tmp && find . -type f -name "*.go" | grep -E "(adaptor|task_video)" | head -20Repository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# First, let's locate the file and examine the actual code
fd -t f "adaptor.go" | grep doubaoRepository: QuantumNous/new-api
Length of output: 99
🏁 Script executed:
# Get the file structure and check the doubao adaptor
git ls-files | grep -E "(relay/channel|controller)" | head -30Repository: QuantumNous/new-api
Length of output: 774
🏁 Script executed:
# Search for the doubao adaptor file
find . -name "adaptor.go" -path "*/doubao/*" 2>/dev/nullRepository: QuantumNous/new-api
Length of output: 101
🏁 Script executed:
# Read the doubao adaptor file to examine the ConvertToOpenAIVideo function
cat -n relay/channel/task/doubao/adaptor.go | head -320Repository: QuantumNous/new-api
Length of output: 11781
🏁 Script executed:
# Search for task_video.go to understand the sanitization patterns
find . -name "task_video.go" 2>/dev/nullRepository: QuantumNous/new-api
Length of output: 89
🏁 Script executed:
# Check the model definitions for TaskStatus
rg "TaskStatus" --type go -A 3 | head -50Repository: QuantumNous/new-api
Length of output: 2328
🏁 Script executed:
# Examine the task_video.go to understand sanitization patterns
cat -n controller/task_video.go | head -350Repository: QuantumNous/new-api
Length of output: 12733
🏁 Script executed:
# Look for the redactVideoResponseBody function
rg "redactVideoResponseBody|bytesBase64Encoded" --type go -B 5 -A 10Repository: QuantumNous/new-api
Length of output: 6983
🏁 Script executed:
# Check model/task.go to verify TaskStatus constants
cat -n model/task.go | grep -A 20 "type TaskStatus"Repository: QuantumNous/new-api
Length of output: 711
🏁 Script executed:
# Check for other similar adaptor implementations to understand patterns
find relay/channel/task -name "adaptor.go" -type f | head -5Repository: QuantumNous/new-api
Length of output: 244
🏁 Script executed:
# Look at another adaptor to see how they handle ConvertToOpenAIVideo
rg "ConvertToOpenAIVideo" --type go -A 20 | head -80Repository: QuantumNous/new-api
Length of output: 5038
🏁 Script executed:
# Check the common.Marshal function to understand what errors it can return
rg "func Marshal" --type go -B 2 -A 8 | head -40Repository: QuantumNous/new-api
Length of output: 423
🏁 Script executed:
# Verify the data: URL sanitization pattern in task_video.go more carefully
sed -n '143,155p' controller/task_video.goRepository: QuantumNous/new-api
Length of output: 416
🏁 Script executed:
# Check if there are any other implementations that show how to properly handle CompletedAt
rg "CompletedAt = " relay/channel/task --type go -B 2 -A 2Repository: QuantumNous/new-api
Length of output: 2751
🏁 Script executed:
# Look at the vertex adaptor more carefully to understand the data: URL pattern
sed -n '410,450p' relay/channel/task/vertex/adaptor.goRepository: QuantumNous/new-api
Length of output: 110
🏁 Script executed:
# Check if doubao's dResp.Status could be inconsistent with originTask.Status
sed -n '248,283p' relay/channel/task/doubao/adaptor.goRepository: QuantumNous/new-api
Length of output: 1202
ConvertToOpenAIVideo: don't ignore marshal errors; avoid setting CompletedAt for non-terminal states; use normalized task status instead of raw API response.
jsonData, _ := common.Marshal(...)(line 309) silently drops errors that can occur during marshaling.CompletedAt = originTask.UpdatedAt(line 299) is set unconditionally, even for queued/running tasks. Should only set for terminal states (SUCCESS/FAILURE), following the pattern in gemini/adaptor.go.if dResp.Status == "failed"(line 302) checks the raw upstream status instead of the normalizedoriginTask.Status. Since ParseTaskResult already maps dResp.Status to internal TaskStatus, use originTask.Status for consistency.
Proposed fix
openAIVideo.CreatedAt = originTask.CreatedAt
-openAIVideo.CompletedAt = originTask.UpdatedAt
+if originTask.Status == model.TaskStatusSuccess || originTask.Status == model.TaskStatusFailure {
+ openAIVideo.CompletedAt = originTask.UpdatedAt
+}
openAIVideo.Model = originTask.Properties.OriginModelName
-if dResp.Status == "failed" {
+if originTask.Status == model.TaskStatusFailure {
openAIVideo.Error = &dto.OpenAIVideoError{
Message: "task failed",
Code: "failed",
}
}
-jsonData, _ := common.Marshal(openAIVideo)
-return jsonData, nil
+jsonData, err := common.Marshal(openAIVideo)
+if err != nil {
+ return nil, errors.Wrap(err, "marshal openai video failed")
+}
+return jsonData, nil🤖 Prompt for AI Agents
In @relay/channel/task/doubao/adaptor.go around lines 286 - 311, In
TaskAdaptor.ConvertToOpenAIVideo, stop ignoring marshal errors, only set
CompletedAt for terminal task states, and use the normalized originTask.Status
when marking errors: check the internal originTask.Status (not dResp.Status) to
decide if the task failed and to determine terminal state (follow the same
terminal-status check used elsewhere, e.g., gemini/adaptor.go), set
openAIVideo.CompletedAt = originTask.UpdatedAt only when originTask.Status is a
terminal state (SUCCESS/FAILURE), and capture/return any error from
common.Marshal(openAIVideo) instead of discarding it.
| case constant.ChannelTypeDoubaoVideo, constant.ChannelTypeVolcEngine: | ||
| return &taskdoubao.TaskAdaptor{} |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# 1) Where is GetTaskAdaptor used (which routes/endpoints)?
echo "=== GetTaskAdaptor usage ==="
rg -n 'GetTaskAdaptor\s*\(' -S
# 2) How is platform/channel_type set for task/video flows?
echo -e "\n=== Channel type / Platform assignments ==="
rg -n 'Set\("channel_type"|GetInt\("channel_type"\)|Set\("platform"|GetString\("platform"\)' -S
# 3) Confirm whether VolcEngine task routes exist and what they expect.
echo -e "\n=== VolcEngine channel/API type references ==="
rg -n 'ChannelTypeVolcEngine|APITypeVolcEngine' -S
# 4) Check the full context of the relay_adaptor.go file around line 151
echo -e "\n=== relay_adaptor.go context (lines 140-160) ==="
sed -n '140,160p' relay/relay_adaptor.goRepository: QuantumNous/new-api
Length of output: 3230
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check what "seedream" is and how it's handled
echo "=== Seedream references ==="
rg -n 'seedream' -i
# Find all task types/models associated with ChannelTypeVolcEngine
echo -e "\n=== VolcEngine channel context ==="
rg -B5 -A5 'ChannelTypeVolcEngine' relay/relay_adaptor.go controller/channel-test.go
# Check if taskdoubao handles seedream or only Doubao video
echo -e "\n=== Taskdoubao adaptor ==="
head -50 relay/channel/doubao/adaptor.go | grep -i 'seedream\|func\|adaptor' || echo "Checking adaptor structure..."
# Look for task type mappings to understand adaptor selection logic
echo -e "\n=== Task type to adaptor mapping ==="
rg -n 'GetTaskAdaptor.*case' relay/relay_adaptor.go -A30Repository: QuantumNous/new-api
Length of output: 1968
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find doubao adaptor files
echo "=== Doubao adaptor files ==="
find . -type f -name "*doubao*" | grep -E '\.(go|ts|js)$'
# Check the complete relay_adaptor.go switch statement
echo -e "\n=== Full GetTaskAdaptor function (lines 132-165) ==="
sed -n '132,165p' relay/relay_adaptor.go
# Check what task adaptors are available
echo -e "\n=== Available task adaptors ==="
ls -la relay/channel/ | grep -E '^d'
# Search for how image generation tasks are handled
echo -e "\n=== Image generation task handling ==="
rg -n 'image.*generation|images/generations' -iRepository: QuantumNous/new-api
Length of output: 91
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check relay_adaptor.go imports and full function
echo "=== relay_adaptor.go (first 180 lines) ==="
head -180 relay/relay_adaptor.go
# List all task channel adaptors
echo -e "\n=== relay/channel directory structure ==="
find relay/channel -maxdepth 1 -type d | sort
# Search for taskdoubao package/import definition
echo -e "\n=== taskdoubao references and imports ==="
rg -n 'taskdoubao\|import.*doubao' -B2 -A2Repository: QuantumNous/new-api
Length of output: 6832
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check if there's a volcengine task adaptor that should be used
echo "=== Looking for volcengine task adaptor ==="
find relay/channel -name "*volcengine*" -o -name "*volc*"
# Check what task types volcengine has in its constants
echo -e "\n=== VolcEngine models in constants ==="
cat relay/channel/volcengine/constants.go | head -30Repository: QuantumNous/new-api
Length of output: 572
VolcEngine routes multiple task types—seedream (image) and seedance (video)—but this mapping misroutes them all to the Doubao video task adaptor.
VolcEngine's model list includes seedream (image generation) and seedance (video generation) in addition to Doubao chat models. Routing all ChannelTypeVolcEngine to taskdoubao.TaskAdaptor at line 151 will misroute image and video generation tasks to the wrong adaptor. Create a dedicated VolcEngine task adaptor or introduce a more specific channel type (e.g., ChannelTypeVolcEngineDoubaoChat) for the Doubao-only subset.
|
按tokens计费吗？ |
是的, 配置倍率即按token计费 |
4 participants
支持模型: doubao-seedance-1-5-pro-251215 (相当于即将推出的梦视频3.5)
Seedance 1.5 pro，可生成音画高精同步的视频内容。支持多人多语言对白，全面覆盖环境音、动作音、合成音、乐器音、背景音及人声，支持首尾帧，实现影视级叙事效果，满足影视、漫剧、电商及广告领域的高阶创作需求.
官方文档: https://console.volcengine.com/ark/region:ark+cn-beijing/model/detail?Id=doubao-seedance-1-5-pro
支持格式: openai 视频生成, 视频编辑
请求示例:
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.