即梦支持多图生视频 - #1788
Conversation
WalkthroughCentralizes task request validation/binding via new helpers in relay/common, standardizes on relaycommon.TaskSubmitReq across adaptors, and shifts image handling to a list-based Images field with HasImage() checks. Adaptor implementations (jimeng, kling, vidu) now delegate validation to shared helpers and build payloads from the unified request shape. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Client
participant Gin as Gin Handler
participant Utils as relaycommon.Validate*()
participant Ctx as Context
participant Adaptor as TaskAdaptor
participant Provider as Upstream API
Client->>Gin: HTTP POST /task
Gin->>Utils: Bind & Validate (prompt, images)
Utils->>Ctx: store(action, TaskSubmitReq)
Utils-->>Gin: TaskError? (optional)
alt valid
Gin->>Adaptor: BuildRequestBody(ctx)
Adaptor->>Ctx: get task_request, action
Adaptor->>Adaptor: Build payload (uses Images/HasImage)
Adaptor->>Provider: Send request
Provider-->>Adaptor: Response
Adaptor-->>Client: Response
else invalid
Gin-->>Client: Error (invalid_request/...)
end
note over Utils,Adaptor: Action set based on presence of images<br/>(e.g., generate vs text-generate)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks (2 passed, 1 warning)❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Poem
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. ✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
relay/channel/task/jimeng/adaptor.go (1)
314-318: Inconsistent/incorrectReqKey; derive from request instead of hardcoding.You hardcode
jimeng_vgfm_i2v_l20here, butFetchTaskusesjimeng_vgfm_t2v_l20. This will break retrieval. DeriveReqKeyfromreq.Modeland fall back by action (i2v when images exist, else t2v):- r := requestPayload{ - ReqKey: "jimeng_vgfm_i2v_l20", + key := req.Model + if key == "" { + if req.HasImage() { + key = "jimeng_vgfm_i2v_l20" + } else { + key = "jimeng_vgfm_t2v_l20" + } + } + r := requestPayload{ + ReqKey: key, Prompt: req.Prompt, AspectRatio: "16:9", // Default aspect ratio Seed: -1, // Default to random }relay/channel/task/vidu/adaptor.go (1)
189-214: Multi-image not supported; ignoresreq.Images.This defeats the PR goal. Use
req.Images(and fall back toreq.Image) and preserve order:-func (a *TaskAdaptor) convertToRequestPayload(req *relaycommon.TaskSubmitReq) (*requestPayload, error) { - var images []string - if req.Image != "" { - images = []string{req.Image} - } +func (a *TaskAdaptor) convertToRequestPayload(req *relaycommon.TaskSubmitReq) (*requestPayload, error) { + images := append([]string(nil), req.Images...) + if len(images) == 0 && strings.TrimSpace(req.Image) != "" { + images = []string{strings.TrimSpace(req.Image)} + } r := requestPayload{ Model: defaultString(req.Model, "viduq1"), Images: images, Prompt: req.Prompt, Duration: defaultInt(req.Duration, 5), Resolution: defaultString(req.Size, "1080p"), MovementAmplitude: "auto", Bgm: false, }relay/channel/task/kling/adaptor.go (2)
143-151: Action override ignoresimages-only requests.If client sends only
images,body.Image/ImageTailremain empty and you flip to text-generation incorrectly. After payload build, set action based on presence of eitherImageorImageTail:- if body.Image == "" && body.ImageTail == "" { - c.Set("action", constant.TaskActionTextGenerate) - } + if body.Image == "" && body.ImageTail == "" { + c.Set("action", constant.TaskActionTextGenerate) + } else { + c.Set("action", constant.TaskActionGenerate) + }
232-261: Usereq.Images(multi-image) and map toimage/image_tail.Support the new list: first image ->
image, last image (if >1) ->image_tail; keep legacyimageas fallback.func (a *TaskAdaptor) convertToRequestPayload(req *relaycommon.TaskSubmitReq) (*requestPayload, error) { - r := requestPayload{ - Prompt: req.Prompt, - Image: req.Image, + var head, tail string + if n := len(req.Images); n > 0 { + head = strings.TrimSpace(req.Images[0]) + if n > 1 { + tail = strings.TrimSpace(req.Images[n-1]) + } + } else { + head = strings.TrimSpace(req.Image) + } + + r := requestPayload{ + Prompt: req.Prompt, + Image: head, + ImageTail: tail, Mode: defaultString(req.Mode, "std"), Duration: fmt.Sprintf("%d", defaultInt(req.Duration, 5)), AspectRatio: a.getAspectRatio(req.Size), ModelName: req.Model, Model: req.Model, // Keep consistent with model_name, double writing improves compatibility CfgScale: 0.5, StaticMask: "", DynamicMasks: []DynamicMask{}, CameraControl: nil, CallbackUrl: "", ExternalTaskId: "", }
🧹 Nitpick comments (4)
relay/common/relay_info.go (1)
484-488: Field added is good; clarify precedence with legacyimage.Keeping
imagewhile addingimagesis fine for compatibility, but please document precedence (“images” overrides “image”) to avoid ambiguity in downstream adaptors.relay/channel/task/jimeng/adaptor.go (2)
321-327: Robust multi-image handling and scheme detection.
- Mixed inputs (some URLs, some base64) are not handled.
HasPrefix("http")is brittle.Minimal fix: split into URL vs base64 buckets; if both provided, prefer URLs and ignore invalids.
- if req.HasImage() { - if strings.HasPrefix(req.Images[0], "http") { - r.ImageUrls = req.Images - } else { - r.BinaryDataBase64 = req.Images - } - } + if req.HasImage() { + var urls, b64s []string + for _, img := range req.Images { + s := strings.TrimSpace(img) + if strings.HasPrefix(s, "http://") || strings.HasPrefix(s, "https://") { + urls = append(urls, s) + } else if s != "" { + b64s = append(b64s, s) + } + } + if len(urls) > 0 { + r.ImageUrls = urls + } else if len(b64s) > 0 { + r.BinaryDataBase64 = b64s + } + }
328-337: Typos in error messages and unnecessary variable name.Tighten the error texts; use
metaBytesand concise messages.- metadata := req.Metadata - medaBytes, err := json.Marshal(metadata) + metadata := req.Metadata + metaBytes, err := json.Marshal(metadata) if err != nil { - return nil, errors.Wrap(err, "metadata marshal metadata failed") + return nil, errors.Wrap(err, "marshal metadata failed") } - err = json.Unmarshal(medaBytes, &r) + err = json.Unmarshal(metaBytes, &r) if err != nil { - return nil, errors.Wrap(err, "unmarshal metadata failed") + return nil, errors.Wrap(err, "apply metadata override failed") }relay/channel/task/vidu/adaptor.go (1)
99-101: Also set action to generate when images are present.Be explicit to avoid relying on prior validator state:
- if len(body.Images) == 0 { - c.Set("action", constant.TaskActionTextGenerate) - } + if len(body.Images) == 0 { + c.Set("action", constant.TaskActionTextGenerate) + } else { + c.Set("action", constant.TaskActionGenerate) + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
relay/channel/task/jimeng/adaptor.go(2 hunks)relay/channel/task/kling/adaptor.go(3 hunks)relay/channel/task/vidu/adaptor.go(2 hunks)relay/common/relay_info.go(1 hunks)relay/common/relay_utils.go(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
relay/common/relay_info.go (1)
relay/common/relay_utils.go (1)
HasImage(18-20)
relay/common/relay_utils.go (2)
common/gin.go (1)
UnmarshalBodyReusable(30-51)constant/task.go (2)
TaskActionTextGenerate(15-15)TaskActionGenerate(14-14)
relay/channel/task/kling/adaptor.go (3)
relay/common/relay_utils.go (1)
ValidateBasicTaskRequest(67-84)constant/task.go (1)
TaskActionGenerate(14-14)relay/common/relay_info.go (1)
TaskSubmitReq(479-488)
relay/channel/task/vidu/adaptor.go (3)
relay/common/relay_utils.go (1)
ValidateTaskRequestWithImageBinding(105-112)relay/channel/adapter.go (1)
TaskAdaptor(32-51)relay/common/relay_info.go (2)
RelayInfo(74-120)TaskSubmitReq(479-488)
relay/channel/task/jimeng/adaptor.go (2)
relay/common/relay_utils.go (2)
ValidateBasicTaskRequest(67-84)HasImage(18-20)constant/task.go (1)
TaskActionGenerate(14-14)
🔇 Additional comments (9)
relay/common/relay_info.go (1)
490-492: Getter looks good.relay/channel/task/jimeng/adaptor.go (2)
91-92: Centralized validation: LGTM.
164-169: Avoid hardcoding req_key in FetchTask — accept and persist req_key/model from the requestFile: relay/channel/task/jimeng/adaptor.go (lines 164–169)
Hardcoded req_key can mismatch submission vs fetch; read req_key (or model) from body with a fallback and persist it in task metadata so FetchTask does not guess.
- payload := map[string]string{ - "req_key": "jimeng_vgfm_t2v_l20", - "task_id": taskID, - } + reqKey := "jimeng_vgfm_t2v_l20" + if v, ok := body["req_key"].(string); ok && strings.TrimSpace(v) != "" { + reqKey = v + } else if v, ok := body["model"].(string); ok && strings.TrimSpace(v) != "" { + reqKey = v + } + payload := map[string]string{ + "req_key": reqKey, + "task_id": taskID, + }relay/channel/task/vidu/adaptor.go (1)
92-92: Type assertion: ok.But ensure the validator always stores the same type (value vs pointer). Current usage is consistent.
relay/channel/task/kling/adaptor.go (1)
113-115: Centralized validation: LGTM.relay/common/relay_utils.go (4)
14-20: Interfaces are appropriate.
45-53: Error helper: LGTM.
67-84: Basic validator correctly mergesimage→images.Good for compatibility.
86-103: Action decision is fine; relies onHasImage().Once
HasImage/binding are fixed, this stays solid.
| // Use the unified validation method for TaskSubmitReq with image-based action determination | ||
| return relaycommon.ValidateTaskRequestWithImageBinding(c, info) | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Validator path can misclassify image requests when only image is provided.
ValidateTaskRequestWithImageBinding doesn’t merge image -> images, so action may be set to text-generation incorrectly. Either switch to ValidateBasicTaskRequest or fix the validator (see relay/common/relay_utils.go). If you keep this path, ensure convert uses req.Images.
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "CWD: $(pwd)"
echo "----- locate validators -----"
rg -n --hidden -S "ValidateTaskRequestWithImageBinding|ValidateBasicTaskRequest|ValidateTaskRequest" || true
echo
echo "----- locate TaskSubmitReq and image fields -----"
rg -n --hidden -S "type\s+TaskSubmitReq\b|TaskSubmitReq\b|req\.Image\b|req\.Images\b|\bImage\b|\bImages\b" || true
echo
echo "----- show reported file if present -----"
if [ -f "relay/channel/task/vidu/adaptor.go" ]; then
echo "---- relay/channel/task/vidu/adaptor.go ----"
sed -n '1,260p' relay/channel/task/vidu/adaptor.go
else
echo "relay/channel/task/vidu/adaptor.go not found; listing relay/channel/task"
ls -la relay/channel || true
ls -la relay/channel/task || true
fi
echo
echo "----- search for relay/common/relay_utils.go -----"
rg -n --hidden -S "relay/common/relay_utils.go|relay_utils.go" || true
if [ -f "relay/common/relay_utils.go" ]; then
echo "---- relay/common/relay_utils.go ----"
sed -n '1,400p' relay/common/relay_utils.go
fi
echo
echo "----- finished -----"Length of output: 17116
Validator misclassifies single-image requests — merge image->images or switch validator
ValidateTaskRequestWithImageBinding binds into TaskSubmitReq but does not copy TaskSubmitReq.Image -> TaskSubmitReq.Images (ValidateBasicTaskRequest does), so HasImage() can be false when only "image" is provided and info.Action becomes text-generation — causing vidu to pick /text2video while the body contains images.
- Fix (recommended): in relay/common/relay_utils.go, inside ValidateTaskRequestWithImageBinding, add the same compatibility copy before calling ValidateTaskRequestWithImage:
if len(req.Images) == 0 && strings.TrimSpace(req.Image) != "" {
req.Images = []string{req.Image}
} - Alternative: change relay/channel/task/vidu/adaptor.go (ValidateRequestAndSetAction) to use ValidateBasicTaskRequest and handle action explicitly.
Files: relay/common/relay_utils.go (ValidateTaskRequestWithImageBinding), relay/channel/task/vidu/adaptor.go (ValidateRequestAndSetAction).
| func (t TaskSubmitReq) HasImage() bool { | ||
| return len(t.Images) > 0 | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
HasImage() misses legacy image path used by some validators.
ValidateTaskRequestWithImageBinding does not merge image -> images, so HasImage() can return false when only image is provided. Either (a) make HasImage() check both fields, or (b) fix the validator (preferred, see relay/common/relay_utils.go comment). If you prefer fixing here, apply:
-func (t TaskSubmitReq) HasImage() bool {
- return len(t.Images) > 0
-}
+func (t TaskSubmitReq) HasImage() bool {
+ if len(t.Images) > 0 {
+ return true
+ }
+ return strings.TrimSpace(t.Image) != ""
+}🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo "---- locate TaskSubmitReq definition ----"
rg -n --hidden --no-ignore-vcs "type\s+TaskSubmitReq\b" -S || true
echo "---- show files defining TaskSubmitReq ----"
rg -n --hidden --no-ignore-vcs "type\s+TaskSubmitReq\b" -S | cut -d: -f1 | uniq | while read -r f; do
echo "=== $f ==="
sed -n '1,240p' "$f" || true
done
echo "---- relay/common/relay_info.go (lines 460-520) ----"
if [ -f relay/common/relay_info.go ]; then
sed -n '460,520p' relay/common/relay_info.go || true
else
rg -n --hidden --no-ignore-vcs "relay/common/relay_info.go" -S || true
fi
echo "---- HasImage implementations ----"
rg -n --hidden --no-ignore-vcs "func\s+\(t\s+\*?TaskSubmitReq\)\s+HasImage\s*\(" -S || true
echo "---- ValidateTaskRequestWithImageBinding occurrences ----"
rg -n --hidden --no-ignore-vcs "ValidateTaskRequestWithImageBinding" -S || true
echo "---- relay/common/relay_utils.go and image mentions ----"
if [ -f relay/common/relay_utils.go ]; then
sed -n '1,240p' relay/common/relay_utils.go || true
else
rg -n --hidden --no-ignore-vcs "relay_utils\.go" -S || true
fi
echo "---- search for Image / Images fields ----"
rg -n --hidden --no-ignore-vcs "\bImage(s)?\b" -S || trueLength of output: 17505
Merge legacy image into images in ValidateTaskRequestWithImageBinding (preferred)
ValidateTaskRequestWithImageBinding uses c.ShouldBindJSON(&req) but does not copy req.Image → req.Images; TaskSubmitReq.HasImage only checks Images, so requests with only "image" are treated as no-image.
Action: in relay/common/relay_utils.go, inside ValidateTaskRequestWithImageBinding (before calling ValidateTaskRequestWithImage), add the same compatibility block used in ValidateBasicTaskRequest: if len(req.Images) == 0 && strings.TrimSpace(req.Image) != "" { req.Images = []string{req.Image} }.
Alternative: change TaskSubmitReq.HasImage to check both fields (e.g. return len(t.Images) > 0 || strings.TrimSpace(t.Image) != "").
🤖 Prompt for AI Agents
In relay/common/relay_utils.go in the ValidateTaskRequestWithImageBinding
function (insert before the call to ValidateTaskRequestWithImage), merge legacy
req.Image into req.Images by adding the compatibility block: if len(req.Images)
== 0 && strings.TrimSpace(req.Image) != "" { req.Images = []string{req.Image} };
this ensures requests using the old "image" field are treated as having images;
alternatively, you can modify TaskSubmitReq.HasImage to check both Images and
Image, but prefer adding the compatibility block in
ValidateTaskRequestWithImageBinding.
| func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError { | ||
| var req TaskSubmitReq | ||
| if err := c.ShouldBindJSON(&req); err != nil { | ||
| return createTaskError(err, "invalid_request_body", http.StatusBadRequest, false) | ||
| } | ||
|
|
||
| return ValidateTaskRequestWithImage(c, info, req) | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Fix ValidateTaskRequestWithImageBinding to merge image → images and reuse reusable-body unmarshal.
Without this, adaptors using this path (e.g., vidu) misclassify requests and lose images.
-func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError {
- var req TaskSubmitReq
- if err := c.ShouldBindJSON(&req); err != nil {
- return createTaskError(err, "invalid_request_body", http.StatusBadRequest, false)
- }
-
- return ValidateTaskRequestWithImage(c, info, req)
-}
+func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError {
+ var req TaskSubmitReq
+ if err := common.UnmarshalBodyReusable(c, &req); err != nil {
+ return createTaskError(err, "invalid_request", http.StatusBadRequest, true)
+ }
+ if len(req.Images) == 0 && strings.TrimSpace(req.Image) != "" {
+ req.Images = []string{req.Image}
+ }
+ return ValidateTaskRequestWithImage(c, info, req)
+}🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf "=== rg: ValidateTaskRequestWithImageBinding ===\n"
rg -n --hidden --no-ignore "ValidateTaskRequestWithImageBinding" -S || true
printf "\n=== show: relay/common/relay_utils.go (first 240 lines) ===\n"
if [ -f relay/common/relay_utils.go ]; then
sed -n '1,240p' relay/common/relay_utils.go || true
else
printf "relay/common/relay_utils.go not found\n"
fi
printf "\n=== rg: TaskSubmitReq definition ===\n"
rg -n --hidden --no-ignore "type\\s+TaskSubmitReq" -S || true
printf "\n=== rg: occurrences of TaskSubmitReq ===\n"
rg -n --hidden --no-ignore "TaskSubmitReq" -S || true
printf "\n=== rg: UnmarshalBodyReusable ===\n"
rg -n --hidden --no-ignore "UnmarshalBodyReusable" -S || true
printf "\n=== rg: createTaskError definition ===\n"
rg -n --hidden --no-ignore "func\\s+createTaskError" -S || true
printf "\n=== rg: ValidateTaskRequestWithImage definition ===\n"
rg -n --hidden --no-ignore "func\\s+ValidateTaskRequestWithImage" -S || trueLength of output: 7308
Fix ValidateTaskRequestWithImageBinding: use UnmarshalBodyReusable and merge single image → images.
Mirror ValidateBasicTaskRequest (use common.UnmarshalBodyReusable and preserve single-image compatibility) — current ShouldBindJSON loses images for adaptors like vidu.
Location: relay/common/relay_utils.go:105-112
-func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError {
- var req TaskSubmitReq
- if err := c.ShouldBindJSON(&req); err != nil {
- return createTaskError(err, "invalid_request_body", http.StatusBadRequest, false)
- }
-
- return ValidateTaskRequestWithImage(c, info, req)
-}
+func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError {
+ var req TaskSubmitReq
+ if err := common.UnmarshalBodyReusable(c, &req); err != nil {
+ return createTaskError(err, "invalid_request", http.StatusBadRequest, true)
+ }
+ if len(req.Images) == 0 && strings.TrimSpace(req.Image) != "" {
+ req.Images = []string{req.Image}
+ }
+ return ValidateTaskRequestWithImage(c, info, req)
+}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError { | |
| var req TaskSubmitReq | |
| if err := c.ShouldBindJSON(&req); err != nil { | |
| return createTaskError(err, "invalid_request_body", http.StatusBadRequest, false) | |
| } | |
| return ValidateTaskRequestWithImage(c, info, req) | |
| } | |
| func ValidateTaskRequestWithImageBinding(c *gin.Context, info *RelayInfo) *dto.TaskError { | |
| var req TaskSubmitReq | |
| if err := common.UnmarshalBodyReusable(c, &req); err != nil { | |
| return createTaskError(err, "invalid_request", http.StatusBadRequest, true) | |
| } | |
| if len(req.Images) == 0 && strings.TrimSpace(req.Image) != "" { | |
| req.Images = []string{req.Image} | |
| } | |
| return ValidateTaskRequestWithImage(c, info, req) | |
| } |
🤖 Prompt for AI Agents
In relay/common/relay_utils.go around lines 105-112, the function currently uses
c.ShouldBindJSON which drops adaptor-provided images; replace the JSON bind with
common.UnmarshalBodyReusable to parse the request into req, then preserve
single-image compatibility by checking if req.Images is empty and req.Image (or
the single-image field used in DTO) is set — if so, populate req.Images with
that single value; finally call ValidateTaskRequestWithImage(c, info, req) as
before. Ensure error handling mirrors ValidateBasicTaskRequest (return
createTaskError on unmarshal failure) and keep the same return type.
2 participants
即梦多图请求示例:
/v1/video/generationsSummary by CodeRabbit