增加vidu图片生成模型支持

[feitianbubu]

官方文档: https://platform.vidu.cn/docs/reference-to-image
支持opeani格式的图片生成 和 图片编辑 接口
请求示例:

curl http://localhost:30001/v1/images/generations \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
  "model": "viduq2",
  "prompt": "可爱的中国小女孩在花园里玩耍"
}'

Summary by CodeRabbit

• New Features
  • Added Vidu as a new image generation API provider
  • Integrated automatic background task monitoring for asynchronous image processing workflows

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Introduces support for a new Vidu API channel type by adding the APITypeVidu constant, implementing a complete Adaptor with image request handling, task polling, and response conversion, and integrating it into the relay factory.

Changes

Cohort / File(s)	Summary
API Type & Channel Mapping constant/api_type.go, common/api_type.go	Added APITypeVidu constant to API type enumeration and mapped ChannelTypeVidu to APITypeVidu in ChannelType2APIType function
Vidu Adaptor Implementation relay/channel/vidu/adaptor.go	Implemented Adaptor type with methods for initialization, request URL construction, header setup, image request conversion, API request routing, and response handling; stubs for non-image request types return not implemented errors
Vidu Data Transfer Objects relay/channel/vidu/dto.go	Defined ImageRequest, ImageResponse, TaskResultResponse, and ImageResult structs with JSON serialization tags for Vidu API communication
Vidu Image Request Handler relay/channel/vidu/image.go	Added helper functions for request transformation, asynchronous task polling with backoff and timeout, response conversion to OpenAI format, and orchestration handler that manages the complete flow
Adaptor Factory Integration relay/relay_adaptor.go	Extended GetAdaptor factory with case for APITypeVidu returning new vidu.Adaptor instance

Sequence Diagram

sequenceDiagram
    participant Client
    participant Adaptor as Vidu Adaptor
    participant API as Vidu API
    participant Task as Task Polling
    participant Response as Response Handler
    
    Client->>Adaptor: ConvertImageRequest(imageRequest)
    Adaptor->>Adaptor: Transform to Vidu ImageRequest
    
    Adaptor->>Adaptor: Setup headers & URL
    Adaptor->>API: POST image request
    API-->>Adaptor: TaskResultResponse (task_id)
    
    Adaptor->>Task: asyncTaskWait(task_id)
    
    loop Poll until completion
        Task->>API: queryTask(task_id)
        API-->>Task: TaskResultResponse (state)
        Task->>Task: Evaluate state
        alt State: processing
            Task->>Task: Wait with backoff
        else State: success
            Task->>Task: Exit loop
        else State: failed
            Task->>Task: Return error
        end
    end
    
    Task-->>Adaptor: Final TaskResultResponse
    
    Adaptor->>Response: response2OpenAIImage(taskResult)
    Response->>Response: Convert to OpenAI format
    Response-->>Adaptor: OpenAI ImageResponse
    
    Adaptor->>Client: Stream final response

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• feat: 增加即梦生图功能 #1363: Adds new relay channel with parallel pattern including APIType constant, ChannelType2APIType mapping, and GetAdaptor factory extension
• feat: add vidu video channel #1425: Also introduces vidu channel support with overlapping adaptor implementation and integration changes
• add submodel.ai #1915: Follows identical pattern for adding new API/channel type, modifying the same switch statement locations

Suggested reviewers

• creamlike1024
• seefs001

---

🐰 Whiskers twitch with delight
New Vidu channel hops into place,
Requests transform and tasks are polled,
Async magic dances with grace,
Images stream back to embrace! ✨🎨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title '增加vidu图片生成模型支持' (Add Vidu image generation model support) directly describes the main change: introducing support for the Vidu image generation model as shown across the entire changeset.

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

Fix all issues with AI Agents 🤖

In @relay/channel/vidu/image.go:
- Around line 78-108: The polling loop in asyncTaskWait blocks on time.Sleep and
doesn't observe request cancellation; obtain ctx := c.Request.Context(), replace
all time.Sleep calls with select statements that wait on either ctx.Done() or
time.After(duration), and check ctx.Err() after waking to return
context.Canceled (or context.Error()) promptly; also update queryTask to accept
a context (e.g., queryTask(ctx, info, taskID)) and use that context for its HTTP
calls so the query is cancellable—ensure every early return returns when
ctx.Done() is triggered.
- Around line 21-36: In oaiImage2ViduImageRequest, stop ignoring the error from
json.Marshal(request.Extra); capture the marshal error (e.g., extraBytes, err :=
json.Marshal(request.Extra)), check if err != nil and return nil, err (or wrap
it with context) before attempting json.Unmarshal into req, ensuring you don't
use the blank identifier for the marshal error and that invalid Extra data fails
fast.

🧹 Nitpick comments (2)

relay/channel/vidu/image.go (1)

78-108: Consider externalizing timeout and retry configuration.

The polling logic uses hardcoded values:

• 5-second initial delay (line 79)
• 20 retry attempts (line 81)
• 10-second sleep between retries (lines 86, 100)

These values make it difficult to:

• Test the polling logic with shorter timeouts
• Adjust behavior for different deployment environments
• Fine-tune based on Vidu API performance characteristics

Consider moving these to configuration constants or environment variables.

relay/channel/vidu/adaptor.go (1)

31-36: Consider adding relay mode validation.

The method assumes that non-edit modes are image generation requests without validation. Consider adding a defensive check to ensure RelayMode is an expected value (e.g., RelayModeImagesGenerations).

🔎 Example validation

 func (a *Adaptor) ConvertImageRequest(c *gin.Context, info *relaycommon.RelayInfo, request dto.ImageRequest) (any, error) {
 	if info.RelayMode == relayconstant.RelayModeImagesEdits {
 		return imageEditFromOai(c, info, request)
 	}
+	if info.RelayMode != relayconstant.RelayModeImagesGenerations {
+		return nil, fmt.Errorf("unsupported relay mode: %d", info.RelayMode)
+	}
 	return oaiImage2ViduImageRequest(info, request)
 }

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ad61c0f and 44a5c10.

📒 Files selected for processing (6)

• common/api_type.go
• constant/api_type.go
• relay/channel/vidu/adaptor.go
• relay/channel/vidu/dto.go
• relay/channel/vidu/image.go
• relay/relay_adaptor.go

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-08-26T09:59:00.337Z

Learnt from: Sh1n3zZ
Repo: QuantumNous/new-api PR: 1659
File: relay/relay_task.go:285-305
Timestamp: 2025-08-26T09:59:00.337Z
Learning: In controller/task_video.go, the redactVideoResponseBody function sanitizes video task responses by removing bytesBase64Encoded fields and truncating base64 strings to 256 characters to prevent large binary data from being stored in task.Data.

Applied to files:

• relay/channel/vidu/image.go

🧬 Code graph analysis (3)

relay/relay_adaptor.go (3)

constant/api_type.go (1)

• APITypeVidu (38-38)

relay/channel/vidu/adaptor.go (1)

• Adaptor (17-17)

relay/channel/adapter.go (1)

• Adaptor (15-32)

common/api_type.go (2)

constant/channel.go (1)

• ChannelTypeVidu (52-52)

constant/api_type.go (1)

• APITypeVidu (38-38)

relay/channel/vidu/adaptor.go (5)

relay/common/relay_info.go (1)

• RelayInfo (82-130)

relay/channel/vidu/dto.go (1)

• ImageRequest (3-12)

relay/constant/relay_mode.go (1)

• RelayModeImagesEdits (15-15)

relay/channel/api_request.go (1)

• DoApiRequest (61-90)

types/error.go (1)

• NewAPIError (89-98)

🔇 Additional comments (16)

common/api_type.go (1)

76-77: LGTM! Consistent mapping implementation.

The Vidu channel type mapping follows the established pattern used by all other channel types in this function.

constant/api_type.go (1)

38-38: LGTM! Correct constant placement.

The new APITypeVidu constant is correctly positioned in the iota sequence before the APITypeDummy sentinel value.

relay/relay_adaptor.go (3)

44-44: LGTM! Import added for Vidu adaptor.

---

121-122: LGTM! Adaptor registration follows the pattern.

The Vidu adaptor is correctly instantiated in the GetAdaptor switch, consistent with all other API type handlers.

---

152-153: LGTM! Task adaptor registration is consistent.

The Vidu task adaptor is properly registered in GetTaskAdaptor following the established pattern.

relay/channel/vidu/dto.go (1)

1-41: LGTM! Well-structured DTOs.

The data structures are clearly defined with appropriate JSON tags and omitempty directives for optional fields. These DTOs properly represent the Vidu API request/response contract.

relay/channel/vidu/image.go (3)

142-155: LGTM! Clean response conversion.

The function properly converts Vidu's TaskResultResponse to OpenAI's ImageResponse format, preserving timestamps and including the original response body in metadata.

---

157-207: LGTM! Well-structured handler with appropriate error handling.

The handler orchestrates the complete flow correctly:

• Reads and parses the initial response
• Waits for async task completion
• Handles various error states
• Converts to OpenAI format
• Updates pricing based on image count

Once the context cancellation issue in asyncTaskWait is addressed, this handler will be robust.

---

110-140: HTTP client timeout is conditionally configured—verify common.RelayTimeout is set appropriately.

The HTTP client returned by service.GetHttpClient() has timeout configuration in InitHttpClient(), but only if common.RelayTimeout is non-zero (lines 44-54 in service/http_client.go). If common.RelayTimeout == 0, the client has no timeout, which can cause indefinite hangs. Additionally, queryTask does not use a request-level context with timeout, relying solely on the client-level timeout.

relay/channel/vidu/adaptor.go (7)

1-17: LGTM!

Package structure and imports are appropriate for the Vidu adaptor implementation.

---

19-19: LGTM!

Empty Init is appropriate since this adaptor is stateless.

---

25-29: LGTM!

Header setup correctly configures Content-Type and Authorization with the Token prefix format.

---

38-44: LGTM!

Request and response handling correctly delegates to standard adaptor patterns and the Vidu-specific image handler.

---

46-52: LGTM!

Model list and channel name are correctly configured for the Vidu provider.

---

54-80: LGTM!

Correctly returns errors for unsupported operations since Vidu is an image-only provider.

---

21-23: The code correctly uses a single endpoint for both image generation and image edits. The ConvertImageRequest method branches based on RelayMode to create different request bodies (edits include the Images field with base64-encoded data, while generation does not), but both routes send their requests to the same /ent/v2/reference2image endpoint. This appears to be the intended design of the Vidu API, which differentiates operations by request payload rather than by endpoint URL.

Likely an incorrect or invalid review comment.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Fix silent error ignore when marshaling Extra field.

Line 28 ignores the error from json.Marshal(request.Extra). If marshaling fails, the error is silently discarded, which could mask configuration issues or invalid data in the Extra field.

🔎 Proposed fix

 func oaiImage2ViduImageRequest(info *relaycommon.RelayInfo, request dto.ImageRequest) (*ImageRequest, error) {
 	req := &ImageRequest{
 		Model:  request.Model,
 		Prompt: request.Prompt,
 	}
 
 	if request.Extra != nil {
-		extraBytes, _ := json.Marshal(request.Extra)
+		extraBytes, err := json.Marshal(request.Extra)
+		if err != nil {
+			return nil, fmt.Errorf("failed to marshal Extra field: %w", err)
+		}
 		err := json.Unmarshal(extraBytes, req)
 		if err != nil {
 			return nil, err
 		}
 	}
 
 	return req, nil
 }

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	func oaiImage2ViduImageRequest(info *relaycommon.RelayInfo, request dto.ImageRequest) (*ImageRequest, error) {
	req := &ImageRequest{
	Model: request.Model,
	Prompt: request.Prompt,
	}
	if request.Extra != nil {
	extraBytes, _ := json.Marshal(request.Extra)
	err := json.Unmarshal(extraBytes, req)
	if err != nil {
	return nil, err
	}
	}
	return req, nil
	}
	func oaiImage2ViduImageRequest(info *relaycommon.RelayInfo, request dto.ImageRequest) (*ImageRequest, error) {
	req := &ImageRequest{
	Model: request.Model,
	Prompt: request.Prompt,
	}
	if request.Extra != nil {
	extraBytes, err := json.Marshal(request.Extra)
	if err != nil {
	return nil, fmt.Errorf("failed to marshal Extra field: %w", err)
	}
	err = json.Unmarshal(extraBytes, req)
	if err != nil {
	return nil, err
	}
	}
	return req, nil
	}

🤖 Prompt for AI Agents

In @relay/channel/vidu/image.go around lines 21-36, In
oaiImage2ViduImageRequest, stop ignoring the error from
json.Marshal(request.Extra); capture the marshal error (e.g., extraBytes, err :=
json.Marshal(request.Extra)), check if err != nil and return nil, err (or wrap
it with context) before attempting json.Unmarshal into req, ensuring you don't
use the blank identifier for the marshal error and that invalid Extra data fails
fast.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

Critical: Implement context cancellation in polling loop.

The asyncTaskWait function can block for up to ~3.4 minutes (5s initial + 20×10s retries) without checking if the request context has been cancelled. This creates several problems:

• Resource leaks: Goroutines and HTTP connections may not be released when clients disconnect
• Degraded UX: Users cannot cancel long-running requests
• Server strain: Accumulation of abandoned polling loops during high load

🔎 Proposed fix with context support

-func asyncTaskWait(c *gin.Context, info *relaycommon.RelayInfo, taskID string) (*TaskResultResponse, []byte, error) {
+func asyncTaskWait(c *gin.Context, info *relaycommon.RelayInfo, taskID string) (*TaskResultResponse, []byte, error) {
+	ctx := c.Request.Context()
+	
+	// Initial delay
+	select {
+	case <-time.After(5 * time.Second):
+	case <-ctx.Done():
+		return nil, nil, ctx.Err()
+	}
-	time.Sleep(5 * time.Second)
 
 	for step := 0; step < 20; step++ {
+		// Check for cancellation before each iteration
+		select {
+		case <-ctx.Done():
+			return nil, nil, ctx.Err()
+		default:
+		}
+		
 		logger.LogDebug(c, fmt.Sprintf("vidu image task wait step %d/20", step))
 		rsp, body, err := queryTask(info, taskID)
 		if err != nil {
 			logger.LogWarn(c, "vidu query task error: "+err.Error())
-			time.Sleep(10 * time.Second)
+			select {
+			case <-time.After(10 * time.Second):
+			case <-ctx.Done():
+				return nil, nil, ctx.Err()
+			}
 			continue
 		}
 
 		switch rsp.State {
 		case "success":
 			return rsp, body, nil
 		case "failed":
 			errMsg := "task failed"
 			if rsp.ErrCode != "" {
 				errMsg = rsp.ErrCode
 			}
 			return rsp, body, errors.New(errMsg)
 		case "created", "queueing", "processing":
-			time.Sleep(10 * time.Second)
+			select {
+			case <-time.After(10 * time.Second):
+			case <-ctx.Done():
+				return nil, nil, ctx.Err()
+			}
 			continue
 		default:
 			return rsp, body, fmt.Errorf("unknown task state: %s", rsp.State)
 		}
 	}
 
 	return nil, nil, errors.New("task timeout")
 }

🤖 Prompt for AI Agents

In @relay/channel/vidu/image.go around lines 78-108, The polling loop in
asyncTaskWait blocks on time.Sleep and doesn't observe request cancellation;
obtain ctx := c.Request.Context(), replace all time.Sleep calls with select
statements that wait on either ctx.Done() or time.After(duration), and check
ctx.Err() after waking to return context.Canceled (or context.Error()) promptly;
also update queryTask to accept a context (e.g., queryTask(ctx, info, taskID))
and use that context for its HTTP calls so the query is cancellable—ensure every
early return returns when ctx.Done() is triggered.