feat: 支持即梦视频渠道 - #1274
Conversation
WalkthroughThis update introduces support for the "Jimeng" video generation platform across the backend and frontend. It adds new constants, relay modes, and a dedicated Jimeng task adaptor. The middleware and controller logic are extended to handle Jimeng-specific requests, and the frontend UI is updated to recognize and display the new platform. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Frontend
participant Middleware
participant Controller
participant Relay
participant JimengAdaptor
User->>Frontend: Submit video generation request (model: jimeng-*)
Frontend->>Middleware: POST /v1/video/generations (model: jimeng-*)
Middleware->>Controller: Route as Jimeng platform
Controller->>Relay: UpdateTaskByPlatform(platform=Jimeng)
Relay->>JimengAdaptor: Submit/Fetch task (signed request)
JimengAdaptor->>Relay: Return parsed TaskInfo
Relay->>Controller: TaskInfo with status/progress/result
Controller->>Frontend: Respond with task status/result
Frontend->>User: Show Jimeng task info
Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (17)
🚧 Files skipped from review as they are similar to previous changes (14)
🧰 Additional context used🧠 Learnings (1)relay/channel/task/kling/adaptor.go (2)🪛 Gitleaks (8.26.0)relay/channel/task/jimeng/adaptor.go179-179: Detected a Generic API Key, potentially exposing access to various services and sensitive operations. (generic-api-key) 🔇 Additional comments (10)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
middleware/distributor.go (1)
183-189: Consider a more scalable approach for platform detection.Currently, the code assumes all non-"jimeng" prefixed models belong to Kling platform. This approach won't scale well when more video platforms are added.
Consider using a map or configuration-based approach to determine platform from model prefix:
var modelPrefixToPlatform = map[string]string{ "jimeng": string(constant.TaskPlatformJimeng), "kling": string(constant.TaskPlatformKling), // Add more platforms here } // Then use it to determine platform platform := string(constant.TaskPlatformKling) // default for prefix, plat := range modelPrefixToPlatform { if strings.HasPrefix(strings.ToLower(modelRequest.Model), prefix) { platform = plat break } }relay/channel/task/jimeng/adaptor.go (2)
344-344: Fix variable name typo.- medaBytes, err := json.Marshal(metadata) + metaBytes, err := json.Marshal(metadata)
348-348: Update variable reference to match the corrected name.- err = json.Unmarshal(medaBytes, &r) + err = json.Unmarshal(metaBytes, &r)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (17)
common/constants.go(2 hunks)constant/task.go(1 hunks)controller/channel-test.go(1 hunks)controller/task.go(1 hunks)controller/task_video.go(4 hunks)middleware/distributor.go(1 hunks)relay/channel/adapter.go(1 hunks)relay/channel/task/jimeng/adaptor.go(1 hunks)relay/channel/task/kling/adaptor.go(6 hunks)relay/channel/task/suno/adaptor.go(1 hunks)relay/common/relay_info.go(1 hunks)relay/constant/relay_mode.go(2 hunks)relay/relay_adaptor.go(2 hunks)relay/relay_task.go(1 hunks)web/src/components/table/TaskLogsTable.js(2 hunks)web/src/constants/channel.constants.js(1 hunks)web/src/i18n/locales/en.json(1 hunks)
🧰 Additional context used
🪛 Gitleaks (8.26.0)
relay/channel/task/jimeng/adaptor.go
179-179: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🔇 Additional comments (23)
common/constants.go (2)
245-245: LGTM! Channel type constant follows established pattern.The new
ChannelTypeJimeng = 51constant follows the existing sequential numbering pattern and is appropriately placed.
302-302: LGTM! Base URL configuration is appropriate.The base URL
https://visual.volcengineapi.comat index 51 correctly corresponds to the new channel type and appears to be ByteDance's visual AI service endpoint, which is appropriate for video generation functionality.web/src/i18n/locales/en.json (1)
803-803: LGTM! Translation entry is appropriate.The English translation "Jimeng" for "即梦" is simple and direct, providing proper localization support for the new video platform.
constant/task.go (1)
9-9: LGTM! Task platform constant follows established pattern.The new
TaskPlatformJimengconstant with value"jimeng"follows the existing naming convention and will provide proper platform identification for Jimeng tasks.web/src/constants/channel.constants.js (1)
133-137: LGTM! Channel option configuration is consistent.The new channel option follows the established structure with:
- Correct value
51matching the backend constant- Appropriate color choice
'blue'- Proper Chinese label
'即梦'that will be translated via i18ncontroller/channel-test.go (1)
46-48: LGTM! Test exclusion is appropriate for video generation channel.The test exclusion for
ChannelTypeJimengfollows the same pattern as other specialized channels (Midjourney, Suno, Kling) that cannot be tested with standard chat completion requests. This is the correct approach for video generation services.relay/relay_adaptor.go (2)
25-25: LGTM: Import addition follows established pattern.The jimeng package import is correctly placed alphabetically with other task adaptor imports.
108-109: LGTM: Adaptor selection follows established pattern.The new case for
TaskPlatformJimengcorrectly returns ajimeng.TaskAdaptor{}instance, consistent with the existing Suno and Kling implementations.relay/relay_task.go (1)
234-234: Verify that parameter name change doesn't break existing API consumers.The parameter name changed from
"id"to"task_id". While this may align with Jimeng platform conventions, it could be a breaking change for existing clients that expect the"id"parameter in the URL path.#!/bin/bash # Description: Check for existing route definitions and API consumers that might expect "id" parameter # Expected: Find route definitions and verify parameter consistency # Search for route definitions that might use this function rg -A 3 -B 3 "videoFetchByIDRespBodyBuilder|/task.*:id|/task.*:task_id" # Look for any documentation or tests that reference the parameter name rg -A 2 -B 2 "Param.*id.*|task.*id"web/src/components/table/TaskLogsTable.js (3)
225-226: LGTM: Function parameter rename improves clarity.Renaming the parameter from
typetoplatformbetter reflects the actual data being passed and makes the function more semantically clear.
235-235: Good color differentiation for Kling platform.Changing the Kling platform color from 'blue' to 'orange' provides better visual distinction from other platforms, especially with the new Jimeng platform using purple.
239-244: LGTM: Jimeng platform UI integration follows established pattern.The new Jimeng case correctly uses:
- Purple color for visual distinction
- Video icon consistent with video generation purpose
- Same tag structure as other platforms
relay/channel/adapter.go (1)
48-48: LGTM: Interface method improves type safety and structure.The new
ParseTaskResultmethod is a good improvement over the previous approach:
- Input change from
map[string]anyto[]byteprovides better type safety- Output change from
stringto*TaskInfoenables structured task information- More consistent with other parsing methods in the codebase
#!/bin/bash # Description: Verify all TaskAdaptor implementations have been updated to implement ParseTaskResult # Expected: All implementations should have ParseTaskResult method, none should have ParseResultUrl # Find all TaskAdaptor implementations ast-grep --pattern 'type $_ struct { $$$ } func ($_ *$_) $$$($$$) { $$$ }' # Search for any remaining ParseResultUrl methods that should be removed rg "ParseResultUrl" # Verify ParseTaskResult implementations exist rg -A 5 "func.*ParseTaskResult"relay/constant/relay_mode.go (1)
44-46: LGTM!The new constants follow the established naming pattern and integrate well with the existing relay mode enumeration.
controller/task.go (1)
77-78: Excellent refactoring!Combining the Kling and Jimeng platform cases eliminates code duplication and makes the platform handling more generic. The platform parameter is properly passed to
UpdateVideoTaskAll.relay/common/relay_info.go (1)
318-336: Well-structured data models!The
TaskSubmitReqandTaskInfostructs provide good type safety and clear interfaces for task operations. The JSON tags and optional fields are properly configured.Consider adding documentation comments to clarify the purpose and expected values:
+// TaskSubmitReq represents a video generation task submission request type TaskSubmitReq struct { Prompt string `json:"prompt"` Model string `json:"model,omitempty"` Mode string `json:"mode,omitempty"` Image string `json:"image,omitempty"` Size string `json:"size,omitempty"` Duration int `json:"duration,omitempty"` Metadata map[string]interface{} `json:"metadata,omitempty"` } +// TaskInfo represents the status and result of a video generation task type TaskInfo struct { + // Code indicates the response status (0 for success, non-zero for errors) Code int `json:"code"` TaskID string `json:"task_id"` Status string `json:"status"` Reason string `json:"reason,omitempty"` Url string `json:"url,omitempty"` Progress string `json:"progress,omitempty"` }controller/task_video.go (3)
64-66: Why are these error checks commented out?The status code check and error code validation are commented out. This could allow failed requests to proceed without proper error handling.
Are these checks handled elsewhere in the adaptor implementations, or should they be restored to ensure proper error handling?
Also applies to: 77-79
15-51: Great improvements to platform flexibility!The addition of the platform parameter and dynamic adaptor selection makes the code more extensible and maintainable. The error handling with proper context is also well implemented.
73-129: Excellent refactoring with type-safe parsing!The switch to
ParseTaskResultmethod and explicit status handling significantly improves code clarity and type safety. The detailed status progression (10% → 20% → 30% → 100%) provides good user feedback.relay/channel/task/kling/adaptor.go (3)
50-67: LGTM! Well-structured response payload.The expanded
responsePayloadstruct provides comprehensive type safety for all response fields including task status, result videos, and timestamps.
109-109: Good unification of context keys.Changing from
"kling_request"to"task_request"aligns with the broader refactoring to support multiple platforms.Also applies to: 134-134
297-326: Robust task result parsing implementation.The new
ParseTaskResultmethod properly:
- Handles JSON unmarshaling with error wrapping
- Maps Kling status strings to internal enums
- Extracts the first video URL if available
- Returns a detailed error for unknown statuses
relay/channel/task/jimeng/adaptor.go (1)
217-319: Well-implemented HMAC-SHA256 signing logic.The request signing implementation correctly follows the standard HMAC-SHA256 authentication pattern with proper canonical request construction, header sorting, and signature generation.
| func (a *TaskAdaptor) ParseTaskResult([]byte) (*relaycommon.TaskInfo, error) { | ||
| return nil, fmt.Errorf("not implement") // todo implement this method if needed | ||
| } |
There was a problem hiding this comment.
🧩 Verification correct
Implement or gracefully handle unimplemented ParseTaskResult method.
The current implementation returns an error with "not implement", which could cause runtime failures when this method is called. Consider either:
- Implementing the method to parse Suno task results into TaskInfo structure
- Ensuring calling code gracefully handles this error case
🏁 Script executed:
#!/bin/bash
# Description: Check if ParseTaskResult is actually called for Suno platform
# Expected: Find usages and verify error handling
# Search for calls to ParseTaskResult method
rg -A 5 -B 5 "ParseTaskResult"
# Look for task result parsing logic in controllers
rg -A 10 -B 5 "suno.*task.*result|task.*result.*suno"Length of output: 3364
Implement or gracefully handle unimplemented ParseTaskResult method.
The current implementation returns an error with "not implement", which could cause runtime failures when this method is called. Consider either:
- Implementing the method to parse Suno task results into TaskInfo structure
- Ensuring calling code gracefully handles this error case
#!/bin/bash
# Description: Check if ParseTaskResult is actually called for Suno platform
# Expected: Find usages and verify error handling
# Search for calls to ParseTaskResult method
rg -A 5 -B 5 "ParseTaskResult"
# Look for task result parsing logic in controllers
rg -A 10 -B 5 "suno.*task.*result|task.*result.*suno"🤖 Prompt for AI Agents
In relay/channel/task/suno/adaptor.go around lines 25 to 27, the ParseTaskResult
method currently returns a generic "not implement" error which may cause runtime
failures if called. You should either implement this method to correctly parse
the Suno task result bytes into a relaycommon.TaskInfo object or ensure that all
calling code properly checks and handles this error case to avoid crashes.
Review the codebase to confirm if this method is invoked and handle accordingly.
| func Path2RelayJimeng(method, path string) int { | ||
| relayMode := RelayModeUnknown | ||
| if method == http.MethodPost && strings.HasSuffix(path, "/video/generations") { | ||
| relayMode = RelayModeJimengSubmit | ||
| } else if method == http.MethodGet && strings.Contains(path, "/video/generations/") { | ||
| relayMode = RelayModeJimengFetchByID | ||
| } | ||
| return relayMode | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Consider refactoring to eliminate code duplication.
The Path2RelayJimeng function is identical to Path2RelayKling except for the return values. This violates the DRY principle.
Consider creating a generic helper function:
+func pathToVideoRelayMode(method, path string, submitMode, fetchMode int) int {
+ relayMode := RelayModeUnknown
+ if method == http.MethodPost && strings.HasSuffix(path, "/video/generations") {
+ relayMode = submitMode
+ } else if method == http.MethodGet && strings.Contains(path, "/video/generations/") {
+ relayMode = fetchMode
+ }
+ return relayMode
+}
func Path2RelayKling(method, path string) int {
- relayMode := RelayModeUnknown
- if method == http.MethodPost && strings.HasSuffix(path, "/video/generations") {
- relayMode = RelayModeKlingSubmit
- } else if method == http.MethodGet && strings.Contains(path, "/video/generations/") {
- relayMode = RelayModeKlingFetchByID
- }
- return relayMode
+ return pathToVideoRelayMode(method, path, RelayModeKlingSubmit, RelayModeKlingFetchByID)
}
func Path2RelayJimeng(method, path string) int {
- relayMode := RelayModeUnknown
- if method == http.MethodPost && strings.HasSuffix(path, "/video/generations") {
- relayMode = RelayModeJimengSubmit
- } else if method == http.MethodGet && strings.Contains(path, "/video/generations/") {
- relayMode = RelayModeJimengFetchByID
- }
- return relayMode
+ return pathToVideoRelayMode(method, path, RelayModeJimengSubmit, RelayModeJimengFetchByID)
}🤖 Prompt for AI Agents
In relay/constant/relay_mode.go around lines 153 to 161, the Path2RelayJimeng
function duplicates logic found in Path2RelayKling except for the specific
return values. Refactor by extracting the shared logic into a generic helper
function that takes parameters for method, path, and the specific relay mode
values to return. Then have Path2RelayJimeng and Path2RelayKling call this
helper with their respective return values to eliminate duplication and adhere
to the DRY principle.
| if strings.HasPrefix(modelRequest.Model, "jimeng") { | ||
| platform = string(constant.TaskPlatformJimeng) |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Consider making the model prefix check case-insensitive.
The current check strings.HasPrefix(modelRequest.Model, "jimeng") is case-sensitive, which might cause issues if users provide model names with different casing.
- if strings.HasPrefix(modelRequest.Model, "jimeng") {
+ if strings.HasPrefix(strings.ToLower(modelRequest.Model), "jimeng") {Also, consider documenting the model naming convention (e.g., "jimeng-*" for Jimeng models) to ensure consistency.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if strings.HasPrefix(modelRequest.Model, "jimeng") { | |
| platform = string(constant.TaskPlatformJimeng) | |
| if strings.HasPrefix(strings.ToLower(modelRequest.Model), "jimeng") { | |
| platform = string(constant.TaskPlatformJimeng) |
🤖 Prompt for AI Agents
In middleware/distributor.go around lines 177 to 178, the model prefix check
using strings.HasPrefix is case-sensitive, which can cause mismatches if the
model name casing varies. Modify the check to be case-insensitive by converting
modelRequest.Model to lowercase before the prefix check. Additionally, add a
comment near this code explaining the model naming convention, such as
"jimeng-*" prefix for Jimeng models, to improve code clarity and consistency.
| if task.FinishTime == 0 { | ||
| task.FinishTime = now | ||
| } | ||
| task.FailReason = taskResult.Url |
There was a problem hiding this comment.
Semantic mismatch: storing URL in FailReason field.
Using the FailReason field to store the success URL is semantically incorrect and could cause confusion.
Consider using a dedicated field or the existing Data field to store the URL:
case model.TaskStatusSuccess:
task.Progress = "100%"
if task.FinishTime == 0 {
task.FinishTime = now
}
- task.FailReason = taskResult.Url
+ // Store URL in Data field or add a dedicated URL field to the Task model
+ task.Data = map[string]interface{}{"url": taskResult.Url}🤖 Prompt for AI Agents
In controller/task_video.go at line 107, the code assigns a URL to the
FailReason field, which is semantically incorrect since FailReason should
describe failure causes. To fix this, store the URL in a more appropriate field
such as a dedicated URL field or the existing Data field instead of FailReason,
ensuring the data matches the field's intended purpose.
| switch resTask.Data.Status { | ||
| case "in_queue": | ||
| taskResult.Status = model.TaskStatusQueued | ||
| taskResult.Progress = "10%" | ||
| case "done": | ||
| taskResult.Status = model.TaskStatusSuccess | ||
| taskResult.Progress = "100%" | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add missing task status mappings.
The method only handles "in_queue" and "done" statuses. Consider adding mappings for other potential statuses like "processing" or "failed" to avoid missing status updates.
switch resTask.Data.Status {
case "in_queue":
taskResult.Status = model.TaskStatusQueued
taskResult.Progress = "10%"
+ case "processing", "in_progress":
+ taskResult.Status = model.TaskStatusInProgress
+ taskResult.Progress = "50%"
+ case "failed", "error":
+ taskResult.Status = model.TaskStatusFailure
+ taskResult.Progress = "100%"
case "done":
taskResult.Status = model.TaskStatusSuccess
taskResult.Progress = "100%"
+ default:
+ // Log unknown status for debugging
+ taskResult.Status = model.TaskStatusQueued
+ taskResult.Progress = "0%"
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| switch resTask.Data.Status { | |
| case "in_queue": | |
| taskResult.Status = model.TaskStatusQueued | |
| taskResult.Progress = "10%" | |
| case "done": | |
| taskResult.Status = model.TaskStatusSuccess | |
| taskResult.Progress = "100%" | |
| } | |
| switch resTask.Data.Status { | |
| case "in_queue": | |
| taskResult.Status = model.TaskStatusQueued | |
| taskResult.Progress = "10%" | |
| case "processing", "in_progress": | |
| taskResult.Status = model.TaskStatusInProgress | |
| taskResult.Progress = "50%" | |
| case "failed", "error": | |
| taskResult.Status = model.TaskStatusFailure | |
| taskResult.Progress = "100%" | |
| case "done": | |
| taskResult.Status = model.TaskStatusSuccess | |
| taskResult.Progress = "100%" | |
| default: | |
| // Log unknown status for debugging | |
| taskResult.Status = model.TaskStatusQueued | |
| taskResult.Progress = "0%" | |
| } |
🤖 Prompt for AI Agents
In relay/channel/task/jimeng/adaptor.go around lines 369 to 376, the switch
statement handling task statuses only covers "in_queue" and "done". To ensure
all task statuses are properly mapped, add cases for other possible statuses
such as "processing" and "failed", assigning appropriate values to
taskResult.Status and taskResult.Progress for each new case.
c19b69f to
05ea0dd
Compare
|
请问即梦是通过 openai接口调用吗还是和可灵一样。 |
不是, 你说的这两个格式是可灵的官方格式 post: body: ``` |
好的 谢谢 |
大佬 还有个问题。即梦的渠道密钥是放火山方舟的 api key吗还是 Secret Access Key，我试了这两个都报400。 |
|
你提了个非常好的问题, 实际上是要两个一起填, 中间用|分隔 |
|
#1322 |
好的 谢谢大佬 |
您好，还想问一下，这个post地址只能生成视频吗。按照生图的参数提交的，也会生成视频，是不是即梦的生图功能暂时不能用。 |
|
是的，这个版本只支持视频， |
需要 谢谢 |
#1363 加了即梦生图功能 @jackwong-hub |
|
大佬您好，今天在测试您这个火山的生图+生视频。 选择渠道为 ”即梦“ 并填入Access Key ID | Secret Access Key 报错没有权限。我认为应该用到API key 但是却不知道该怎么组合填入了。请指点一下。 |
|
我仔细阅读了官方文档和jimeng/adaptor.go代码。解决了我现阶段的问题： 我目前存在的问题是：
|
|
…-jimeng feat: 支持即梦视频渠道
4 participants
Summary by CodeRabbit
New Features
Improvements
Bug Fixes