feat: add openai video remix endpoint - #2412
Conversation
WalkthroughAdds a "remixGenerate" task action and end-to-end remix flow: new POST /v1/videos/:video_id/remix route, middleware routing, Sora adaptor remix validation and URL, relay task origin-task loading and parameter propagation, and frontend UI/i18n support for remix tasks. Changes
Sequence DiagramsequenceDiagram
participant Client
participant Router as Router (video-router.go)
participant Middleware as Middleware (distributor.go)
participant Relay as RelayTask (relay_task.go)
participant Adaptor as Sora Adaptor (adaptor.go)
participant DB as Database
participant SoraAPI as Sora API
Client->>Router: POST /v1/videos/:video_id/remix
Router->>Middleware: forward request
Middleware->>Middleware: detect "/remix", set RelayModeVideoSubmit, disable channel selection
Middleware->>Relay: call RelayTaskSubmit
Relay->>Relay: detect Remix action from URL, extract video_id → OriginTaskID
Relay->>DB: load origin task by user & OriginTaskID
alt origin task found
Relay->>Relay: derive OriginModelName/platform from origin task
Relay->>DB: if channel differs, load origin channel & api key
Relay->>Relay: update channel context (baseURL,type,apiKey)
Relay->>Relay: extract seconds/size/ratios from origin task data, normalize
end
Relay->>Adaptor: Validate request & build request URL
Adaptor->>Adaptor: validateRemixRequest (JSON prompt required)
Adaptor->>SoraAPI: POST /v1/videos/{originTaskID}/remix with payload
SoraAPI-->>Adaptor: response
Adaptor-->>Relay: relay response/result
Relay->>DB: insert new task record
Relay-->>Client: return success/failure
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45–60 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
relay/relay_task.go (1)
76-100: Consider renaming the local variable to avoid shadowing thechannelpackage import.The variable
channelon line 77 shadows the importedchannelpackage. While Go handles this correctly, it reduces readability and could cause confusion in future maintenance.- if originTask.ChannelId != info.ChannelId { - channel, err := model.GetChannelById(originTask.ChannelId, true) + if originTask.ChannelId != info.ChannelId { + originChannel, err := model.GetChannelById(originTask.ChannelId, true) if err != nil { taskErr = service.TaskErrorWrapperLocal(err, "channel_not_found", http.StatusBadRequest) return } - if channel.Status != common.ChannelStatusEnabled { + if originChannel.Status != common.ChannelStatusEnabled { taskErr = service.TaskErrorWrapperLocal(errors.New("the channel of the origin task is disabled"), "task_channel_disable", http.StatusBadRequest) return } - key, _, newAPIError := channel.GetNextEnabledKey() + key, _, newAPIError := originChannel.GetNextEnabledKey() if newAPIError != nil { taskErr = service.TaskErrorWrapper(newAPIError, "channel_no_available_key", newAPIError.StatusCode) return } common.SetContextKey(c, constant.ContextKeyChannelKey, key) - common.SetContextKey(c, constant.ContextKeyChannelType, channel.Type) - common.SetContextKey(c, constant.ContextKeyChannelBaseUrl, channel.GetBaseURL()) + common.SetContextKey(c, constant.ContextKeyChannelType, originChannel.Type) + common.SetContextKey(c, constant.ContextKeyChannelBaseUrl, originChannel.GetBaseURL()) - info.ChannelBaseUrl = channel.GetBaseURL() + info.ChannelBaseUrl = originChannel.GetBaseURL() info.ChannelId = originTask.ChannelId - info.ChannelType = channel.Type + info.ChannelType = originChannel.Type info.ApiKey = key platform = originTask.Platform }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
constant/task.go(1 hunks)middleware/distributor.go(1 hunks)relay/channel/task/sora/adaptor.go(2 hunks)relay/relay_task.go(1 hunks)router/video-router.go(1 hunks)web/src/components/table/task-logs/TaskLogsColumnDefs.jsx(3 hunks)web/src/constants/common.constant.js(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-26T09:59:00.337Z
Learnt from: Sh1n3zZ
Repo: QuantumNous/new-api PR: 1659
File: relay/relay_task.go:285-305
Timestamp: 2025-08-26T09:59:00.337Z
Learning: In controller/task_video.go, the redactVideoResponseBody function sanitizes video task responses by removing bytesBase64Encoded fields and truncating base64 strings to 256 characters to prevent large binary data from being stored in task.Data.
Applied to files:
router/video-router.go
🧬 Code graph analysis (5)
middleware/distributor.go (1)
relay/constant/relay_mode.go (1)
RelayModeVideoSubmit(44-44)
relay/relay_task.go (9)
constant/task.go (1)
TaskActionRemix(18-18)service/error.go (2)
TaskErrorWrapperLocal(134-138)TaskErrorWrapper(140-157)model/task.go (1)
GetByTaskId(264-277)common/json.go (1)
Unmarshal(9-11)model/channel.go (1)
GetChannelById(338-353)common/constants.go (1)
ChannelStatusEnabled(198-198)common/gin.go (1)
SetContextKey(62-64)constant/context_key.go (3)
ContextKeyChannelKey(38-38)ContextKeyChannelType(27-27)ContextKeyChannelBaseUrl(26-26)types/price_data.go (1)
PriceData(11-27)
router/video-router.go (1)
controller/relay.go (1)
RelayTask(392-435)
relay/channel/task/sora/adaptor.go (5)
common/gin.go (1)
UnmarshalBodyReusable(35-60)service/error.go (1)
TaskErrorWrapperLocal(134-138)relay/channel/adapter.go (1)
TaskAdaptor(34-53)constant/task.go (1)
TaskActionRemix(18-18)relay/common/relay_utils.go (1)
ValidateMultipartDirect(121-188)
web/src/components/table/task-logs/TaskLogsColumnDefs.jsx (1)
web/src/constants/common.constant.js (4)
TASK_ACTION_REMIX_GENERATE(45-45)TASK_ACTION_REMIX_GENERATE(45-45)TASK_ACTION_REFERENCE_GENERATE(44-44)TASK_ACTION_REFERENCE_GENERATE(44-44)
🔇 Additional comments (13)
web/src/constants/common.constant.js (1)
45-45: LGTM!The constant follows the existing naming convention and correctly mirrors the backend
TaskActionRemixvalue.constant/task.go (1)
18-18: LGTM!The constant follows the existing
TaskAction*naming pattern and the value is consistent with the frontend constant.middleware/distributor.go (1)
184-187: LGTM!The remix path detection is correctly placed before the general
/v1/videoscheck to ensure more specific matching takes precedence. SettingshouldSelectChannel = falseis appropriate since remix operations inherit the channel from the origin task.router/video-router.go (1)
17-17: LGTM!The remix route follows OpenAI's API pattern for video operations and is correctly placed within the authenticated router group.
relay/channel/task/sora/adaptor.go (3)
72-83: LGTM!The validation function correctly checks for a required prompt field and uses consistent error wrapping. The
strings.TrimSpacecheck handles whitespace-only prompts appropriately.
85-90: LGTM!The branching logic correctly delegates remix requests to the dedicated validator while preserving the existing multipart validation path for other actions.
92-97: LGTM!The URL construction correctly uses
OriginTaskIDfor the remix endpoint. The upstream validation inrelay_task.goensuresOriginTaskIDis populated before this method is called.web/src/components/table/task-logs/TaskLogsColumnDefs.jsx (3)
42-42: LGTM!Import correctly added alongside other task action constants.
129-134: LGTM!The remix tag rendering follows the established pattern for video generation actions with consistent styling and icon usage.
365-370: LGTM!Correctly extends the video task detection to include remix actions, enabling video preview functionality for successful remix tasks.
relay/relay_task.go (3)
35-47: LGTM!The remix action detection and video_id extraction are correctly implemented with proper validation.
51-75: LGTM!The origin task retrieval and model name derivation logic correctly handles multiple fallback sources with appropriate error handling.
102-121: LGTM!The parameter extraction from the origin task correctly handles the
secondsandsizevalues with appropriate defaults for pricing calculations.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
relay/relay_task.go (2)
77-77: Variablechannelshadows the imported package.The variable name
channelshadows the importedrelay/channelpackage (line 17). While it doesn't cause immediate issues since the package isn't used after this point in the function, this is a code smell that could lead to confusion during future maintenance.- channel, err := model.GetChannelById(originTask.ChannelId, true) + originChannel, err := model.GetChannelById(originTask.ChannelId, true)And update subsequent references on lines 82, 86, 91-93, 96, 98 accordingly.
103-121: Remix parameter extraction logic is reasonable, but unmarshal error should be handled.The pricing parameter extraction for remix tasks handles edge cases well (defaulting seconds to 4, setting size ratios). However, the JSON unmarshal error on line 106 is ignored, which could lead to incorrect pricing if the data is corrupted.
// 使用原始任务的参数 if info.Action == constant.TaskActionRemix { var taskData map[string]interface{} - _ = json.Unmarshal(originTask.Data, &taskData) + if err := json.Unmarshal(originTask.Data, &taskData); err != nil { + common.SysLog(fmt.Sprintf("failed to unmarshal origin task data for remix: %v", err)) + }web/src/i18n/locales/ru.json (1)
558-558: Russian Remix label is fine; stylistic variant optional
"视频Remix": "Видео ремикс"works and matches nearby short labels. If you prefer slightly more idiomatic Russian, consider"Видеоремикс"or"Ремикс видео", but current text is acceptable.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
relay/relay_task.go(1 hunks)web/src/components/table/task-logs/TaskLogsColumnDefs.jsx(3 hunks)web/src/i18n/locales/en.json(1 hunks)web/src/i18n/locales/fr.json(1 hunks)web/src/i18n/locales/ja.json(1 hunks)web/src/i18n/locales/ru.json(1 hunks)web/src/i18n/locales/vi.json(1 hunks)web/src/i18n/locales/zh.json(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
web/src/components/table/task-logs/TaskLogsColumnDefs.jsx (1)
web/src/constants/common.constant.js (4)
TASK_ACTION_REMIX_GENERATE(45-45)TASK_ACTION_REMIX_GENERATE(45-45)TASK_ACTION_REFERENCE_GENERATE(44-44)TASK_ACTION_REFERENCE_GENERATE(44-44)
🔇 Additional comments (10)
web/src/i18n/locales/ja.json (1)
513-513: LGTM!The Japanese translation "動画リミックス" is appropriate for "Video Remix" and follows the existing translation patterns in this file.
web/src/i18n/locales/zh.json (1)
546-546: LGTM!The Chinese translation entry is appropriate and the space between "视频" and "Remix" improves readability in the UI.
relay/relay_task.go (2)
35-47: LGTM on remix path detection and video_id extraction.The path-based detection for remix actions and video_id validation logic is sound.
76-101: Channel switching logic is well-structured.The code properly validates that the origin task's channel is enabled, retrieves a valid API key, and updates all necessary context values. Error handling is comprehensive.
web/src/i18n/locales/vi.json (1)
513-513: VI translation OK; key usage confirmed"Remix video" is acceptable and is used in web/src/components/table/task-logs/TaskLogsColumnDefs.jsx. The key is consistently present across all locale files (zh, fr, ru, ja, en, vi). Consider a later pass to normalize "Video/video" capitalization across VI strings, as the locale shows inconsistent casing (e.g., lines 512, 593, 982, 2338, 2535, 2641).
web/src/i18n/locales/fr.json (1)
554-554: FR translation OK; verified cross‑locale presence and UI usage"Remix vidéo" reads well for a tag/badge. Key is present in all six locales (zh, en, fr, ru, vi, ja) with appropriate translations. UI usage confirmed in
web/src/components/table/task-logs/TaskLogsColumnDefs.jsxwhere it's rendered as a blue tag with icon.web/src/i18n/locales/en.json (1)
551-551: Remix i18n key added correctlyThe new
"视频Remix": "Video remix"entry is consistent with surrounding keys and will localize the Remix tag as expected.web/src/components/table/task-logs/TaskLogsColumnDefs.jsx (3)
38-43: New TASK_ACTION_REMIX_GENERATE import is properly wiredImporting
TASK_ACTION_REMIX_GENERATEalongside the other task action constants keeps this file aligned withcommon.constant.js, and the symbol is used below so there’s no unused-import concern.
91-135: RenderType support for Remix is consistent with existing video tagsThe new
case TASK_ACTION_REMIX_GENERATEreuses the blue circularTagwithSparklesicon and the i18n labelt('视频Remix'), matching the visual pattern of other video generation actions (图生视频,文生视频, etc.). With the new key present in all locales, this should render correctly in the TYPE column.
365-371: Including REMIX_GENERATE inisVideoTasklooks right; confirm backend URL behaviorExtending
isVideoTaskto treatTASK_ACTION_REMIX_GENERATE(andTASK_ACTION_REFERENCE_GENERATE) as video tasks ensures Remix jobs can use the same preview logic as other video generations.One thing to double‑check: the preview link is only shown when
isSuccess && isVideoTask && isUrl, whereisUrlis based onfail_reasonbeing anhttp(s)URL, even though the actual video URL is derived fromtask_id. Please confirm that for Remix tasks the backend still setsfail_reasonto a URL (or you’re OK with that contract); otherwise successful Remix tasks might not show the preview link despite having a valid/v1/videos/{task_id}/contentendpoint.
| var taskData map[string]interface{} | ||
| _ = json.Unmarshal(originTask.Data, &taskData) | ||
| if m, ok := taskData["model"].(string); ok && m != "" { | ||
| info.OriginModelName = m | ||
| platform = originTask.Platform | ||
| } | ||
| } |
There was a problem hiding this comment.
Ignored JSON unmarshal error could lead to silent failures.
If json.Unmarshal fails on originTask.Data, taskData will be an empty map, causing the subsequent field accesses to silently return zero values. This could result in OriginModelName not being set when it should be, affecting billing and routing.
Consider logging the error or handling the failure case:
} else {
var taskData map[string]interface{}
- _ = json.Unmarshal(originTask.Data, &taskData)
+ if err := json.Unmarshal(originTask.Data, &taskData); err != nil {
+ common.SysLog(fmt.Sprintf("failed to unmarshal origin task data: %v", err))
+ }
if m, ok := taskData["model"].(string); ok && m != "" {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| var taskData map[string]interface{} | |
| _ = json.Unmarshal(originTask.Data, &taskData) | |
| if m, ok := taskData["model"].(string); ok && m != "" { | |
| info.OriginModelName = m | |
| platform = originTask.Platform | |
| } | |
| } | |
| } else { | |
| var taskData map[string]interface{} | |
| if err := json.Unmarshal(originTask.Data, &taskData); err != nil { | |
| common.SysLog(fmt.Sprintf("failed to unmarshal origin task data: %v", err)) | |
| } | |
| if m, ok := taskData["model"].(string); ok && m != "" { | |
| info.OriginModelName = m | |
| platform = originTask.Platform | |
| } | |
| } |
🤖 Prompt for AI Agents
In relay/relay_task.go around lines 68-74, json.Unmarshal on originTask.Data is
called and its error is ignored which can lead to silent failures and missing
OriginModelName; change the code to capture the unmarshal error, and handle it
by logging the error with contextual fields (e.g., originTask ID, Platform, raw
payload) and then either continue/skip this originTask or return the error up
(choose consistent behavior with surrounding code), only proceeding to extract
taskData["model"] when unmarshal succeeded. Ensure the log message is
descriptive so downstream billing/routing issues can be diagnosed.
feat: add openai video remix endpoint
3 participants
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.