Repository navigation
feat(video): first/last frame for Seedance 2.0 - #2699
Merged
Merged
Conversation
Enable image-to-video first/last frame conditioning for ByteDance Seedance 2.0 (and 2.0 Fast). The playground hid the frame inputs behind supportsVideoFrameInput, and the gateway only forwarded the first frame to ByteDance. - gateway: forward last_frame as a role:"last_frame" content entry and allow inputMode "frames" for Seedance 2.0 mappings - playground: expose First/Last frame pickers for Seedance 2.0 - add a ByteDance video mock handler plus forwarding/rejection tests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Warning Review limit reached
More reviews will be available in 6 minutes and 8 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
WalkthroughAdds first/last frame input support for ByteDance Seedance 2.0 by introducing a model helper, gating constraints, and extending request construction with ChangesSeedance 2.0 Frame Input Support
Video Content Delivery
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/playground/src/lib/video-gen.ts`:
- Around line 105-107: The code at line 105-107 in the isSeedance2ReferenceModel
check enables frame support for ByteDance, but the mappingSupportsVideoRequest
function (around line 230 onwards) still rejects ByteDance in frame mode,
creating an inconsistency. Update the mappingSupportsVideoRequest function to
remove or modify the logic that currently filters out ByteDance when frame mode
is enabled, ensuring that ByteDance models can pass frame option validation just
as the capability detection at line 105-107 indicates they should.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 99a854ee-687e-42fb-a968-4c4d01493aba
📒 Files selected for processing (5)
apps/gateway/src/test-utils/mock-openai-server.tsapps/gateway/src/videos/videos.spec.tsapps/gateway/src/videos/videos.tsapps/playground/src/lib/video-gen.spec.tsapps/playground/src/lib/video-gen.ts
Add a "First/last frame inputs" subsection to the video generation docs covering the image/last_frame fields, provider support (Seedance 2.0 specifically), rules, and a Seedance 2.0 example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/features/video-generation.mdx`:
- Line 72: The documentation in the "Not combinable with references" section on
line 72 of video-generation.mdx lists the non-combinable reference inputs as
`reference_images`, `reference_videos`, and `reference_audios`, but omits
`input_reference` which the API validation also rejects when combined with frame
mode inputs. Add `input_reference` to the list of reference input parameters
that cannot be combined with frame inputs (`image`, `last_frame`) to align the
documentation with the actual API behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: c2c14190-809a-4f23-883d-78c2689df2ab
📒 Files selected for processing (1)
apps/docs/content/features/video-generation.mdx
The playground polled video status via the session/org-authorized API app but loaded the actual content through the project-scoped gateway API key. When the active playground key's project differed from the project that owns the job (e.g. after an org/project switch), the gateway returned 404 "Video not found" and the player showed "Video unavailable" even though the job was completed with a valid content URL. - api: video status route now returns a keyless signed content URL (buildSignedGatewayVideoLogContentUrl) scoped by the same org-access check it already performs - playground: content proxy resolves that signed URL via the session-authorized API instead of the project-scoped gateway key, so playback works for any video the user can access and history URLs stay durable Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/playground/src/app/api/video/`[videoId]/content/route.ts:
- Around line 40-48: Both external fetch calls in the route handler lack
timeouts which could cause requests to hang indefinitely. At the statusResponse
fetch call (line 40-48) to apiBackendUrl and the fetch call to sourceUrl (line
70-75), add timeout protection using AbortController. Create an AbortController
for each fetch, set a timeout (e.g., 30 seconds) using setTimeout to call
controller.abort(), pass the signal property from the controller in the fetch
options, and handle potential AbortError exceptions to provide graceful timeout
failures instead of indefinite hangs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 661b4b61-8292-47dc-af2f-ce21c6853a8b
📒 Files selected for processing (2)
apps/api/src/routes/video.tsapps/playground/src/app/api/video/[videoId]/content/route.ts
Comment on lines
40
to
48
| const statusResponse = await fetch( | ||
| `${apiBackendUrl}/video/${encodeURIComponent(videoId)}`, | ||
| { | ||
| headers: { | ||
| Authorization: `Bearer ${apiKey}`, | ||
| "x-source": "chat.llmgateway.io", | ||
| ...(rangeHeader ? { Range: rangeHeader } : {}), | ||
| Cookie: cookieHeader, | ||
| }, | ||
| cache: "no-store", | ||
| }, | ||
| ); |
Contributor
There was a problem hiding this comment.
Add timeouts to external fetch calls to prevent hanging requests.
Both fetch calls to apiBackendUrl (line 40) and sourceUrl (line 70) lack timeouts. If either backend is slow or unresponsive, the request could hang indefinitely, degrading availability.
🛡️ Proposed fix using AbortController
+const FETCH_TIMEOUT_MS = 30_000;
+
// Resolve the signed, keyless content URL through the session-authorized API
// (org-scoped access) rather than the project-scoped gateway API key, so
// playback works for any video the user can access regardless of which
// playground API key is currently active.
const { apiBackendUrl } = getConfig();
+const statusAbort = AbortSignal.timeout(FETCH_TIMEOUT_MS);
const statusResponse = await fetch(
`${apiBackendUrl}/video/${encodeURIComponent(videoId)}`,
{
headers: {
Cookie: cookieHeader,
},
cache: "no-store",
+ signal: statusAbort,
},
); const rangeHeader = req.headers.get("Range");
+const contentAbort = AbortSignal.timeout(FETCH_TIMEOUT_MS);
const response = await fetch(sourceUrl, {
headers: {
...(rangeHeader ? { Range: rangeHeader } : {}),
},
cache: "no-store",
+ signal: contentAbort,
});Also applies to: 70-75
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/playground/src/app/api/video/`[videoId]/content/route.ts around lines 40
- 48, Both external fetch calls in the route handler lack timeouts which could
cause requests to hang indefinitely. At the statusResponse fetch call (line
40-48) to apiBackendUrl and the fetch call to sourceUrl (line 70-75), add
timeout protection using AbortController. Create an AbortController for each
fetch, set a timeout (e.g., 30 seconds) using setTimeout to call
controller.abort(), pass the signal property from the controller in the fetch
options, and handle potential AbortError exceptions to provide graceful timeout
failures instead of indefinite hangs.
- video-gen: allow Seedance 2.0 (and 2.0 Fast) through frame-mode size/duration filtering in mappingSupportsVideoRequest, matching supportsVideoFrameInput so the selectors aren't emptied once a frame is added; non-2.0 bytedance models stay rejected - content proxy: add a 10s timeout to the status fetch (504 on timeout); leave the streaming fetch untimed so large downloads aren't cut off - docs: include input_reference in the frame/reference non-combinable list - tests: cover frame-mode size/duration selection for Seedance 2.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
steebchen
enabled auto-merge
June 16, 2026 17:07
steebchen
added this pull request to the merge queue
Jun 16, 2026
steebchen
added a commit
that referenced
this pull request
Jun 16, 2026
## Summary Two related changes, both surfaced while testing ByteDance Seedance 2.0 video generation in the playground. ### 1. First/last frame inputs for Seedance 2.0 ByteDance Seedance 2.0 supports first/last frame conditioning for image-to-video, but the gateway and playground didn't expose it: - The playground hid the **First/Last frame** pickers behind `supportsVideoFrameInput()`, which only returned `true` for Veo / MiniMax / Grok models. - The gateway's `createBytedanceVideoJob()` only forwarded the **first** frame — a `last_frame` was silently dropped, and frame inputs were rejected for ByteDance by the constraint checker. Changes: - **Gateway** (`videos.ts`): forward `last_frame` as a `role: "last_frame"` content entry; allow `inputMode === "frames"` for Seedance 2.0 mappings (non-2.0 ByteDance models still rejected with a clear message); renamed `isBytedanceReferenceModel` → `isBytedanceSeedance2Model`. - **Playground** (`video-gen.ts`): `supportsVideoFrameInput()` returns `true` for Seedance 2.0. - **Docs** (`video-generation.mdx`): new **First/last frame inputs** section with a provider table (Seedance 2.0 called out), rules, and example. - **Tests**: ByteDance video mock handler + forwarding/rejection tests; playground capability test. ### 2. Fix "Video unavailable" for cross-project video playback After generating a video, the playground could show **"Video unavailable"** even though the job completed successfully with a valid content URL. Root cause: the playground polls video **status** via the session/org-authorized API app, but loaded the **content** through the project-scoped gateway API key. When the active playground key's project differed from the project that owns the job (e.g. after an org/project switch, or a rotated auto-generated key), the gateway returned `404 "Video not found"`. Verified against a real completed Seedance 2.0 job: the gateway returned the full video to the owning key (200) but 404 to a different project's key — exactly the playground's symptom. Changes: - **API** (`routes/video.ts`): the video status route now returns a keyless signed content URL (`buildSignedGatewayVideoLogContentUrl`), scoped by the same org-access check it already performs. - **Playground** (`[videoId]/content/route.ts`): the content proxy resolves that signed URL via the session-authorized API instead of the project-scoped gateway key. Playback now works for any video the user can access, and stored history URLs stay durable (each load re-resolves a fresh signed URL). ## Verification - `pnpm format` ✅ · `pnpm build` ✅ · docs build ✅ - Gateway video spec ✅ 30/30 · playground video-gen spec ✅ 13/13 - End-to-end against a real completed job: API-route-produced signed URL streams the full 8.9 MB video keyless (HTTP 200); confirmed the gateway 404s for a wrong-project key (the original bug) and 200s for the owning key. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added support for Seedance 2.0 `image`/`last_frame` inputs using the Bytedance provider. * Video job status responses for completed videos can now include `content` (signed playback URLs). * **Bug Fixes** * Enforced frame/reference validation rules and improved rejection messaging for incompatible model/provider combinations. * **Tests** * Added/extended test coverage for Seedance 2.0 Bytedance frame requests, compatibility detection, and OpenAI mock task generation. * **Documentation** * Updated `POST /v1/videos` supported fields and added first/last frame interpolation guidance and constraints. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 17, 2026
This was referenced Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related changes, both surfaced while testing ByteDance Seedance 2.0 video generation in the playground.
1. First/last frame inputs for Seedance 2.0
ByteDance Seedance 2.0 supports first/last frame conditioning for image-to-video, but the gateway and playground didn't expose it:
supportsVideoFrameInput(), which only returnedtruefor Veo / MiniMax / Grok models.createBytedanceVideoJob()only forwarded the first frame — alast_framewas silently dropped, and frame inputs were rejected for ByteDance by the constraint checker.Changes:
videos.ts): forwardlast_frameas arole: "last_frame"content entry; allowinputMode === "frames"for Seedance 2.0 mappings (non-2.0 ByteDance models still rejected with a clear message); renamedisBytedanceReferenceModel→isBytedanceSeedance2Model.video-gen.ts):supportsVideoFrameInput()returnstruefor Seedance 2.0.video-generation.mdx): new First/last frame inputs section with a provider table (Seedance 2.0 called out), rules, and example.2. Fix "Video unavailable" for cross-project video playback
After generating a video, the playground could show "Video unavailable" even though the job completed successfully with a valid content URL. Root cause: the playground polls video status via the session/org-authorized API app, but loaded the content through the project-scoped gateway API key. When the active playground key's project differed from the project that owns the job (e.g. after an org/project switch, or a rotated auto-generated key), the gateway returned
404 "Video not found".Verified against a real completed Seedance 2.0 job: the gateway returned the full video to the owning key (200) but 404 to a different project's key — exactly the playground's symptom.
Changes:
routes/video.ts): the video status route now returns a keyless signed content URL (buildSignedGatewayVideoLogContentUrl), scoped by the same org-access check it already performs.[videoId]/content/route.ts): the content proxy resolves that signed URL via the session-authorized API instead of the project-scoped gateway key. Playback now works for any video the user can access, and stored history URLs stay durable (each load re-resolves a fresh signed URL).Verification
pnpm format✅ ·pnpm build✅ · docs build ✅🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
image/last_frameinputs using the Bytedance provider.content(signed playback URLs).POST /v1/videossupported fields and added first/last frame interpolation guidance and constraints.