Skip to content

feat(browser-export): 100% client-side MP4/WebM export via WebCodecs (mediabunny) - #1905

Closed
HackingCorp wants to merge 1 commit into
heygen-com:mainfrom
HackingCorp:feat/browser-export
Closed

feat(browser-export): 100% client-side MP4/WebM export via WebCodecs (mediabunny)#1905
HackingCorp wants to merge 1 commit into
heygen-com:mainfrom
HackingCorp:feat/browser-export

Conversation

@HackingCorp

Copy link
Copy Markdown

Summary

Implements the browser-side export path discussed in #1661: a new @hyperframes/browser-export package that renders a live composition to MP4/WebM entirely in the browser — no server, no FFmpeg, no headless Chrome. It complements @hyperframes/producer (which remains the reference for deterministic, pixel-perfect renders); the target is browser-based editors, template SaaS frontends, and quick exports delivered straight to the user's Downloads folder.

Pairs naturally with @hyperframes/lint/browser (#1749/#1773) for a fully client-side validate → render loop.

How it works

  1. Plan — locate the composition root ([data-composition-id] or #root), read dimensions, resolve duration from the master GSAP timeline in window.__timelines (or options.duration).
  2. Audio — parse <audio>/<video> clip metadata (data-start, data-duration/data-end, data-media-start, data-volume — the producer's contract), decode and mix offline with OfflineAudioContext.
  3. Video — per frame: pause + seek every registered timeline at the quantized frame time (Math.round(t·fps)/fps, the producer's parity contract), await <video> layer seeks, rasterize the root to a canvas via SVG foreignObject (html-to-image, fonts embedded once).
  4. Encode — WebCodecs via mediabunny (avc+aac in MP4, vp9+opus in WebM), finalized into a downloadable Blob. Abortable via AbortSignal, per-phase onProgress.

Package shape

  • Single browser entry; tsup platform: "browser" build — the same compile-time node-free guarantee as @hyperframes/lint/browser.
  • Deps: mediabunny (already used by studio's mediaProbe) + html-to-image.
  • Limitations documented in the README/docs page: WebCodecs support matrix, CORS requirements for foreignObject rasterization, async <video> frame alignment with a 500 ms guard, main-thread rendering.

Testing

  • 26 unit tests on the pure logic (composition discovery, clip parsing, frame timing/quantization, deterministic timeline seeks, codec mapping, filenames) — the browser-only modules (OfflineAudioContext / WebCodecs / canvas) are isolated behind small seams.
  • Typecheck clean; browser build + d.ts generation verified; docs page registered in docs.json and the root build order.

🤖 Generated with Claude Code

https://claude.ai/code/session_018TXJXMe4QmSBnmYoR3brtk

…(mediabunny)

New @hyperframes/browser-export package (discussion in heygen-com#1661):
deterministic quantized GSAP seeks (producer parity contract), SVG
foreignObject rasterization (html-to-image), OfflineAudioContext mix of
data-start/data-duration audio clips, WebCodecs encoding via mediabunny
(avc+aac MP4, vp9+opus WebM). Browser-only tsup build (platform:
"browser" compile-time guarantee), 26 unit tests on the pure logic,
docs page + build-order registration.
HackingCorp added a commit to HackingCorp/ltc-motion that referenced this pull request Jul 3, 2026

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the new package locally and in headless Chrome. The simple video-only browser export does produce a WebM Blob, but I found two correctness issues that can make browser exports differ from HyperFrames' existing runtime/producer semantics.

Audited: packages/browser-export/src/* end-to-end, package/docs wiring, monorepo build, browser smoke for export + audio collection.
Trusting: lockfile dependency resolution details beyond frozen install/build.

Strengths:

  • packages/browser-export/src/frameCapture.ts:26 caches embedded font CSS once instead of re-fetching fonts per frame, which is the right shape for frame-by-frame capture.
  • packages/browser-export/src/exporter.ts:85 keeps frame progress and frame encoding in one deterministic loop, which made the browser smoke straightforward to exercise.

Blockers:

  • packages/browser-export/src/exporter.ts:73prepareAudio scopes audio discovery to plan.root.ownerDocument, so exporting one composition from an editor page mixes/fetches every <audio>/<video> in the whole document, not just media inside the composition root. I reproduced this in Chrome with exportComposition(document.getElementById("root")): a sibling <audio src="/outside.wav"> outside #root was fetched during export. That will leak editor/shell media into exports; this should collect from plan.root (and preserve the producer media contract) instead of the owner document.
  • packages/browser-export/src/timelineSeek.ts:52seekTimelines seeks every registered timeline to the same absolute frame time. Existing render/player code activates sibling timelines but seeks the root/master timeline; it explicitly avoids absolute-seeking registered child timelines because nested/sub-composition timelines are offset-relative and the root seek already propagates their local time. This implementation can overwrite that propagated child state and render sub-compositions at the wrong frame. The helper should mirror the runtime render seek path rather than iterating Object.values(registry) with absolute t.

Verification:

  • bun install --frozen-lockfile
  • bun run --filter @hyperframes/browser-export test (26 passed)
  • bun run --filter @hyperframes/browser-export typecheck
  • bun run --filter @hyperframes/browser-export build
  • bunx oxlint packages/browser-export/src packages/browser-export/tsup.config.ts
  • bunx oxfmt --check packages/browser-export/src packages/browser-export/tsup.config.ts packages/browser-export/package.json packages/browser-export/tsconfig.json package.json docs/docs.json
  • bun run build
  • Puppeteer/Vite browser smoke: video-only export returned video/webm, 1 frame, 563-byte Blob; audio-enabled scoped repro fetched /outside.wav from outside the composition root.

Verdict: REQUEST CHANGES
Reasoning: The basic browser encode path works, but audio collection and timeline seeking currently diverge from the established HyperFrames contracts in realistic editor/nested-composition cases.

— Codex

@miga-heygen

Copy link
Copy Markdown
Contributor

Closing — this PR has merge conflicts and has been inactive. Thank you for the contribution! Please feel free to reopen with a rebased branch if you'd like to continue.

@miga-heygen miga-heygen closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants