Skip to content
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/guides/rendering.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,9 @@ Video assets referenced by a composition (a `<video src="...">` clip) are decode
- VP8 / VP9 and ProRes 4444 (with alpha; see [Transparent Video](#transparent-video))
- HDR sources (HLG / PQ) are tone-mapped for SDR renders; see the [HDR guide](/guides/hdr)

The one caveat is **live preview**: `preview`, `play`, Studio, and published player pages play the file in a real browser, so playback there depends on that browser's codec support. Chrome (107+), Edge, and Safari hardware-decode HEVC on most modern machines; Firefox does not. If an HEVC asset shows a black frame in preview while rendering fine, generate an H.264 proxy for authoring (for example with `ffmpeg -i asset.mp4 -c:v libx264 -crf 18 proxy.mp4`, or via the media-use skill) and swap the original back in for the final render, or just keep the HEVC source, since the rendered output is unaffected.
The one caveat is **live preview**: `preview`, `play`, Studio, and published player pages play the file in a real browser, so playback there depends on that browser's codec support. Chrome (107+), Edge, and Safari hardware-decode HEVC on most modern machines; Firefox does not. Preview handles this automatically: on first use, it transcodes the asset into a cached H.264 proxy (stored under `.transcode-cache/`, safe to delete) and serves that for playback, while render always uses the original file. Publishing bakes the same proxies into the published archive so static player pages work too. Opt out per command with `--no-proxy`, or project-wide by setting `media.autoProxy` to `false` in `hyperframes.json`.

`hyperframes lint` emits an info-level `hevc_preview_codec` note when a composition references an HEVC video, as a reminder of this preview-only limitation.
`hyperframes lint` emits an info-level `hevc_preview_codec` note when a composition references an HEVC video, naming the affected asset(s).

## Animated GIF

Expand Down
12 changes: 6 additions & 6 deletions docs/guides/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -137,15 +137,15 @@ If your issue is about a specific coding mistake (animations not working, video
</Accordion>

<Accordion title="Video asset plays black in preview but renders fine (HEVC/H.265)">
Rendering decodes video assets with FFmpeg, so HEVC (H.265) inputs render correctly on every platform. Live preview is different: `preview`, `play`, Studio, and published player pages play the file in your browser, and not every browser decodes HEVC (Chrome 107+, Edge, and Safari do on most modern hardware; Firefox does not).
Rendering decodes video assets with FFmpeg, so HEVC (H.265) inputs render correctly on every platform. Live preview is different: `preview`, `play`, Studio, and published player pages play the file in your browser, and not every browser decodes HEVC (Chrome 107+, Edge, and Safari do on most modern hardware; Firefox does not). This is now self-healing: preview detects an undecodable asset and automatically swaps in a cached H.264 proxy, so this should no longer show as a persistent black frame.

If an HEVC asset shows a black or frozen frame in preview:
If a hostile asset is still stuck black in preview:

1. Confirm the render itself is fine: `npx hyperframes render` output will contain the video.
2. For authoring, generate an H.264 proxy (`ffmpeg -i asset.mp4 -c:v libx264 -crf 18 proxy.mp4`) and point the composition at it while you iterate.
3. Swap the HEVC original back before the final render, or keep the proxy; both render correctly.
1. Confirm auto-proxying is not disabled: no `--no-proxy` flag, and `media.autoProxy` is not `false` in `hyperframes.json`.
2. Run `npx hyperframes doctor` to confirm ffmpeg/ffprobe are installed; the proxy transcode needs both.
3. Run `npx hyperframes lint`: the info-level `hevc_preview_codec` finding names every affected asset.

`npx hyperframes lint` flags HEVC assets with an info-level `hevc_preview_codec` note. See [Rendering: Input Video Codecs](/guides/rendering#input-video-codecs).
See [Rendering: Input Video Codecs](/guides/rendering#input-video-codecs).
</Accordion>
</AccordionGroup>

Expand Down
198 changes: 198 additions & 0 deletions packages/cli/src/commands/play.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { Hono } from "hono";
import type { ProjectDir } from "../utils/project.js";

// `registerCompositionRoute` reaches these two studio-server helpers via a
// module, since both files sit at the same depth under packages/cli/src/.
//
// The error class lives inside this `vi.hoisted` block (not a plain top-level
// `class`) because `vi.mock` factories run during static-import resolution —
// before any of the test file's own top-level statements execute — so a
// `class` declared below would still be in its temporal dead zone.
const mocks = vi.hoisted(() => {
class FakeProxyTranscodeError extends Error {
readonly exitCode: number | null;
readonly stderrTail: string;
constructor(message: string, exitCode: number | null = null, stderrTail = "") {
super(message);
this.name = "ProxyTranscodeError";
this.exitCode = exitCode;
this.stderrTail = stderrTail;
}
}
return {
resolveProxy: vi.fn<(projectDir: string, absoluteSourcePath: string) => Promise<string>>(
// Benign default so `injectMediaCodecMap`'s fire-and-forget pre-warm
// (called for every hostile map entry) always gets a promise to
// `.catch()`, even in tests that don't care about the proxy path.
async () => "/unused-prewarm-proxy-path",
),
scanProjectMediaCodecMap: vi.fn<
(
...args: unknown[]
) => Promise<
Record<
string,
{ codecName: string; browserHostile: boolean; representativeMime: string | null }
>
>
>(async () => ({})),
ProxyTranscodeError: FakeProxyTranscodeError,
};
});
const FakeProxyTranscodeError = mocks.ProxyTranscodeError;

vi.mock("@hyperframes/studio-server/proxy-transcoder", () => ({
resolveProxy: mocks.resolveProxy,
ProxyTranscodeError: mocks.ProxyTranscodeError,
}));

// The shared injection helper ships as a self-contained dist bundle (its copy
// of scanProjectMediaCodecMap is inlined), so it must be mocked wholesale —
// mocking the media-codec-map subpath can't reach inside it. The fake mirrors
// the real contract (scan → inject tag) via this file's scan mock so the
// existing injection assertions stay meaningful.
vi.mock("@hyperframes/studio-server/media-proxy-preview", () => ({
injectMediaCodecMapIntoHtml: vi.fn(
async (html: string, projectDir: string, htmlSources: unknown[]) => {
const map = await mocks.scanProjectMediaCodecMap(projectDir, htmlSources);
const tag = `<script data-hf-media-codec-map>window.__HF_MEDIA_CODEC_MAP__=${JSON.stringify(map)};</script>`;
return html.includes("</head>")
? html.replace("</head>", `${tag}\n</head>`)
: `${tag}\n${html}`;
},
),
}));

const { registerCompositionRoute } = await import("./play.js");

let dir: string | undefined;

function tmpProject(): ProjectDir {
dir = mkdtempSync(join(tmpdir(), "hf-play-test-"));
return { dir, name: "test-project", indexPath: join(dir, "index.html") };
}

afterEach(() => {
mocks.resolveProxy.mockReset();
mocks.resolveProxy.mockResolvedValue("/unused-prewarm-proxy-path");
mocks.scanProjectMediaCodecMap.mockReset();
mocks.scanProjectMediaCodecMap.mockResolvedValue({});
if (dir) rmSync(dir, { recursive: true, force: true });
dir = undefined;
});

async function buildApp(project: ProjectDir, autoProxy: boolean): Promise<Hono> {
const app = new Hono();
await registerCompositionRoute(app, project, autoProxy);
return app;
}

describe("registerCompositionRoute", () => {
it("answers a Range request on a plain asset with 206 + the requested byte slice", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "clip.mp4"), Buffer.from("0123456789", "utf-8"));
const app = await buildApp(project, true);

const res = await app.request("/composition/clip.mp4", { headers: { Range: "bytes=2-5" } });

expect(res.status).toBe(206);
expect(res.headers.get("accept-ranges")).toBe("bytes");
expect(res.headers.get("content-range")).toBe("bytes 2-5/10");
expect(await res.text()).toBe("2345");
});

it("serves the resolved proxy's bytes for ?hf-proxy=h264 on a hostile asset", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "clip.mp4"), "original-hevc-bytes");
const proxyPath = join(project.dir, "proxy.mp4");
writeFileSync(proxyPath, "transcoded-h264-bytes");
mocks.resolveProxy.mockResolvedValue(proxyPath);
const app = await buildApp(project, true);

const res = await app.request("/composition/clip.mp4?hf-proxy=h264");

expect(res.status).toBe(200);
expect(await res.text()).toBe("transcoded-h264-bytes");
expect(mocks.resolveProxy).toHaveBeenCalledWith(project.dir, join(project.dir, "clip.mp4"));
});

it("serves ?hf-proxy=h264 for a .mov hostile asset as Content-Type video/mp4 (the proxy IS mp4)", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "clip.mov"), "original-prores-bytes");
const proxyPath = join(project.dir, "proxy.mp4");
writeFileSync(proxyPath, "transcoded-h264-bytes");
mocks.resolveProxy.mockResolvedValue(proxyPath);
const app = await buildApp(project, true);

const res = await app.request("/composition/clip.mov?hf-proxy=h264");

expect(res.status).toBe(200);
expect(res.headers.get("content-type")).toBe("video/mp4");
expect(await res.text()).toBe("transcoded-h264-bytes");
});

it("answers 502 (not a silent failure) when the proxy transcode fails", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "clip.mp4"), "original-hevc-bytes");
mocks.resolveProxy.mockRejectedValue(
new FakeProxyTranscodeError("ffmpeg exited with code 1", 1),
);
const app = await buildApp(project, true);

const res = await app.request("/composition/clip.mp4?hf-proxy=h264");

expect(res.status).toBe(502);
});

it("404s ?hf-proxy=h264 for a non-video asset without attempting a transcode", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "image.png"), "not-a-video");
const app = await buildApp(project, true);

const res = await app.request("/composition/image.png?hf-proxy=h264");

expect(res.status).toBe(404);
expect(mocks.resolveProxy).not.toHaveBeenCalled();
});

it("injects __HF_MEDIA_CODEC_MAP__ into served composition HTML", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "index.html"), "<html><head></head><body></body></html>");
mocks.scanProjectMediaCodecMap.mockResolvedValue({
"/clip.mp4": {
codecName: "hevc",
browserHostile: true,
representativeMime: 'video/mp4; codecs="hvc1.1.6.L120.B0"',
},
});
const app = await buildApp(project, true);

const res = await app.request("/composition/index.html");
const html = await res.text();

expect(html).toContain("__HF_MEDIA_CODEC_MAP__");
expect(html).toContain("/clip.mp4");
});

it("opt-out (autoProxy=false) skips codec-map injection and 404s the proxy param", async () => {
const project = tmpProject();
writeFileSync(join(project.dir, "index.html"), "<html><head></head><body></body></html>");
writeFileSync(join(project.dir, "clip.mp4"), "original-hevc-bytes");
mocks.scanProjectMediaCodecMap.mockResolvedValue({
"/clip.mp4": { codecName: "hevc", browserHostile: true, representativeMime: null },
});
const app = await buildApp(project, false);

const htmlRes = await app.request("/composition/index.html");
expect(await htmlRes.text()).not.toContain("__HF_MEDIA_CODEC_MAP__");
expect(mocks.scanProjectMediaCodecMap).not.toHaveBeenCalled();

const proxyRes = await app.request("/composition/clip.mp4?hf-proxy=h264");
expect(proxyRes.status).toBe(404);
expect(mocks.resolveProxy).not.toHaveBeenCalled();
});
});
98 changes: 79 additions & 19 deletions packages/cli/src/commands/play.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,17 @@ export const examples: Example[] = [
"Open with CDP enabled (requires browser path + isolated profile)",
"hyperframes play --browser-path /usr/bin/chromium --user-data-dir /tmp/hf-profile --remote-debugging-port 9222",
],
[
"Disable auto-proxying of browser-hostile video codecs (HEVC, ProRes, AV1)",
"hyperframes play --no-proxy",
],
];
import { resolve } from "node:path";
import type { Hono } from "hono";
import * as clack from "@clack/prompts";
import { c } from "../ui/colors.js";
import { resolveProject } from "../utils/project.js";
import { resolveProject, type ProjectDir } from "../utils/project.js";
import { resolveAutoProxy } from "../utils/projectConfig.js";
import {
openBrowser,
parseRemoteDebuggingPort,
Expand All @@ -27,8 +33,11 @@ import {
resolvePlayerPath,
listenOnFreePort,
injectRuntime,
injectMediaCodecMap,
buildRangeResponse,
assetContentType,
} from "../utils/compositionServer.js";
import { resolveProxy, ProxyTranscodeError } from "@hyperframes/studio-server/proxy-transcoder";

export default defineCommand({
meta: { name: "play", description: "Play a composition in a lightweight browser player" },
Expand All @@ -52,6 +61,12 @@ export default defineCommand({
type: "string",
description: "Chromium remote debugging port (requires --browser-path and --user-data-dir)",
},
proxy: {
type: "boolean",
description:
"Auto-transcode browser-hostile video codecs (HEVC, ProRes, AV1) to a cached H.264 proxy for preview (default: on; overrides hyperframes.json's media.autoProxy)",
negativeDescription: "Disable auto-proxying of browser-hostile video codecs",
},
},
async run({ args }) {
const project = resolveProject(args.dir);
Expand Down Expand Up @@ -107,7 +122,6 @@ export default defineCommand({

const { Hono } = await import("hono");
const { createAdaptorServer } = await import("@hono/node-server");
const { isSafePath } = await import("@hyperframes/core/studio-api");

const app = new Hono();

Expand All @@ -127,23 +141,8 @@ export default defineCommand({
});
});

// Serve composition files (HTML + assets)
app.get("/composition/*", (ctx) => {
const reqPath = ctx.req.path.replace("/composition/", "");
const filePath = resolve(project.dir, reqPath);

// Security: don't allow path traversal outside project dir. isSafePath
// canonicalizes symlinks and applies a trailing-separator guard, so neither
// an in-project symlink to an external target nor a sibling dir whose name
// shares the project-dir prefix (e.g. `<dir>-evil`) can escape.
if (!isSafePath(project.dir, filePath)) return ctx.text("Forbidden", 403);
if (!existsSync(filePath)) return ctx.text("Not found", 404);
// HTML gets the runtime injected; other assets pass through with a guessed type.
if (filePath.endsWith(".html")) {
return ctx.html(injectRuntime(readFileSync(filePath, "utf-8")));
}
return ctx.body(readFileSync(filePath), 200, { "Content-Type": assetContentType(filePath) });
});
const autoProxy = resolveAutoProxy(project.dir, args.proxy as boolean | undefined);
await registerCompositionRoute(app, project, autoProxy);

// Main page — the player wrapper
app.get("/", (ctx) => {
Expand Down Expand Up @@ -181,6 +180,67 @@ export default defineCommand({
},
});

/**
* Registers the `/composition/*` route: serves composition HTML (runtime +
* `__HF_MEDIA_CODEC_MAP__` injected) and asset files, with byte-Range support
* (`play` previously did a whole-file `readFileSync`, so seeking/duration
* probing on media elements never worked) and a `?hf-proxy=h264` branch that
* serves the cached H.264 authoring proxy for a browser-hostile video asset
* (per docs/plans/2026-07-14-002-feat-transparent-media-proxies-plan.md,
* unit U4). Exported standalone (rather than inlined in `run()`) so tests can
* exercise it via `app.request(...)` without booting a real HTTP listener,
* `clack` UI, or `openBrowser`.
*/
export async function registerCompositionRoute(
app: Hono,
project: ProjectDir,
autoProxy: boolean,
): Promise<void> {
const { isSafePath } = await import("@hyperframes/core/studio-api");

// fallow-ignore-next-line complexity
app.get("/composition/*", async (ctx) => {
const reqPath = ctx.req.path.replace("/composition/", "");
const filePath = resolve(project.dir, reqPath);

// Security: don't allow path traversal outside project dir. isSafePath
// canonicalizes symlinks and applies a trailing-separator guard, so neither
// an in-project symlink to an external target nor a sibling dir whose name
// shares the project-dir prefix (e.g. `<dir>-evil`) can escape.
if (!isSafePath(project.dir, filePath)) return ctx.text("Forbidden", 403);
if (!existsSync(filePath)) return ctx.text("Not found", 404);

// HTML gets the runtime + codec-map injected; other assets pass through.
if (filePath.endsWith(".html")) {
let html = injectRuntime(readFileSync(filePath, "utf-8"));
if (autoProxy) {
html = await injectMediaCodecMap(html, project.dir, [{ html, compSrcPath: reqPath }]);
}
return ctx.html(html);
}

const contentType = assetContentType(filePath);
if (ctx.req.query("hf-proxy") === "h264") {
// Opt-out (or a non-video asset) 404s the param without attempting a
// transcode; a missing asset already 404'd above.
if (!autoProxy || !contentType.startsWith("video/")) return ctx.text("Not found", 404);
try {
const proxyPath = await resolveProxy(project.dir, filePath);
// The proxy IS an mp4 regardless of the source's extension (.mov,
// .mkv, ...) — serve its real type, matching the preview route.
return buildRangeResponse(proxyPath, "video/mp4", ctx.req.header("Range"));
} catch (err) {
if (err instanceof ProxyTranscodeError) {
return ctx.text(`Proxy transcode failed: ${err.message}`, 502);
}
throw err;
}
}

return buildRangeResponse(filePath, contentType, ctx.req.header("Range"));
});
}

function buildPlayerPage(projectName: string): string {
return `<!doctype html>
<html lang="en">
Expand Down
Loading
Loading