Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/windows-render.yml
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,11 @@ jobs:
# invocations, CRLF, file URLs, etc.) in existing vitest suites.
# The producer package is skipped because its tests require Docker /
# Linux-only tooling (Dockerfile.test, LFS golden MP4 baselines).
# The aws-lambda package is skipped because it targets the AWS Lambda
# Linux runtime exclusively (`@sparticuz/chromium` is Linux-only; the
# ZIP layout is for `/var/task` on AL2023). Its handler.test.ts also
# trips a bun-on-Windows workspace-symlink quirk reading the
# producer's transitive `hono` dep.
# -------------------------------------------------------------------
test-windows:
name: Tests on windows-latest
Expand Down Expand Up @@ -407,9 +412,9 @@ jobs:
shell: pwsh
run: bun run build

- name: Run tests (all packages except producer)
- name: Run tests (all packages except producer + aws-lambda)
shell: pwsh
run: bun run --filter "!@hyperframes/producer" test
run: bun run --filter "!@hyperframes/producer" --filter "!@hyperframes/aws-lambda" test

- name: Run runtime contract test
shell: pwsh
Expand Down
1 change: 1 addition & 0 deletions Dockerfile.test
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ COPY packages/producer/package.json packages/producer/package.json
COPY packages/cli/package.json packages/cli/package.json
COPY packages/studio/package.json packages/studio/package.json
COPY packages/shader-transitions/package.json packages/shader-transitions/package.json
COPY packages/aws-lambda/package.json packages/aws-lambda/package.json
RUN bun install --frozen-lockfile

# Copy source
Expand Down
207 changes: 196 additions & 11 deletions bun.lock

Large diffs are not rendered by default.

113 changes: 113 additions & 0 deletions packages/aws-lambda/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# @hyperframes/aws-lambda

AWS Lambda adapter for HyperFrames distributed rendering. Wraps the OSS
`plan` / `renderChunk` / `assemble` primitives into a single Lambda handler
that Step Functions can dispatch on, plus a build pipeline that bundles
the handler + Chrome runtime + ffmpeg into a deployable ZIP.

The Lambda adapter ships in two parts: the foundation (this package + the
SAM example) validates the architecture end-to-end on real AWS; the
user-facing surface (CLI, CDK construct, migration guide) lands in
follow-up PRs.

## Architecture

```
┌──────────────────────────────────────────────────────────────────┐
│ Step Functions state machine │
│ Plan → Map(N) RenderChunk → Assemble │
└──────────────────────────────────────────────────────────────────┘
│ dispatches by event.Action
▼
┌──────────────────────────────────────────────────────────────────┐
│ One Lambda function (this package's `dist/handler.zip`) │
│ handler.mjs │
│ ├─ Action="plan" → @hyperframes/producer/distributed │
│ ├─ Action="renderChunk" → @hyperframes/producer/distributed │
│ └─ Action="assemble" → @hyperframes/producer/distributed │
│ bin/ffmpeg — ffmpeg-static │
│ node_modules/@sparticuz/chromium/ — Lambda-optimised Chromium │
└──────────────────────────────────────────────────────────────────┘
│ pure functions over local paths
▼
┌──────────────────────────────────────────────────────────────────┐
│ S3 bucket — plan tarball + per-chunk outputs + final mp4 │
└──────────────────────────────────────────────────────────────────┘
```

The handler downloads inputs from S3 into `/tmp`, calls the OSS primitive,
uploads outputs back to S3, and returns a small JSON result that fits
inside Step Functions' history budget (under 200 bytes per chunk).

## Chrome runtime

The package supports two Chromium sources:

| Source | Default | Size | When to pick it |
| ------------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------- |
| `@sparticuz/chromium` | yes | ~70 MiB compressed | Lambda. Decompresses into `/tmp` at runtime; the rest of the ecosystem already uses it for headless-Chrome-in-Lambda. |
| Bundled `chrome-headless-shell` | no | ~140 MiB | Fallback. Used if `@sparticuz/chromium` ever drops `HeadlessExperimental.beginFrame` support. |

Pick the source at build time:

```bash
bun run --cwd packages/aws-lambda build:zip
bun run --cwd packages/aws-lambda build:zip -- --source=chrome-headless-shell
```

The handler reads `HYPERFRAMES_LAMBDA_CHROME_SOURCE` at boot. The build
script sets that env var via Lambda function configuration in
`examples/aws-lambda/template.yaml`.

## BeginFrame regression guard

HyperFrames' renderer drives Chrome via the CDP
`HeadlessExperimental.beginFrame` command — same path the K8s deploy uses.
The Lambda adapter assumes that `@sparticuz/chromium`'s
chrome-headless-shell build honours BeginFrame. To prove it (and re-prove
it on every release), the package ships a Docker probe:

```bash
# Build the Lambda-like container and run the probe.
bun run --cwd packages/aws-lambda probe:beginframe:docker
```

The probe boots `@sparticuz/chromium` inside
`public.ecr.aws/lambda/nodejs:22` and asserts CDP `beginFrame` with
`screenshot: true` returns a PNG buffer. Exit code 0 = green; non-zero =
fall back to bundling chrome-headless-shell directly via `--source=chrome-headless-shell`.

## Building the ZIP

```bash
bun install # at the monorepo root
bun run --cwd packages/aws-lambda build:zip # → packages/aws-lambda/dist/handler.zip
bun run --cwd packages/aws-lambda verify:zip-size # CI gate
```

The build script bundles `src/handler.ts` via esbuild, stages
`@sparticuz/chromium` and `puppeteer-core` under `node_modules/`, copies
ffmpeg-static into `bin/`, and zips the result. The unzipped layout is
designed to extract cleanly into Lambda's `/var/task/`.

`verify:zip-size` enforces:

- Unzipped ≤ 248 MiB (in-house budget; Lambda hard ceiling is 250 MiB unzipped — AWS docs label this "250 MB" but use binary mebibytes)
- Zipped ≤ 150 MiB (in-house budget; Lambda has no hard zipped cap for S3-deployed functions)

CI fails the PR if either is exceeded.

## Running tests

```bash
bun run --cwd packages/aws-lambda test # unit tests (no Chrome)
bun run --cwd packages/aws-lambda probe:beginframe # local probe (Linux only)
```

## What's NOT in this PR

- `examples/aws-lambda/template.yaml` (SAM template — separate PR).
- Real-AWS deploy + smoke workflow (separate PR).
- `npx hyperframes lambda deploy` CLI — follow-up.
- CDK construct (`HyperframesRenderStack`) — follow-up.
- Migration guide — follow-up.
55 changes: 55 additions & 0 deletions packages/aws-lambda/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"name": "@hyperframes/aws-lambda",
"version": "0.0.1",
"description": "AWS Lambda adapter for HyperFrames distributed rendering — Plan/RenderChunk/Assemble handler + ZIP bundling.",
"repository": {
"type": "git",
"url": "https://github.com/heygen-com/hyperframes",
"directory": "packages/aws-lambda"
},
"files": [
"src/",
"scripts/",
"README.md"
],
"type": "module",
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts",
"./handler": "./src/handler.ts"
},
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"scripts": {
"build": "tsc --noEmit",
"build:zip": "tsx scripts/build-zip.ts",
"probe:beginframe": "tsx scripts/probe-beginframe.ts",
"probe:beginframe:docker": "docker build -f scripts/probe-beginframe.dockerfile -t hyperframes-lambda-probe:local ../.. && docker run --rm hyperframes-lambda-probe:local",
"test": "bun test",
"typecheck": "tsc --noEmit",
"verify:zip-size": "tsx scripts/verify-zip-size.ts"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.700.0",
"@hyperframes/producer": "workspace:^",
"@sparticuz/chromium": "148.0.0",
"ffmpeg-static": "^5.2.0",
"ffprobe-static": "^3.1.0",
"puppeteer-core": "^24.39.1",
"tar": "^7.4.3"
},
"devDependencies": {
"@types/aws-lambda": "^8.10.146",
"@types/node": "^25.0.10",
"@types/tar": "^6.1.13",
"esbuild": "^0.25.12",
"tsx": "^4.21.0",
"typescript": "^5.7.2"
},
"engines": {
"node": ">=22"
}
}
15 changes: 15 additions & 0 deletions packages/aws-lambda/scripts/_formatBytes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/**
* Shared binary-unit byte formatter for the build/verify scripts.
*
* The Lambda ZIP-size budget is in mebibytes (Lambda's 250 MB / 248 MiB
* gate is binary, not decimal), so logs and CI failure messages use
* KiB / MiB / GiB. This is intentionally a different unit system from
* `packages/cli/src/ui/format.ts`'s `formatBytes` (KB / MB, decimal) —
* don't conflate them.
*/
export function formatBytes(bytes: number): string {
if (bytes < 1024) return `${bytes} B`;
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KiB`;
if (bytes < 1024 * 1024 * 1024) return `${(bytes / (1024 * 1024)).toFixed(1)} MiB`;
return `${(bytes / (1024 * 1024 * 1024)).toFixed(2)} GiB`;
}
Loading
Loading