Skip to content

fix(engine): warn when a video input declares alpha but decodes opaque - #3259

Open
santhiprakash wants to merge 1 commit into
heygen-com:mainfrom
santhiprakash:fix/engine-input-alpha-opaque-warning
Open

fix(engine): warn when a video input declares alpha but decodes opaque#3259
santhiprakash wants to merge 1 commit into
heygen-com:mainfrom
santhiprakash:fix/engine-input-alpha-opaque-warning

Conversation

@santhiprakash

Copy link
Copy Markdown
Contributor

Implements #3226 — a video input whose alpha_mode=1 tag outlives its alpha plane composites as a solid opaque rectangle, and nothing tells the user their file, not the renderer, is the problem (#3220 is the worked example).

What

  • packages/engine/src/utils/alphaPlaneProbe.ts (new): probeInputAlphaPlane samples a video input's decoded alpha plane (forced libvpx-vp9 input decoder, up to 3 frames at 8x8 rgba, 30s bound, ~768-byte ceiling); sampledRgbaAlphaIsFullyOpaque decides uniformly-opaque / transparent / inconclusive. Sibling of the CLI's post-render webmAlphaCheck.sampledAlphaIsFullyOpaque, rebuilt for the engine (async via runFfmpeg, temp-file output).
  • packages/engine/src/services/videoFrameExtractor.ts: during extraction preflight, for every input whose metadata declares alpha (hasAlpha) and whose codec can carry alpha (vp9/vp8/prores), run the probe. If the alpha plane is uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy (deduped per src, abort-aware).
  • packages/engine/src/utils/alphaPlaneProbe.test.ts (new): 8 unit tests covering the byte logic and the warning text.

Why

alpha_mode=1 is container metadata that can outlive the alpha it describes — a remux can drop the BlockAdditional sidecar while keeping the tag. Warning only, never an error: an opaque video used as a full-frame background is legitimate, and an inconclusive probe stays silent. Sampling a few frames rather than first-frame-only is an explicit choice (the decision #3226 asked to make): it catches a clip that is opaque at the head and transparent later. The engine-level hook covers every render path (local CLI, Docker, cloud).

Verification

  • Engine suite: 1483 passed / 4 failed — the 4 failures are pre-existing on clean upstream main in this checkout (they reference a generated hdr-regression PNG fixture and the @hyperframes/core/audio-fx-runtime subpath, both absent locally; confirmed by stashing the change and re-running).
  • Engine typecheck clean; bun run lint 0 warnings; bun run format:check clean on the changed files.
  • Real-fixture checks with the exact probe command: a VP9 WebM carrying ALPHA_MODE=1 but no alpha plane → warns; a genuine yuva420p VP9 WebM (alpha 128) → silent; both agree with the alphaextract cross-check from Warn when a video input declares alpha but decodes fully opaque #3226.

Notes

  • Local commit used --no-verify: this checkout's lefthook typecheck/fallow steps fail on clean upstream main (missing generated artifacts), so they were skipped locally; the staged files themselves pass the hook's lint + format steps. CI runs the authoritative checks.

@santhiprakash
santhiprakash force-pushed the fix/engine-input-alpha-opaque-warning branch from ccfb862 to 76fd3cc Compare August 13, 2026 05:17
- Problem: a video input whose alpha_mode=1 tag outlives its alpha plane (a remux drops the BlockAdditional sidecar while keeping the tag) composites as a solid rectangle, and nothing tells the user their file, not the renderer, is the problem (heygen-com#3220 / heygen-com#3226).
- Fix: during extraction preflight, when an input declares alpha (hasAlpha) and its codec can carry alpha, sample the decoded alpha plane (up to 3 frames at 8x8 rgba, bounded 30s, ~768-byte ceiling). If uniformly opaque, emit a non-blocking stderr warning naming the file with the re-export remedy. Inconclusive probes stay silent; opaque full-frame backgrounds are legitimate and never fail the render.
- Verification: 8 new unit tests (byte logic + message); engine suite 1483/1487 with 4 pre-existing failures on clean main (missing hdr-regression PNG fixture in this clone); typecheck/lint/format clean; real-fixture checks: lying-tag WebM warns, genuine-alpha WebM stays silent, both agree with ffmpeg alphaextract.

Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
@santhiprakash
santhiprakash force-pushed the fix/engine-input-alpha-opaque-warning branch from 76fd3cc to 0b3f186 Compare August 15, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant