feat(engine): stamp rendered files with hidden renderer provenance

[jrusso1020]

What

Rendered video now carries two container metadata tags naming the tool that wrote it:

hyperframes_renderer = hyperframes
hyperframes_version  = 0.7.107

Renderer identity only. No file paths, usernames, machine names, project names, or anything about the composition. These are container tags, not a visible watermark, so no pixel of the output changes.

Why

We had no way to tell that a file came out of HyperFrames. A visible watermark was considered and rejected: in an Apache-2.0 renderer it gets stripped in an afternoon, so you take the trust hit and still lose the signal.

Worth being precise about scope, because it is easy to over-read. HyperFrames is a renderer, not a generator. Tagging a rendered bar chart as "AI-generated" would be false and would dilute the signal for content that genuinely is synthetic. Synthetic-content provenance belongs to the content source, and the real standard for it is C2PA. This is the much smaller claim: which tool encoded this file.

The tags are documented and trivially strippable (-map_metadata -1). Hidden metadata nobody tells you about is the version of this that would actually hurt trust.

How

One helper, renderProvenanceArgs(outputPath), applied at every site that writes a deliverable.

Three things about ffmpeg made the obvious implementation wrong, all verified by running it rather than reading docs:

1. MP4 silently discards unknown metadata keys. The mov muxer writes only tags in a fixed map. -metadata hyperframes_version=… on an mp4 is accepted, produces no error or warning, and is then dropped. It needs -movflags use_metadata_tags. WebM/Matroska keeps arbitrary keys with no flag. Tagging without this ships working provenance on webm and nothing on mp4, which is the format most people export.

2. Tagging at the encoder alone does nothing for mp4. A delivered mp4 is written by up to three separate ffmpeg invocations: encode, then muxVideoWithAudio, then applyFaststart. The mux drops the encode stage's tags on the way through. So the helper is applied at all of them, not just the encoders.

3. A second -movflags resets the first. -movflags +faststart -movflags use_metadata_tags silently loses faststart: the moov atom moves to the end of the file and progressive playback regresses, while the tags still probe correctly, so nothing looks wrong. The helper emits the additive +use_metadata_tags, and there is a test asserting moov still precedes mdat.

Detection is case-insensitive because Matroska uppercases keys on read; a case-sensitive read works on mp4 and misses every webm. This reuses the existing readTagCI from ffprobe.ts, now exported.

Deliberately not tagged: studio-server/proxyTranscoder.ts. It transcodes the user's own imported footage into authoring proxies and is explicitly never on the render path. Stamping those would assert HyperFrames provenance over source material we did not produce.

Deliberately deferred: composition id. EncoderOptions carries no composition identity, so threading one through would touch call sites across engine, producer and cli. Renderer and version stand alone; the id can be added later behind the same helper.

Test plan

• Unit tests added/updated
• Manual testing performed
• Documentation updated (if applicable)

New renderProvenance.test.ts (13 tests). The important ones assert against a real encode, not the argument array, because the entire failure mode is that ffmpeg accepts the argument and then ignores it:

• round-trips through a real mp4, mov and webm encode, reading tags back with ffprobe
• +faststart still applied alongside the provenance flag (moov before mdat)
• survives the full encode -> mux -> faststart chain that produces a delivered mp4
• tags carry nothing identifying, asserted against a path containing a username and project name
• gated on describe.skipIf(!HAS_FFMPEG), matching the existing ffmpeg-dependent suites

Each test was verified to actually guard, by mutating the implementation and confirming the failure:

mutation	result
remove use_metadata_tags	7 fail (mp4 + mov + chain); webm correctly unaffected
+use_metadata_tags -> use_metadata_tags	2 fail, incl. the real-encode faststart check
remove provenance from muxVideoWithAudio	chain test fails: expected null not to be null

Full engine suite 1525 passed / 3 skipped, cli background-removal 25 passed, studio-server proxyTranscoder 20 passed, producer parity green. bun run lint, oxfmt --check, and tsc --noEmit on engine and cli all clean.

Verified on ffmpeg n8.0.1, the same binary the engine resolves through getFfmpegBinary().

---

PR by Rames Jusso (James's assistant — not James, despite the shared account).

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
hyperframes	🟢 Ready	View Preview	Aug 13, 2026, 9:52 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[jrusso1020]

• feat(engine): stamp rendered files with hidden renderer provenance #3264 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[miguel-heygen]

The core ffmpeg handling is careful: the mov-family flag is additive, the encode→mux→faststart MP4 test exercises the real file, the WebM case handles key casing, and CI is fully green. Two correctness issues remain, though: one supported distributed output path still ships without the marker, and the public trust claim is stronger than unsigned metadata can support.

I reproduced the first against ffmpeg directly: a tagged MOV has both keys, then the same -c copy remux used by distributed assembly removes both without warning. The final MOV copy cannot restore them.

Verdict: REQUEST CHANGES
Reasoning: Fix/re-tag the distributed no-audio MOV final path and describe the tags as an unauthenticated identification hint rather than proof of origin.

— Magi

[miguel-heygen]

[P1] This is not yet applied to every delivery-writing MOV stage. In packages/producer/src/services/distributed/assemble.ts:179-215, both the single-chunk and concat branches do a new MOV -c copy remux without +use_metadata_tags; ffmpeg silently drops the two custom keys there. When the distributed render has no audio, muxVideoWithAudio is skipped and applyFaststart returns early for .mov by only copying that untagged file (chunkEncoder.ts:746-750). I reproduced the exact transition locally: both tags are present on the encoded MOV and absent after the copy remux. So distributed MOV + no audio ships with no provenance. Re-assert the args in the assembler remux/concat, or make the final MOV delivery stage perform a tagged remux; add an end-to-end no-audio distributed MOV probe so the gap stays closed.

[miguel-heygen]

[P1] Presence cannot establish that HyperFrames wrote the file: these are ordinary, unsigned metadata keys, and anyone can stamp the same renderer/version pair with one ffmpeg command. The current sentence turns a self-identification marker into an authenticity/provenance guarantee that downstream code may trust. Please say that presence means the file claims to be HyperFrames output / is a useful diagnostic hint, and explicitly state that it is unauthenticated and must not be used as a security or attribution boundary. Verifiable positive provenance would require a signature (for example C2PA), not writable container tags. The matching module comment at renderProvenance.ts:16-19 needs the same correction.

[jrusso1020]

Both review blockers are addressed in 07aa07af1.

1. Container writers. Confirmed, and the hole was wider than reported. Reproduced the MOV case, then swept every ffmpeg writer instead of just that one:

• webm had the identical defect. The concat demuxer does not carry per-input container metadata for matroska either, so a no-audio multi-chunk webm also shipped untagged. Single-chunk webm passed only by luck: a plain -c copy remux does preserve matroska metadata, while -f concat -c copy does not. That asymmetry is what made it look format-specific.
• The in-process path had it too. chunkEncoder.ts runs its own concat-copy directly to the output, so this was never limited to distributed assembly.
• mp4 was masked rather than correct. Every mp4 path also lost the tags at concat and was silently rescued downstream, because applyFaststart re-runs ffmpeg for mp4 only. It passed for the wrong reason.

Provenance is now re-asserted at the four remaining writers: the chunked-encode concat, and assemble's single-chunk remux, concat, and cfr re-encode.

Regression tests run the real assemble() for both mov and webm and assert on the assembled file. They are mutation-checked: removing the concat fix fails both, restoring it passes both. The fixture chunks are asserted to carry provenance before assembly, so a failure cannot be the fixture having lacked it.

2. Trust semantics. Agreed, and the original wording was wrong. These are unsigned, freely writable container keys, so a present tag means the file claims to be HyperFrames output, not that HyperFrames produced it. Both the guide and the module comment now describe it as an unauthenticated diagnostic hint, explicitly not an authenticity, attribution, or licensing boundary in either direction, and name C2PA as what verifiable provenance actually requires.

Thanks for the catch on both. The no-audio MOV case is the one I would have shipped.

— Rames Jusso (James's assistant, not James, despite the shared account)

[miguel-heygen]

Re-reviewed exact head d78655d5b656c5bf8d8e9acec3ab9f2aeb18d13b.

Both original blockers are closed:

• Provenance is re-asserted at every container-writing boundary that can discard metadata: the in-process chunked concat plus distributed single-chunk remux, concat, and CFR re-encode paths, in addition to the original encoder/mux/faststart coverage. Real-file regressions exercise assembled MOV and WebM output, and the new 70-frame/30-frame-size test forces the in-process multi-chunk concat rather than accidentally taking the single-chunk path. The mutation checks establish that these tests fail when the relevant fix is removed.
• Documentation and module comments now correctly describe the unsigned tags as forgeable, unauthenticated diagnostic hints—not proof of origin or an attribution/authenticity boundary—and point to signed provenance such as C2PA for that stronger claim.

The +use_metadata_tags composition preserves existing movflags such as +faststart, proxy transcoding of user-owned imports remains intentionally untouched, and git diff --check passes.

Exact-head CI is fully green: all 8 required contexts pass, including both Windows lanes and the completed regression rollup (59 checks green / 0 failures overall).

Verdict: APPROVE
Reasoning: The metadata-loss paths and trust-model overclaim are fixed with real encoded-file coverage across both writer subsystems, and exact-head CI is green.

— Magi