build(container): restrict the in-tree ffmpeg to a narrow media-codec allowlist (OPS-7665)

[dmitry-tokarev-nv]

Summary

The in-tree ffmpeg already restricted encoders/muxers, but it still shipped ffmpeg's full default decoder set (291 video + 212 audio decoders) in the libav*.so copied into every runtime image. Narrow it to the media formats we actually build and use.

This switches the wheel_builder ffmpeg configure to an allowlist: blanket --disable-decoders/--disable-demuxers/--disable-parsers/--disable-protocols plus exactly the components the two real paths need:

• encode CLI (imageio): rawvideo demux/decode over a pipe → h264_nvenc/libvpx_vp9
• Rust media-ffmpeg VideoDecoder: VP8/VP9 in mp4/webm/mkv

Image decode does not use ffmpeg (it goes through the Rust image crate), so no still-image decoders are enabled. HW NVDEC (h264_cuvid/hevc_cuvid) can be re-added explicitly if a decode feature ever needs H.264/H.265 (not wired today — noted in the media README).

Also bumps ffmpeg 8.1 → 8.1.2 (an upstream maintenance release) to pick up security fixes and trim the decode surface further.

Test fixtures are re-encoded to VP9-in-mp4; filenames and every Rust/Python test that consumes them are unchanged.

Pairs with the compliance-side gate in #11632. See OPS-7665 for background.

Validation

• Built ffmpeg 8.1.2 with this exact allowlist — configure accepts every component token (caught one bug: webm is not a separate demuxer token; matroska covers it) and the build succeeds LGPL-2.1.
• Decoded all five regenerated fixtures through that restricted build to raw YUV420p; byte counts match frames × w × h × 1.5 exactly (2×2/10f, 320×240/1f/10f/100f, 3840×2160/10f).
• container/render.py renders the sglang runtime (the enable_media_ffmpeg=true consumer) with the new configure block intact.
• Fixtures committed as Git LFS; native_packages.yaml ffmpeg version bumped to match.

Linear: https://linear.app/nvidia/issue/OPS-7665/disable-ffmpeg-decoders-and-add-image-scan

🤖 Generated with Claude Code

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-019f720e-3b29-7099-b207-9068a00f224d.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🔍 Pre-existing --disable-bsfs may affect VP9 decode in MP4 containers

The --disable-bsfs flag at container/templates/wheel_builder.Dockerfile:357 was already present before this PR and disables all bitstream filters. Some demuxers/decoders use bitstream filters internally (e.g., vp9_superframe for VP9 in MP4). If VP9-in-MP4 decode fails at runtime due to missing BSFs, this pre-existing flag would be the cause. The test fixtures were regenerated with -g 1 (all keyframes), which may mask the issue since superframe handling is primarily needed for non-keyframe packets. Worth monitoring in integration tests with real-world VP9-in-MP4 content that has B-frames or superframes.

(Refers to line 357)

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[coderabbitai]

Walkthrough

Changes

FFmpeg version and codec policy

Layer / File(s)	Summary
FFmpeg version metadata container/compliance/native_packages.yaml, container/context.yaml	FFmpeg references are updated from 8.1 to 8.1.2, with comments documenting the CVE-2026-8461 fix and MagicYUV removal.
FFmpeg codec build policy container/templates/wheel_builder.Dockerfile	The wheel builder documents codec constraints and configures allowlisted VP8, VP9, and rawvideo decoders with corresponding container and parser restrictions.
Media codec documentation lib/llm/src/preprocessor/media/README.md, lib/llm/src/preprocessor/media/decoders/video.rs	Documentation identifies supported VP9-in-MP4 fixtures and unsupported H.264, H.265, and AAC frontend decoding.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description covers the change well, but it misses the template's required 'Where should the reviewer start?' and 'Related Issues' sections.	Add the missing sections, especially a reviewer-start file list and the required issue linkage or explicit 'not linked' confirmation.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: restricting in-tree FFmpeg to a codec allowlist.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/llm/src/preprocessor/media/decoders/video.rs`:
- Around line 318-320: Update the ffmpeg regeneration command in the video
fixture documentation to append an MP4 output filename after the encoding
options, making the example executable while preserving its existing input,
codec, frame-count, and keyframe settings.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 040fe901-aba8-4e7a-99ee-3261d77e31a7

📥 Commits

Reviewing files that changed from the base of the PR and between 2bec928 and 9ed1a9a.

⛔ Files ignored due to path filters (5)

• lib/llm/tests/data/media/2160p_10.mp4 is excluded by !**/*.mp4
• lib/llm/tests/data/media/240p_1.mp4 is excluded by !**/*.mp4
• lib/llm/tests/data/media/240p_10.mp4 is excluded by !**/*.mp4
• lib/llm/tests/data/media/240p_100.mp4 is excluded by !**/*.mp4
• lib/llm/tests/data/media/2p_10.mp4 is excluded by !**/*.mp4

📒 Files selected for processing (5)

• container/compliance/native_packages.yaml
• container/context.yaml
• container/templates/wheel_builder.Dockerfile
• lib/llm/src/preprocessor/media/README.md
• lib/llm/src/preprocessor/media/decoders/video.rs

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the target file around the cited lines.
sed -n '300,335p' lib/llm/src/preprocessor/media/decoders/video.rs

# Look for similar ffmpeg regeneration instructions elsewhere.
rg -n "ffmpeg -f lavfi|libvpx-vp9|testsrc2|240p_10\.mp4|Fixtures are VP9-in-mp4" lib -S

Repository: ai-dynamo/dynamo

Length of output: 2531

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '310,330p' lib/llm/src/preprocessor/media/decoders/video.rs

Repository: ai-dynamo/dynamo

Length of output: 1124

---

Add an output path to the ffmpeg example. The regeneration command is incomplete as written; append an MP4 filename so it can actually run.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/llm/src/preprocessor/media/decoders/video.rs` around lines 318 - 320,
Update the ffmpeg regeneration command in the video fixture documentation to
append an MP4 output filename after the encoding options, making the example
executable while preserving its existing input, codec, frame-count, and keyframe
settings.

Source: Path instructions

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 40.90% (-8.27%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 09391d2 | Docs | Give us feedback!}

[dmitry-tokarev-nv]

Harrison — thanks for the SGLang codec removal. It lines up exactly with the rc.10 codec audit I ran: SGLang had the most codec sources of the three images (in-tree ffmpeg + a vendored ai_dynamo_runtime.libs copy + PyAV av.libs + decord2.libs + opencv), so removing ffmpeg + purging those wheels is the right call. A few follow-ups on this PR:

CI test failures (root-caused):

• sglang-runtime / Test → test_sglang_deployment[video_e_pd_qwen] fails with No module named 'decord'. SGLang's video-multimodal serving path still imports decord, which your commit (correctly) removed. The test needs to be removed/skipped — SGLang video decode is intentionally gone now.
• trtllm-runtime / Test → test_deployment[video_diffusion] fails: h264_nvenc encode→mp4 "Broken pipe", empty ffmpeg stderr. This is the diffusion video encode path. I can't tell from the log whether it's a regression from my decoder allowlist or pre-existing NVENC flakiness (the empty stderr leans toward the latter). It needs a GPU repro — build the allowlist ffmpeg and run the exact command: ffmpeg -f rawvideo -i - -vcodec h264_nvenc -pix_fmt yuv420p out.mp4. If it is the allowlist, the likely fix is adding --enable-parser=h264 (a parser, not a codec — no royalty impact) so the mp4 muxer can pull SPS/PPS from the h264_nvenc stream. I'm happy to run this down if you'd like.

Open review nits from my original commits:

• OPS-7665 appears in container/templates/wheel_builder.Dockerfile:303 and lib/llm/src/preprocessor/media/README.md:56 — Devin/CodeRabbit flag it (repo bans Linear IDs in source). Should drop both.
• lib/llm/src/preprocessor/media/decoders/video.rs:318-320 — the ffmpeg regen command in the docstring is missing the output filename (CodeRabbit).

Devin's --disable-bsfs/VP9 comment is a non-issue: I validated VP9-in-mp4 decode against the exact allowlist build (which has --disable-bsfs) — all fixtures decoded to the correct frame counts. The vp9_superframe BSF isn't needed for the mov-demuxer + vp9-decoder + vp9-parser path.

Sibling PRs: #11607 (opencv removal) and #11632 (the compliance gate, which should merge last). Glad to take any of the above off your plate — just say the word.

[dmitry-tokarev-nv]

Addressed the review comments on my commits (pushed c45909b182, 642de0e3e1):

• Removed the internal ticket references from wheel_builder.Dockerfile and the media README.md (Devin/CodeRabbit).
• video.rs docstring ffmpeg regen command now includes the output filename so it's executable (CodeRabbit).
• trtllm video_diffusion encode failure (h264_nvenc → mp4, broken pipe): my decoder allowlist had dropped the h264 parser, which the mp4 muxer needs to package the h264_nvenc bitstream. Added it back (--enable-parser=vp8,vp9,h264). A parser is not a codec — no software H.264 encode/decode is reintroduced; h264_nvenc stays the sanctioned HW encoder. Caveat: the failure log had an empty ffmpeg stderr, which can also indicate an NVENC/env flake, so the trtllm GPU test is the real confirmation — but the parser is a correct, zero-risk addition regardless.
• Devin's --disable-bsfs/VP9 concern: validated a non-issue — I decoded all VP9-in-mp4 fixtures with the exact allowlist build (which has --disable-bsfs); the vp9_superframe BSF isn't needed for the mov-demuxer + vp9-decoder + vp9-parser path.

Still open (not a review comment — a consequence of the SGLang codec removal): the SGLang video_agg_qwen and video_e_pd_qwen serve tests fail with No module named 'decord'. SGLang no longer has any video decoder (decord/av/ffmpeg all removed), so these two video-multimodal tests can't pass and need to be skipped or removed. Happy to push a skip with a clear reason if you'd like, @hsaturleyhal — flagging since it's your removal + implies SGLang drops video-multimodal input.

[dmitry-tokarev-nv]

Added the vLLM codec cleanup (eaabac23c6) so #11632's gate can pass on vllm-runtime:

• Removes the video-decode wheels inherited from the vllm-openai base — av (PyAV), opencv-python(-headless), decord/decord2, torchcodec, and PyNvVideoCodec — each of which bundles a full software ffmpeg (PyAV/decord also ship GPL libx264/libx265). Dynamo's vLLM component imports none of them.
• Keeps the in-tree LGPL ffmpeg + imageio-ffmpeg for the omni HW video-encode path (h264_nvenc, sanctioned).
• Guards fail the build if cv2/av survive.

Lower risk than the SGLang removal: no vLLM serve/deployment test decodes video (the video_url references in the vLLM tests are request-parser unit mocks + omni output formatting, not decode), so this shouldn't add a CI test failure. It does drop software video-decode input from vLLM — the same compliance trade-off as SGLang.

Note: the #11632 scan also flagged PyNvVideoCodec — if it's present on the trtllm image too (it's new since my rc.10 audit), trtllm will need the same strip. Worth a check when trtllm's scan runs.

[dmitry-tokarev-nv]

Combined into #11836 (with the vLLM codec cleanup added on top). Keeping this open for the review thread + CI. Review comments addressed (OPS refs removed, video.rs regen command fixed, Devin --disable-bsfs/VP9 validated as a non-issue). Open CI items: sglang Test still fails on the removed decord (video_agg_qwen/video_e_pd_qwen need a skip — @hsaturleyhal, your call); trtllm/vllm Test are running (the h264 parser fix should clear the trtllm encode failure).

[dmitry-tokarev-nv]

superseeded by #11836