[krishung5]

Summary

Forward ordered video URLs and data URIs through the native TensorRT-LLM OpenEngine sidecar. Preserve mixed image/video prompt order and explicit caller UUIDs across aggregated and disaggregated requests. Discover video support during startup and require both the portable multimodal capability and the engine's video modality declaration.

The image foundation (#16613) is merged. This PR adds video support to main's merged image implementation. NVIDIA/TensorRT-LLM#20654 is also merged and included in the official nightly used by the integration job. No personal-fork engine pin or runtime source overlay is required.

Nightly GPU coverage

Extend the merged upstream-nightly workflow with independent image and video GPU cases. Both use the same automatically resolved latest regular upstream RC/final nightly, pinned by digest with the installed source commit recorded. Keep the MPI hostname/root settings and temporary Cargo target directory from #16613.

The video case uses real Qwen3-VL-2B weights and the checked-in triangle MP4 as a data URI. It validates the fixture checksum and pinned tokenizer, requires a triangle description and engine-expanded prompt/completion usage, and writes response JSON, engine logs, Rust test output, image provenance, and JUnit artifacts. The exact live Rust test must execute once: zero executed or ignored tests fail the harness. These jobs run on scheduled nightlies and manual dispatches of nightly-ci.yml, rather than on every merge or PR push.

CPU harness tests run in PR CI. Full HTTP frontend coverage using Dynamo's shared in-process runtime container is split into #17673, stacked on the image HTTP follow-up #17414, pending the runtime upgrade. This nightly test covers the native sidecar-to-OpenEngine path.

The merged image startup policy is retained: DYN_MM_ALLOW_REMOTE_URLS=0 rejects HTTP(S) image and video sources before engine dispatch while allowing inline data URIs. Mock-engine regression coverage exercises both modalities.

Validation

The latest local fix holds trailing hidden stop tokens until the engine sends terminal usage. Otherwise the frontend can detect EOS and close the stream before receiving expanded prompt usage. A unit regression covers hidden EOS with no visible stops and preserves terminal prompt/cached-prompt usage. This is the existing narrow fix from image-routing work, without the routing feature.

• 381 sidecar/testkit unit tests passed across the five sidecar/testkit crates; six pre-existing live-only TRT-LLM cases remain ignored in that unit invocation.
• All 11 mock-engine integration tests passed, with no skips.
• Clippy with warnings denied, formatting, whitespace and targeted pre-commit hooks passed.
• Four full HTTP video cases passed locally: HTTP URL and data URI, each with JSON and SSE responses, through Dynamo frontend → native sidecar → OpenEngine → real Qwen3-VL-2B inference. Prompt usage is now 193, matching the engine, rather than the previous placeholder count of 21. Checks cover the expected triangle subject, complete SSE, prompt usage and completion/total consistency; they do not certify every motion/count detail.

This GPU run used the unmodified official nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc3.dev202610300406 image (source ac6fb72d9fa1caf2b2d730aa4797727e6b3f6d23), on one RTX 5880 Ada GPU with one aggregated worker. An isolated container used matching userspace driver libraries to work around the host library mismatch; host configuration was unchanged. Live disaggregated serving and multiworker routing were not rerun for this accounting fix.

Completion accounting follows the shared frontend's delivered-token convention: the engine reports 62 including hidden EOS; HTTP reports 61 after that EOS is removed, and total 254 (=193+61). The checks explicitly verify the removed EOS and do not claim raw engine/HTTP completion-counter equality.

The official-nightly video GPU job passed at the prior published stack. The new fix has local GPU validation; CI for the new head is separate. Full HTTP CI remains in #17673, stacked on #17414.

Relates to DIS-2971; this PR covers video passthrough and does not close the full image/video/audio scope. MM-aware video routing is outside this PR.

[KrishnanPrash]

Thank you!