Skip to content

test(trtllm): cover video in shared-runtime sidecar HTTP E2E - #17673

Draft
krishung5 wants to merge 1 commit into
trtllm-sidecar-image-e2efrom
trtllm-sidecar-video-e2e
Draft

krishung5 wants to merge 1 commit into
trtllm-sidecar-image-e2efrom
trtllm-sidecar-video-e2e

Conversation

@krishung5

Copy link
Copy Markdown
Contributor

Overview

Extend the image HTTP E2E scenario from #17414 with a video request through Dynamo's public API, Rust frontend, TRT-LLM sidecar, and OpenEngine. The response must identify the triangle in the checked-in MP4.

Draft, stacked on #17414. Also depends on video wiring #16870 and a shared Dynamo TRT-LLM runtime container containing the merged upstream video support from NVIDIA/TensorRT-LLM#20654. Refresh this branch with the video wiring before GPU validation. No personal-fork engine pin or runtime source overlay is added.

Details

  • Reuse the image scenario's single Qwen3-VL-2B engine for both requests; rename the scenario to trtllm_aggregated_multimodal.
  • Start the local media fixture server for this scenario and enable local URL access only in its test environment.
  • Install the validated OpenCV decoder package for the test and retain post-merge scheduling, the one-GPU marker, and the existing KV budget.
  • Reuse test(trtllm): add post-merge image sidecar E2E coverage #17414's policy that required TRT-LLM CI fails if OpenEngine is missing rather than silently skipping.

The official-upstream nightly video test belongs to #16870. This follow-up covers the full HTTP path using Dynamo's shared runtime container.

Validation

  • Seven existing runtime-policy/probe CPU cases passed in an isolated copy, without loading an inference backend.
  • Targeted pre-commit checks and whitespace checks passed.
  • GPU HTTP serving and the combined scenario's memory profile remain unvalidated. The local host reports an NVIDIA driver/library mismatch; the shared runtime upgrade and prerequisites must be validated before merge.

Where should the reviewer start?

trtllm_aggregated_multimodal and its media-server setup in tests/serve/test_sidecar.py. This PR changes only that file relative to #17414.

Related Issues

Signed-off-by: krishung5 <krish@nvidia.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

✅ Dynamo PR CI passed — run 41019625031 (attempt 1) on a242fb2b7a

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

Posted automatically by Devin for run 41019625031. Updated on every full-CI run of this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant