[TRTLLM-15277][feat] BREAKING: Redesign VisualGen reference inputs and server-to-worker IPC

[luyiyun1021]

Dev Engineer Review

• Added typed image_reference, video_reference, and audio_reference support.
• Added MediaRef, Role, RoleSpec, and RefSlotSpec APIs.
• Added reference-slot metadata exchange and validation.
• Migrated supported VisualGen pipelines from legacy reference parameters.
• Retained deprecated input_reference routing for compatibility.
• Updated serving logic for uploads, base64/data URIs, roles, lists, modality checks, and media storage.
• Updated examples, documentation, configurations, and API stability references.
• No test-list files were changed.

QA Engineer Review

• Updated VisualGen parameter, serving endpoint, end-to-end, and utility tests.
• Added coverage for typed references, modality validation, role and cardinality validation, deprecated alias routing, precedence, media storage, cleanup, and encoded inputs.
• Updated multipart tests for image_reference and video_reference.
• Test-list coverage was not provided.
• Verdict: needs follow-up.

Description

Replaces VisualGen's two untyped conditioning-input channels with a typed, per-modality reference API (Scheme C), and makes a reference travel as bytes from the client to the worker instead of being written to disk and re-read per rank.

Before, "which reference does this model take, and in what role?" was implicit and undiscoverable. The Python API had one params.image field with no notion of modality or role; serve had one input_reference whose modality was guessed by sniffing the payload — an image landed on params.image, a video was smuggled through extra_params["video"]. Both are removed (breaking).

Details below.

• Typed slots — image_reference / video_reference / audio_reference replace params.image and input_reference. The field name fixes the modality, so nothing is sniffed to route a payload. Each item is a MediaRef(content, format, role) (serve: MediaReferenceItem), re-exported from tensorrt_llm and tensorrt_llm.visual_gen; a bare string or bare bytes is rejected with a message naming the replacement rather than coerced.
• Explicit format, nothing guessed — required on every reference. This deletes two stacked guesses: a bare string used to be decoded as base64 and, failing that, read as a local file, so malformed base64 could silently reach the filesystem; separately _is_local_path decided passthrough-vs-materialize with os.path.exists, so a mistyped path silently reclassified as base64. A declared path that does not exist is now a clean client error. MediaReferenceItem is also promoted to a gated api-stability model — it was previously invisible to the gate, so a required field on it would have landed with green CI and an empty api_stability diff.
• ref_slot_specs — a model declares what it accepts — each pipeline states its slots, roles, and per-role arity (RefSlotSpec / RoleSpec), travelling worker→coordinator in the READY handshake alongside extra_param_specs. validate_visual_gen_params enforces it at the engine entry and as a serve pre-flight, so a missing required reference, an excess one, or an unsupported role is a deterministic 400 at the boundary instead of a crash deep in the worker. role is required only where a slot declares more than one (Wan I2V first/last frame); a single-role slot infers it, so you never write role="reference" for FLUX.2. min >= 1 marks a reference required; min == 0 leaves the slot optional.
• Bytes end to end, one decode path — prepare_reference_slots resolves every declared wire form to raw bytes on the coordinator, at a single choke point, before the request is broadcast, and rewrites format to "bytes" so a stale format cannot hand a worker a path while still claiming base64. Nothing touches disk, so there is nothing to reclaim: the on_finish hook, cleanup_reference_files, resolve_media_storage_path and the VisualGenResult cancellation branch that existed only to fire that hook are gone. Each pipeline decodes the bytes with PIL at its own call site, keeping the exact alpha handling it had before — dropped for the sites that read convert("RGB"), composited onto white for Cosmos3, which reached load_image with a path. Per-model checks that only restated the choke point's own sniff are dropped.
• Coordinator → rank0 over shared memory — the payload rides as a SharedTensorContainer handle instead of inside the request pickle, which would copy every reference byte to cross one process boundary. rank0 restores it to bytes before the rank broadcast, because a handle is consumed exactly once and minting one per rank would free the block N-1 times. On the rank hop the payload is lifted out of the object and broadcast as a raw tensor beside a slim request.
• Migrated pipelines — FLUX.2, Wan I2V (first/last), Wan TI2V, LTX-2 (single + two-stage), Qwen-Image-Edit, Qwen-Image-Layered, Cosmos3 (image + video), HunyuanVideo 1.5.
• Image edit on the same path, without losing its checks — /v1/images/edits no longer writes its inputs to media storage. Each entry is decoded at the boundary — the same read a multipart upload already gets — and handed to the engine as MediaRef(format="bytes"). Every check the materialize path performed stays: paths and URLs refused, per-image and total byte limits, PNG/JPEG only. Only the disk write is gone, which removes cleanup_materialized_conditioning_inputs and the four helpers behind it. The request schema is untouched — OpenAI's image field has nowhere to declare a wire form, so the format is implied by the transport (str → base64, upload → bytes) and there is no role.
• Cosmos3 keeps its transfer controls in extra_params — only the conditioning clip moves to the typed video_reference slot, read where the transfer feature reads it so source dimensions still drive the resolved output size. edge / blur / depth are unchanged.
• input_reference kept as a deprecated alias — still sniff-routed to image_reference / video_reference. It has no wrapper object to hold a format, so it gains the sibling input_reference_format; a missing one now defaults to base64 with a warning rather than a 422, so callers written against the old field keep working.
• format="path" can be turned off — a path reference asks the server to read its own disk, which is what a co-located client wants and a remote one has no business doing. Allowed by default, disabled with TRTLLM_DISALLOW_LOCAL_MEDIA_PATH=1 (HTTP boundary only; the local Python API is unaffected). One switch, both directions: main already used that name for response_format="path", which discloses where the server wrote, and the two now share a single predicate. A path is stat()ed first and must be a regular file, so a character device or FIFO cannot turn a request into an unbounded read.

Shared-memory blocks

A block is freed by consuming its handle, and rank0 consumes each one as soon as it takes the request off the queue. Nothing else reclaims: measured on a B200, torch_shm_manager unlinks every block the producer registered when that process dies, under SIGTERM and SIGKILL alike, so a crash leaks nothing.

An earlier revision tracked sent handles and released them when a worker was seen dead. Measuring that path showed it barely fires — ZeroMqQueue is a zmq.PAIR socket, so with the workers gone the next put() blocks instead of failing, and the release check that followed it in the same loop never runs again. It is removed; a switch that only works when no further request arrives reads as if the case were handled.

Two findings from that measurement, both outside this PR: a dead worker silently deadlocks the sender thread with no error or timeout, and SharedTensorContainer has no producer-side release — freeing a handle from outside the library means reaching into method_key, storage_handle and the base64 filename, and a CUDA handle has no unlinkable name at all. Filed as follow-ups.

Wire forms

format	Content	Notes
path	a local file	bare path or file://; must exist; passes through in place (not copied, not deleted)
url	an http(s) URL	fetched through the SSRF-guarded loader
base64	base64 text	a data: URI is also accepted
bytes	raw bytes	Python API only; rejected over JSON (422) — use base64 or a multipart upload

Usage change

Python API — before / after

Before:

params = vg.default_params
params.image = "ref.png"                       # untyped; modality/role implicit

After:

from tensorrt_llm import MediaRef

params = vg.default_params

# Every reference declares the wire form of its content; nothing is guessed.
params.image_reference = MediaRef(content="ref.png", format="path")
params.image_reference = [MediaRef(content="a.png", format="path"),      # multi-reference
                          MediaRef(content=raw_bytes, format="bytes")]   # (FLUX.2 / Qwen-edit)

# Multi-role slot (Wan I2V first/last): role is required to disambiguate.
params.image_reference = [MediaRef(content="start.png", format="path", role="first_frame"),
                          MediaRef(content="end.png",   format="path", role="last_frame")]

# V2V (Cosmos): video is now a typed field, not extra_params["video"].
params.video_reference = MediaRef(content="clip.mp4", format="path")

trtllm-serve — before / after

Before — one input_reference; the server sniffed the bytes to decide image-vs-video:

curl .../v1/videos -F "prompt=..." -F "input_reference=@start.png"   # sniffed -> image
curl .../v1/videos -F "prompt=..." -F "input_reference=@clip.mp4"    # sniffed -> extra_params["video"]

After — modality is declared by the field name; JSON additionally supports role and lists:

# multipart file upload (single ref, no role)
curl .../v1/videos -F "prompt=..." -F "image_reference=@start.png"
curl .../v1/videos -F "prompt=..." -F "video_reference=@clip.mp4"

# JSON body: a {content, format, role} object or a list of them. format is path | url | base64.
curl .../v1/videos -H 'content-type: application/json' -d '{
  "prompt": "...",
  "image_reference": [
    {"content": "<base64>", "format": "base64", "role": "first_frame"},
    {"content": "https://example.com/end.png", "format": "url", "role": "last_frame"}
  ]
}'

Each reference carries a required format (path / url / base64 / bytes) declaring how to read its content, so no wire form is ever sniffed. A multipart file upload needs no format — the transport implies it, so every existing curl -F / SDK upload keeps working unchanged. format is materialized to a local file and handed to the pipeline as a MediaRef holding that path; decode stays model-specific in the worker. url reuses the LLM multimodal path's SSRF-guarded fetch (private-address block, redirect re-validation, timeout, size cap); path accepts a bare path or a file:// URI and base64 accepts a data: URI, but neither prefix is needed any more since format already states the form. Content is validated against the declared modality (image signature + HEIF/AVIF reject; video container sniff), so a mismatch is an HTTP 400 up front. Materialized reference files are input-only and are reclaimed automatically when the request completes or fails — synchronously inline, and in the async background task (including on cancellation) — so conditioned requests do not accumulate in TRTLLM_MEDIA_STORAGE_PATH; delete_video needs no extra step because completion/failure cleanup runs first.

Performance

Measured over the real ZeroMqQueue and a real cuda:nccl,cpu:gloo group, medians of 15 trials after 3 warmups, with the alternatives interleaved so machine drift hits them equally.

The full path, against the transport this replaces. Every column is timed from "the coordinator holds the resolved payload" to "every rank holds a decoded image", so each covers the coordinator-side prep, both hops, and the decode on every rank.

A path-based transport only works if the coordinator and the workers share a filesystem, so it is measured both ways: against local disk, where it is at its best because the coordinator's own write leaves the file in page cache, and against the NFS mount it actually requires as soon as the workers are not on the coordinator's node.

ranks	reference	path, local disk	path, NFS	shared memory + bytes
1	0.5 MB	5.3 ms	18.8 ms	4.5 ms
1	2 MB	9.1 ms	30.9 ms	10.3 ms
1	8 MB	27.1 ms	80.4 ms	27.9 ms
1	32 MB	100.8 ms	283.5 ms	103.3 ms
8	0.5 MB	10.9 ms	22.5 ms	12.1 ms
8	2 MB	13.5 ms	40.2 ms	20.0 ms
8	8 MB	36.7 ms	96.9 ms	46.2 ms
8	32 MB	123.1 ms	342.6 ms	151.5 ms

Against local disk this is level at one rank — faster at 0.5 MB, where decoding from memory beats PIL reading a file and the broadcast hop is skipped, and within 2.5 ms everywhere else. At eight ranks it runs 1–9 ms behind for the sizes that actually occur (a PNG or an MP4 conditioning clip, ~0.5–8 MB) and 28 ms behind at 32 MB, because bytes on a collective will always cost more than a filename: the broadcast phase alone is 2.9–34.3 ms for the payload against 2.0–2.4 ms for a path.

Against NFS it is 2.0–2.3x faster at eight ranks and up to 2.9x at one — 96.9 → 46.2 ms for 8 MB at eight ranks, 342.6 → 151.5 ms at 32 MB — because the path transport pays a network write on the coordinator (8–62 ms against 0.5–10 ms local) and then a network read per rank. That comparison is conservative: all eight ranks here share one node's NFS client cache, whereas a real multi-node deployment would fetch on each node.

Keeping the payload out of the object pickle is what holds the local-disk gap down. Isolating the broadcast, sending the payload inside broadcast_object_list against sending it as a raw tensor beside a slim object: 9.0 → 4.9 ms for 8 MB at 8 ranks, 49.4 → 22.9 ms for 32 MB at 8 ranks, 39.4 → 18.4 ms for 32 MB at 2 ranks, and 20.2 ms → nothing at all for 32 MB at 1 rank, where there is no hop but the pickle would still copy.

One caveat: the totals include a small instrumentation collective that every column pays equally, so absolute values are inflated at 8 ranks while the differences stand.

Test Coverage

Unit tests updated and passing: 465 in the reference / serve / decode gate (test_visual_gen_params.py, test_visual_gen_utils.py, test_media_decode.py, test_flux2_image_conditioning.py, test_trtllm_serve_endpoints.py, test_cosmos3_pipeline.py) plus api_stability. Coverage includes reference resolution per declared format with every form resolving to identical bytes, format/content mismatches erroring instead of falling back, the format-rewrite broadcast-consistency property, zero filesystem writes, role / list / wrong-modality / arity validation, the alpha semantics of each decode site, and NVDEC window selection against a checked-in H.264 fixture.

The transport has its own tests: the handle round trip is byte-identical and survives a real pickle, the payload verifiably leaves the request pickle, ref_sizes lets a peer size its buffers from the object hop alone, and a count mismatch raises instead of silently emptying references. Shared-memory blocks are identified by the filename inside each handle rather than by scanning /dev/shm, which is machine-global, and are checked on the normal path, when a request never ships, when a restore fails partway, and when the workers exit before reading what was sent — each of those fails when its fix is reverted.

Golden runs on a real B200 (manual — not yet committed CI tests). For each model the same seeded generation runs twice with the same reference declared two different ways (format="path" vs format="base64"); under the bytes-canonical transport both must resolve to identical bytes, so the outputs must be bitwise equal, and each run also asserts the output is non-degenerate so a pair of failures cannot pass as a match. Wan2.2-I2V-A14B, FLUX.2-dev, Cosmos3-Super I2V and Cosmos3-Super V2V all pass on the current head, and Cosmos3-Super V2V + Wan2.2-I2V-A14B pass at ulysses_size=2 to exercise the broadcast hop. test_trtllm_serve_e2e.py::TestWanImageToVideo (real weights, sync + async lifecycle, mp4 + avi) passes. LTX-2 decodes its reference and generates a non-degenerate video, but is nondeterministic run-to-run — a path-vs-path control with a fixed seed also differs — so bitwise equivalence is not measurable there.

Two real defects were caught by these golden runs rather than by unit tests, both from the bytes migration: str type guards in pipeline_wan_i2v / pipeline_cosmos3 that rejected bytes, and a second Cosmos3 I2V decode dispatch (isinstance(image, str)) that no unit test reached.

PR Checklist

• PR description clearly explains what and why.

• PR Follows TRT-LLM CODING GUIDELINES.

• Test cases are provided for new code paths.

• Any new dependencies have been scanned for license and vulnerabilities.

• CODEOWNERS updated if ownership changes.

• Documentation updated as needed.

• Update tava architecture diagram if significant design change.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Visual generation now uses typed image, video, and audio references. Pipelines declare reference slots and validate roles and counts. Serving code materializes references into MediaRef values. Examples, documentation, and tests use the new fields.

Changes

Visual-generation reference inputs

Layer / File(s)	Summary
Reference contracts and validation tensorrt_llm/visual_gen/..., tensorrt_llm/serve/openai_protocol.py, tensorrt_llm/_torch/visual_gen/pipeline.py, tensorrt_llm/__init__.py	Adds MediaRef, modality-specific reference fields, MediaReferenceItem, and reference-slot validation for modalities, roles, and counts.
Serving and media materialization tensorrt_llm/serve/visual_gen_utils.py, tensorrt_llm/serve/openai_video_routes.py	Decodes, validates, stores, and normalizes image, video, and audio references. Deprecated input_reference routing remains supported.
Pipeline reference-slot integration tensorrt_llm/_torch/visual_gen/executor.py, tensorrt_llm/_torch/visual_gen/models/*, tensorrt_llm/visual_gen/visual_gen.py	Pipelines declare supported reference slots and consume typed references during inference. Executor metadata carries slot specifications to validation.
Examples, documentation, and test coverage examples/visual_gen/..., docs/source/models/visual-generation.md, tests/unittest/_torch/visual_gen/..., tests/integration/..., tests/scripts/...	Updates examples, API documentation, performance configurations, and tests to use modality-specific references and verify materialization and validation behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🔵 Low · up to 477b8

The PR introduces typed, role-aware media references with boundary validation. Remaining risks are bounded to incomplete example annotations, documentation that does not fully match supported input behavior, and synchronous examples that may wait indefinitely without a timeout; the PR is mergeable with explicit owner awareness or follow-up.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant openai_video_routes
  participant visual_gen_utils
  participant VisualGen
  Client->>openai_video_routes: submit image_reference or video_reference
  openai_video_routes->>visual_gen_utils: decode and materialize media
  visual_gen_utils-->>openai_video_routes: return MediaRef values
  openai_video_routes->>VisualGen: validate references with ref_slot_specs
  VisualGen->>VisualGen: run pipeline inference

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 45.45% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the breaking VisualGen reference-input redesign and server-to-worker IPC change. It uses a valid ticket and feature prefix.
Description check	✅ Passed	The description is complete and relevant. It explains the motivation, implementation, breaking API changes, migration scope, performance results, test coverage, and checklist items.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 8

🧹 Nitpick comments (1)

tensorrt_llm/serve/visual_gen_utils.py (1)

114-114: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Annotate the duck-typed reference parameters.

_reference_payload_and_role does not annotate ref. _build_reference_list does not annotate value or ref_cls, and it declares no return type. The repository guidelines require an annotation on every function, precise types instead of Any, and a Protocol for structural interfaces when no suitable ABC exists.

A union alias plus a type[ImageRef | VideoRef | AudioRef] bound keeps the call sites checkable.

♻️ Proposed annotations

+# A single raw HTTP reference: base64/data-URI string, multipart upload,
+# or a typed reference item from the request model.
+RawReference = Union[str, "UploadFile", "ImageReferenceItem", "VideoReferenceItem"]
+
-def _reference_payload_and_role(ref, data_field: str) -> tuple[bytes, Optional[str]]:
+def _reference_payload_and_role(
+    ref: RawReference, data_field: str
+) -> tuple[bytes, Optional[str]]:

 def _build_reference_list(
-    value, *, modality: str, data_field: str, ref_cls, id: str, media_storage_path: Optional[str]
-):
+    value: Union[RawReference, list[RawReference], None],
+    *,
+    modality: str,
+    data_field: str,
+    ref_cls: type,
+    id: str,
+    media_storage_path: Optional[str],
+) -> Optional[list]:

As per coding guidelines: "Annotate every function, use None for procedures, avoid unnecessary Any and type: ignore ... use Protocol for structural interfaces when no suitable ABC exists."

Also applies to: 167-169

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/serve/visual_gen_utils.py` at line 114, Annotate every parameter
and return value in _reference_payload_and_role and _build_reference_list,
including ref, value, and ref_cls. Define a precise union alias for ImageRef,
VideoRef, and AudioRef instances, use it for duck-typed references, and
constrain ref_cls as a type of that union; introduce a Protocol if the
structural interface requires it, avoiding Any and type ignores.

Source: Coding guidelines

🔇 Additional comments (44)

tensorrt_llm/serve/visual_gen_utils.py (4)

93-111: LGTM!

---

159-164: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

⚠️ Unverified finding
Sandbox verification was unavailable.

Constrain the written path to media_storage_path.

Static analysis flags line 161 as a path-traversal sink. The current callers pass a server-generated video_id (f"video_{uuid.uuid4().hex}"), so no client-controlled segment reaches ref_id today. parse_visual_gen_params accepts id as a plain parameter, so a future caller could pass a client value and escape the storage directory.

Add a containment check so the guarantee does not depend on caller discipline.

🛡️ Proposed hardening

     if media_storage_path is None:
         raise ValueError(f"media_storage_path is required to store the {modality}_reference.")
-    ref_path = os.path.join(media_storage_path, ref_id)
+    storage_root = os.path.realpath(media_storage_path)
+    ref_path = os.path.realpath(os.path.join(storage_root, ref_id))
+    if os.path.commonpath([storage_root, ref_path]) != storage_root:
+        raise ValueError("reference id resolves outside the media storage directory.")
     with open(ref_path, "wb") as f:
         f.write(payload)
     return ref_path

Run the following script to confirm that no caller passes a client-controlled id:

---

177-193: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Reference files are never reclaimed.

_build_reference_list writes each decoded payload to media_storage_path as it iterates. Two paths leak files:

1. Partial failure. If entry k fails _materialize_reference, entries 0..k-1 are already on disk. The ValueError propagates to the route and returns HTTP 400. The written files stay.
2. Successful jobs. Reference files use the name {video_id}_{data_field}_ref_{i}. delete_video removes only job.output_paths / job.output_path, or probes {video_id}{ext} and {video_id}_0{ext} against _KNOWN_VIDEO_OUTPUT_SUFFIXES. Neither pattern matches a reference file.

A long-running server accumulates reference bytes with no reclaim path. A client that repeatedly sends a valid reference plus an invalid one grows storage without ever creating a job.

Clean up the partial writes at minimum, and record the reference paths on the job so delete_video can remove them.

🧹 Proposed cleanup for partial failures

     raw_items = value if isinstance(value, list) else [value]
     refs = []
-    for i, item in enumerate(raw_items):
-        payload, role = _reference_payload_and_role(item, data_field)
-        ref_path = _materialize_reference(
-            payload,
-            modality=modality,
-            ref_id=f"{id}_{data_field}_ref_{i}",
-            media_storage_path=media_storage_path,
-        )
-        kwargs = {data_field: ref_path}
-        if role is not None:
-            kwargs["role"] = role
-        refs.append(ref_cls(**kwargs))
+    written: list[str] = []
+    try:
+        for i, item in enumerate(raw_items):
+            payload, role = _reference_payload_and_role(item, data_field)
+            ref_path = _materialize_reference(
+                payload,
+                modality=modality,
+                ref_id=f"{id}_{data_field}_ref_{i}",
+                media_storage_path=media_storage_path,
+            )
+            written.append(ref_path)
+            kwargs = {data_field: ref_path}
+            if role is not None:
+                kwargs["role"] = role
+            refs.append(ref_cls(**kwargs))
+    except (ValueError, OSError):
+        for path in written:
+            with contextlib.suppress(OSError):
+                os.remove(path)
+        raise
     return refs

Run the following script to confirm that no existing code deletes reference files:

---

265-301: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Confirm that image requests do not accept image_reference.

The reference block sits inside the VideoGenerationRequest branch. The ImageGenerationRequest branch at lines 236-238 sets only num_images_per_prompt. Two consequences follow if ImageGenerationRequest declares image_reference:

1. A client can send image_reference to /v1/images/generations and the field is silently dropped. The request succeeds and produces an unconditioned image.
2. openai_image_generation calls parse_visual_gen_params(request, image_id, self.generator) without media_storage_path, so the image path could not materialize a reference even if the branch were reached.

The PR migrates FLUX.2, Qwen-edit, Qwen-Layered, and Cosmos3, which serve through /v1/images/generations.

Run the following script to check whether ImageGenerationRequest declares the reference fields:

tensorrt_llm/serve/openai_video_routes.py (1)

340-345: LGTM!

tensorrt_llm/serve/openai_server.py (1)

2582-2583: LGTM!

tensorrt_llm/visual_gen/params.py (3)

15-58: LGTM!

Also applies to: 121-137, 198-198

---

283-320: LGTM!

---

59-68: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Add complete, precise annotations to new reference functions.

New functions use untyped parameters or return values. _normalize_refs also uses Any, bare type, and an unparameterized list. Define the concrete request and pipeline-output types, then annotate each new function without Any.

• tensorrt_llm/visual_gen/params.py#L59-L68: use a bounded TypeVar and parameterized list types for _normalize_refs.
• tensorrt_llm/visual_gen/params.py#L138-L151: annotate validator input and normalized reference-list return types.
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py#L396-L411: annotate ref_slot_specs and infer with the existing request and output types.
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py#L407-L431: annotate ref_slot_specs and infer with the existing request and output types.

As per coding guidelines, “Annotate every function” and “avoid unnecessary Any.”

tensorrt_llm/_torch/visual_gen/pipeline.py (2)

63-84: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Constrain reference slot declarations.

RoleSpec permits negative bounds and max < min. RefSlotSpec permits an empty roles list. An empty declaration can cause validate_visual_gen_params to index role_specs[0] instead of returning a validation error.

Constrain modality and role to their supported literals. Require min >= 0, max >= 0 when set, at least one role, unique roles, and max >= min.

As per coding guidelines, use “Literal for fixed values” and “built-in constrained numeric and length types where applicable.”

---

356-365: LGTM!

tensorrt_llm/serve/openai_protocol.py (1)

1676-1763: LGTM!

tests/unittest/api_stability/references/trtllm_serve_api.yaml (1)

1468-1485: 📐 Maintainability & Code Quality

⚠️ Unverified finding
Sandbox verification was unavailable.

Verify API validation coverage.

Test coverage summary: No test function changed in this manifest. No integration test-list entry is required for this manifest-only change. Coverage verdict: needs follow-up until unit tests confirm image, video, audio, role, and arity validation for VideoGenerationRequest.

As per path instructions, “Always produce a test coverage summary, even if no issues are found.”

tensorrt_llm/visual_gen/__init__.py (1)

57-57: LGTM!

Also applies to: 81-83, 116-118

tensorrt_llm/__init__.py (1)

65-67: LGTM!

Also applies to: 108-110, 179-181

tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py (1)

39-39: LGTM!

tensorrt_llm/visual_gen/visual_gen.py (1)

31-31: LGTM!

Also applies to: 41-41, 295-304, 421-421

tensorrt_llm/_torch/visual_gen/executor.py (1)

372-372: LGTM!

Also applies to: 416-416, 677-677, 1033-1033

tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py (1)

30-30: LGTM!

Also applies to: 49-49, 344-383

tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py (1)

47-47: LGTM!

Also applies to: 383-410

tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py (1)

23-28: LGTM!

Also applies to: 1392-1410

tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py (1)

1228-1242: LGTM!

tensorrt_llm/_torch/visual_gen/models/qwen_image/pipeline_qwen_image_edit.py (1)

21-21: LGTM!

Also applies to: 164-164, 353-354

tensorrt_llm/_torch/visual_gen/models/qwen_image_layered/pipeline_qwen_image_layered.py (1)

26-31: LGTM!

Also applies to: 741-743

examples/visual_gen/models/cosmos3/cosmos3.py (1)

188-188: LGTM!

Also applies to: 211-211

examples/visual_gen/models/flux2.py (1)

121-121: LGTM!

examples/visual_gen/models/qwen_image_edit.py (1)

67-67: LGTM!

examples/visual_gen/models/qwen_image_layered.py (1)

65-65: LGTM!

tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml (1)

38-38: LGTM!

tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py (1)

384-384: LGTM!

Also applies to: 421-421

tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py (1)

703-704: LGTM!

Also applies to: 818-830, 849-897, 1070-1082

tests/unittest/_torch/visual_gen/test_visual_gen_params.py (2)

55-57: LGTM!

Also applies to: 102-120, 304-306, 399-412, 1038-1041, 1360-1360

---

938-983: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Validate the actual Wan I2V slot declaration.

These tests validate no slot specification and then validate manually constructed specifications. They do not validate WanImageToVideoPipeline.ref_slot_specs.

Pass the pipeline declaration to validate_visual_gen_params. Add cases for its required image slot and rejected video slot. This will detect a wrong pipeline declaration before serve preflight uses it.

tests/unittest/_torch/visual_gen/test_visual_gen_utils.py (1)

268-307: 🎯 Functional Correctness

⚠️ Unverified finding
Sandbox verification was unavailable.

Verify audio-reference and data-URI coverage.

The supplied ranges cover image and video references through uploads, base64, roles, and cleanup. Confirm that the suite also covers audio_reference and data-URI materialization, as required by the new serve contract.

Also applies to: 327-462, 574-602

examples/visual_gen/models/wan_i2v.py (1)

26-26: LGTM!

Also applies to: 65-69

examples/visual_gen/serve/README.md (1)

289-289: LGTM!

Also applies to: 355-355, 364-369

examples/visual_gen/serve/async_video_gen.py (2)

85-89: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

⚠️ Unverified finding
Sandbox verification was unavailable.

Use a client contract that supports image_reference.

create_params["image_reference"] is passed as a top-level keyword to client.videos.create. The stock OpenAI Python client currently declares input_reference, not image_reference; this call raises before the multipart request unless the repository installs a patched client. (github.com)

Verify the installed client. If it is stock, use a request method or client schema that explicitly supports image_reference.

---

44-44: LGTM!

Also applies to: 54-54, 65-66, 272-272

examples/visual_gen/serve/sync_video_gen.py (1)

43-43: LGTM!

Also applies to: 53-73, 257-257

docs/source/models/visual-generation.md (2)

112-112: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Separate Python and serve input forms.

This sentence says that both APIs accept paths and raw bytes. The serve examples below document base64/data URIs and multipart uploads. Clarify the transport-specific forms so users do not send a filesystem path or raw bytes in a JSON request.

Proposed wording

-Conditioning references are supplied through the typed, per-modality fields `image_reference`, `video_reference`, and `audio_reference`. These fields share the **same names and shapes** across the Python API (`VisualGenParams`) and the serve request (`VideoGenerationRequest`), and each accepts a path, raw bytes, a single reference, or a list.
+Conditioning references use the typed, per-modality fields `image_reference`, `video_reference`, and `audio_reference`. In the Python API (`VisualGenParams`), each field accepts a path, raw bytes, a single reference, or a list. Serve requests use base64/data URIs or uploaded files, with role-bearing objects and lists supported in JSON.

Verify the wording against VideoGenerationRequest:

---

114-164: LGTM!

tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md (1)

153-153: LGTM!

tests/integration/defs/perf/visual_gen_perf_utils.py (1)

108-110: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

⚠️ Unverified finding
Sandbox verification was unavailable.

Verify the fallback for audio_reference.

The new reference contract includes audio_reference, but this fallback checks only image and video references. If audio-conditioned video is supported, a client config with only audio_reference and no explicit generation_mode is assigned to the t2v bucket.

Add audio_reference to the appropriate conditioned-mode predicate, or document and test the intentional exclusion.

tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml (1)

41-41: LGTM!

Also applies to: 101-101

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/visual_gen/serve/async_video_gen.py`:
- Around line 85-89: Close opened image files using local context managers in
both upload paths: in examples/visual_gen/serve/async_video_gen.py lines 85-89,
keep the OpenAI request within the context managing the file assigned for
image_reference; in examples/visual_gen/serve/sync_video_gen.py lines 89-91,
keep requests.post within the context managing the multipart file. Ensure each
handle closes after its request completes.
- Line 31: Update the input reference parameter annotation to str | None in both
examples/visual_gen/serve/async_video_gen.py (lines 31-31) and
examples/visual_gen/serve/sync_video_gen.py (lines 30-30), preserving their
existing None defaults.

In `@tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py`:
- Around line 274-287: Update the reference-slot validation used by infer() to
support a cross-slot exclusivity constraint, then declare image_reference and
video_reference as mutually exclusive in Cosmos3’s ref_slot_specs property.
Ensure requests containing both slots are rejected during preflight, before
reaching forward().
- Around line 274-287: Add the return annotation dict[str, RefSlotSpec] to each
ref_slot_specs property declaration in
tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py lines 274-287,
tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py lines 372-381,
tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py lines 1382-1390,
tensorrt_llm/_torch/visual_gen/models/qwen_image/pipeline_qwen_image_edit.py
lines 134-142, and
tensorrt_llm/_torch/visual_gen/models/qwen_image_layered/pipeline_qwen_image_layered.py
lines 247-255. Preserve each property's existing metadata contents.

In `@tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py`:
- Around line 396-411: Update WanPipeline.ref_slot_specs to return an empty
mapping unless is_wan22_5b is true; only the Wan 2.2 TI2V-5B variant should
expose the image_reference first_frame slot, so unsupported image requests fail
during preflight.

In `@tensorrt_llm/serve/openai_video_routes.py`:
- Around line 256-262: Update the multipart field handling in the form iteration
to retrieve all values for each key via form.getlist(key), preserving every
uploaded reference file before passing data to _build_reference_list and its
slot-count validation. Keep the existing uploaded-file detection and passthrough
behavior for each collected value.

In `@tensorrt_llm/visual_gen/params.py`:
- Line 282: Update the reference-validation condition around ref_slot_specs to
enter the block whenever ref_slot_specs is not None, including an empty mapping.
Preserve None as the only case that skips validation, so requests containing
references are rejected when the pipeline declares no slots.

In `@tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py`:
- Line 179: Update the mock configuration around ref_slot_specs to declare
optional image and video reference slots required by the happy-path tests,
including requests using image_reference and video_reference. Retain an empty
ref_slot_specs mapping only for tests that specifically verify reference
rejection.

---

Nitpick comments:
In `@tensorrt_llm/serve/visual_gen_utils.py`:
- Line 114: Annotate every parameter and return value in
_reference_payload_and_role and _build_reference_list, including ref, value, and
ref_cls. Define a precise union alias for ImageRef, VideoRef, and AudioRef
instances, use it for duck-typed references, and constrain ref_cls as a type of
that union; introduce a Protocol if the structural interface requires it,
avoiding Any and type ignores.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6600942e-69d2-4133-a7d6-22359d24e7ac

📥 Commits

Reviewing files that changed from the base of the PR and between 07b3e82 and 790d111.

📒 Files selected for processing (37)

• docs/source/models/visual-generation.md
• examples/visual_gen/models/cosmos3/cosmos3.py
• examples/visual_gen/models/flux2.py
• examples/visual_gen/models/qwen_image_edit.py
• examples/visual_gen/models/qwen_image_layered.py
• examples/visual_gen/models/wan_i2v.py
• examples/visual_gen/serve/README.md
• examples/visual_gen/serve/async_video_gen.py
• examples/visual_gen/serve/sync_video_gen.py
• tensorrt_llm/__init__.py
• tensorrt_llm/_torch/visual_gen/executor.py
• tensorrt_llm/_torch/visual_gen/models/cosmos3/defaults.py
• tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py
• tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py
• tensorrt_llm/_torch/visual_gen/models/qwen_image/pipeline_qwen_image_edit.py
• tensorrt_llm/_torch/visual_gen/models/qwen_image_layered/pipeline_qwen_image_layered.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py
• tensorrt_llm/_torch/visual_gen/pipeline.py
• tensorrt_llm/serve/openai_protocol.py
• tensorrt_llm/serve/openai_server.py
• tensorrt_llm/serve/openai_video_routes.py
• tensorrt_llm/serve/visual_gen_utils.py
• tensorrt_llm/visual_gen/__init__.py
• tensorrt_llm/visual_gen/params.py
• tensorrt_llm/visual_gen/visual_gen.py
• tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md
• tests/integration/defs/perf/visual_gen_perf_utils.py
• tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml
• tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml
• tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py
• tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
• tests/unittest/_torch/visual_gen/test_visual_gen_params.py
• tests/unittest/_torch/visual_gen/test_visual_gen_utils.py
• tests/unittest/api_stability/references/trtllm_serve_api.yaml

💤 Files with no reviewable changes (1)

• tensorrt_llm/_torch/visual_gen/models/cosmos3/defaults.py

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

tests/unittest/_torch/visual_gen/test_visual_gen_utils.py (1)

263-267: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for audio_reference.

parse_visual_gen_params now materializes three modalities, but this class tests only image_reference and video_reference. The audio branch in _materialize_reference is the only branch that skips signature sniffing, so it has distinct behavior and no test. Add at least one test that a base64 audio_reference is persisted byte-identical and that a missing media_storage_path raises.

Do you want me to draft those tests?

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/visual_gen/test_visual_gen_utils.py` around lines 263 -
267, Add audio_reference coverage to TestInputReferenceMaterialization: test
that a base64 audio reference is materialized byte-identically, and test that
omitting media_storage_path raises the expected error. Exercise
parse_visual_gen_params and the audio branch of _materialize_reference while
preserving the existing image and video tests.

tensorrt_llm/serve/visual_gen_utils.py (1)

114-114: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Annotate the remaining helper parameters and return types.

ref in _reference_payload_and_role, and value, ref_cls, and the return type in _build_reference_list carry no annotations. The coding guidelines require annotating every function. A union alias keeps the signature readable.

♻️ Suggested annotations

-def _reference_payload_and_role(ref, data_field: str) -> tuple[bytes, Optional[str]]:
+RawReference = Union[str, "UploadFile", "ReferenceItem"]
+
+
+def _reference_payload_and_role(
+    ref: RawReference, data_field: str
+) -> tuple[bytes, Optional[str]]:

 def _build_reference_list(
-    value, *, modality: str, data_field: str, ref_cls, id: str, media_storage_path: Optional[str]
-):
+    value: Optional[Union[RawReference, list[RawReference]]],
+    *,
+    modality: str,
+    data_field: str,
+    ref_cls: type[BaseModel],
+    id: str,
+    media_storage_path: Optional[str],
+) -> Optional[list[BaseModel]]:

As per coding guidelines: "Annotate every function, use None for procedures, avoid unnecessary Any and type: ignore".

Also applies to: 167-169

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/serve/visual_gen_utils.py` at line 114, Complete type
annotations for _reference_payload_and_role by annotating ref, and for
_build_reference_list by annotating value, ref_cls, and its return type. Define
or reuse a readable union alias for the accepted reference/value types, and
annotate any procedure return as None without introducing unnecessary Any or
type ignores.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/serve/visual_gen_utils.py`:
- Around line 167-193: Update _build_reference_list to clean up all reference
files successfully materialized during the current loop when a later item raises
ValueError. Track each created ref_path, catch the validation/materialization
failure, remove the tracked files from media_storage_path, then re-raise the
original error; preserve normal list construction and return behavior when all
entries succeed.

---

Nitpick comments:
In `@tensorrt_llm/serve/visual_gen_utils.py`:
- Line 114: Complete type annotations for _reference_payload_and_role by
annotating ref, and for _build_reference_list by annotating value, ref_cls, and
its return type. Define or reuse a readable union alias for the accepted
reference/value types, and annotate any procedure return as None without
introducing unnecessary Any or type ignores.

In `@tests/unittest/_torch/visual_gen/test_visual_gen_utils.py`:
- Around line 263-267: Add audio_reference coverage to
TestInputReferenceMaterialization: test that a base64 audio reference is
materialized byte-identically, and test that omitting media_storage_path raises
the expected error. Exercise parse_visual_gen_params and the audio branch of
_materialize_reference while preserving the existing image and video tests.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: df5aabeb-4354-4ab7-bc70-0c90918da79f

📥 Commits

Reviewing files that changed from the base of the PR and between 69eea16 and 9574048.

📒 Files selected for processing (37)

• docs/source/models/visual-generation.md
• examples/visual_gen/models/cosmos3/cosmos3.py
• examples/visual_gen/models/flux2.py
• examples/visual_gen/models/qwen_image_edit.py
• examples/visual_gen/models/qwen_image_layered.py
• examples/visual_gen/models/wan_i2v.py
• examples/visual_gen/serve/README.md
• examples/visual_gen/serve/async_video_gen.py
• examples/visual_gen/serve/sync_video_gen.py
• tensorrt_llm/__init__.py
• tensorrt_llm/_torch/visual_gen/executor.py
• tensorrt_llm/_torch/visual_gen/models/cosmos3/defaults.py
• tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py
• tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py
• tensorrt_llm/_torch/visual_gen/models/qwen_image/pipeline_qwen_image_edit.py
• tensorrt_llm/_torch/visual_gen/models/qwen_image_layered/pipeline_qwen_image_layered.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py
• tensorrt_llm/_torch/visual_gen/pipeline.py
• tensorrt_llm/serve/openai_protocol.py
• tensorrt_llm/serve/openai_server.py
• tensorrt_llm/serve/openai_video_routes.py
• tensorrt_llm/serve/visual_gen_utils.py
• tensorrt_llm/visual_gen/__init__.py
• tensorrt_llm/visual_gen/params.py
• tensorrt_llm/visual_gen/visual_gen.py
• tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md
• tests/integration/defs/perf/visual_gen_perf_utils.py
• tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml
• tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml
• tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py
• tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
• tests/unittest/_torch/visual_gen/test_visual_gen_params.py
• tests/unittest/_torch/visual_gen/test_visual_gen_utils.py
• tests/unittest/api_stability/references/trtllm_serve_api.yaml

💤 Files with no reviewable changes (1)

• tensorrt_llm/_torch/visual_gen/models/cosmos3/defaults.py

🚧 Files skipped from review as they are similar to previous changes (31)

• tests/integration/defs/perf/README_test_visual_gen_perf_sanity.md
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py
• examples/visual_gen/models/wan_i2v.py
• tensorrt_llm/visual_gen/init.py
• tensorrt_llm/serve/openai_server.py
• tensorrt_llm/_torch/visual_gen/models/qwen_image_layered/pipeline_qwen_image_layered.py
• examples/visual_gen/models/qwen_image_layered.py
• tensorrt_llm/visual_gen/visual_gen.py
• examples/visual_gen/models/qwen_image_edit.py
• examples/visual_gen/models/flux2.py
• tests/scripts/perf-sanity/visual_gen/wan22_i2v_a14b_blackwell.yaml
• tensorrt_llm/_torch/visual_gen/models/qwen_image/pipeline_qwen_image_edit.py
• tensorrt_llm/_torch/visual_gen/pipeline.py
• tests/integration/defs/perf/visual_gen_perf_utils.py
• tests/scripts/perf-sanity/visual_gen/ltx2_blackwell.yaml
• tensorrt_llm/serve/openai_video_routes.py
• tensorrt_llm/_torch/visual_gen/executor.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py
• tests/unittest/api_stability/references/trtllm_serve_api.yaml
• tensorrt_llm/serve/openai_protocol.py
• examples/visual_gen/serve/README.md
• tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py
• tensorrt_llm/_torch/visual_gen/models/cosmos3/pipeline_cosmos3.py
• examples/visual_gen/models/cosmos3/cosmos3.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py
• tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py
• tensorrt_llm/init.py
• tests/unittest/_torch/visual_gen/test_visual_gen_params.py
• tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
• tensorrt_llm/visual_gen/params.py

[luyiyun1021]

/bot reuse-pipeline

[tensorrt-cicd]

PR_Github #71171 [ reuse-pipeline ] triggered by Bot. Commit: cd27779 Link to invocation

[tensorrt-cicd]

PR_Github #71171 [ reuse-pipeline ] completed with state SUCCESS. Commit: cd27779
Reusing PR_Github #71099 for commit cd27779

Link to invocation