From be72f41cb3ec02d30ad72aebbb09c88feaf0646c Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Mon, 23 Mar 2026 14:02:12 -0700 Subject: [PATCH 1/6] [None][doc] Add visual generation models to supported models page Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/models/supported-models.md | 29 +++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/docs/source/models/supported-models.md b/docs/source/models/supported-models.md index bd1aefe5d0b3..b5f52e923cdd 100644 --- a/docs/source/models/supported-models.md +++ b/docs/source/models/supported-models.md @@ -85,4 +85,31 @@ Note: # Visual Generation Models -For diffusion-based image and video generation models, see the [Visual Generation](./visual-generation.md) documentation. +TensorRT-LLM supports diffusion-based image and video generation via the **VisualGen** subsystem (prototype). +For full documentation, see the [Visual Generation](./visual-generation.md) page. + +## Supported Models + +| HuggingFace Model ID | Tasks | +|---|---| +| `black-forest-labs/FLUX.1-dev` | Text-to-Image | +| `black-forest-labs/FLUX.2-dev` | Text-to-Image | +| `Wan-AI/Wan2.1-T2V-1.3B-Diffusers` | Text-to-Video | +| `Wan-AI/Wan2.1-T2V-14B-Diffusers` | Text-to-Video | +| `Wan-AI/Wan2.1-I2V-14B-480P-Diffusers` | Image-to-Video | +| `Wan-AI/Wan2.1-I2V-14B-720P-Diffusers` | Image-to-Video | +| `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | Text-to-Video | +| `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | Image-to-Video | +| `Lightricks/LTX-Video` | Text-to-Video (with Audio), Image-to-Video (with Audio) | + +## Feature Matrix + +| Model | FP8 blockwise | NVFP4 | TeaCache | CFG Parallelism | Ulysses Parallelism | Parallel VAE | CUDA Graph | torch.compile | trtllm-serve | +|---|---|---|---|---|---|---|---|---|---| +| **FLUX.1** | Yes | Yes | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | +| **FLUX.2** | Yes | Yes | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | +| **Wan 2.1** | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| **Wan 2.2** | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | +| **LTX-2** | Yes | Yes | No | Yes | Yes | No | No | Yes | Yes | + +[^vg1]: FLUX models use embedded guidance and do not have a separate negative prompt path, so CFG parallelism is not applicable. From 8d095ce6811879d307c5a39c3009ad7236dd06c9 Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Mon, 23 Mar 2026 14:16:21 -0700 Subject: [PATCH 2/6] [None][doc] Reword visual generation intro sentence Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/models/supported-models.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/models/supported-models.md b/docs/source/models/supported-models.md index b5f52e923cdd..fe33c48de9a1 100644 --- a/docs/source/models/supported-models.md +++ b/docs/source/models/supported-models.md @@ -85,7 +85,7 @@ Note: # Visual Generation Models -TensorRT-LLM supports diffusion-based image and video generation via the **VisualGen** subsystem (prototype). +TensorRT-LLM provides prototype support for diffusion-based image and video generation. For full documentation, see the [Visual Generation](./visual-generation.md) page. ## Supported Models From fe9475a33fac07075687ff2aeafeb049768dd4da Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:56:18 -0700 Subject: [PATCH 3/6] [None][doc] Fix LTX-2 HuggingFace model ID Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/models/supported-models.md | 2 +- docs/source/models/visual-generation.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/models/supported-models.md b/docs/source/models/supported-models.md index fe33c48de9a1..b26e47c1248a 100644 --- a/docs/source/models/supported-models.md +++ b/docs/source/models/supported-models.md @@ -100,7 +100,7 @@ For full documentation, see the [Visual Generation](./visual-generation.md) page | `Wan-AI/Wan2.1-I2V-14B-720P-Diffusers` | Image-to-Video | | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | Text-to-Video | | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | Image-to-Video | -| `Lightricks/LTX-Video` | Text-to-Video (with Audio), Image-to-Video (with Audio) | +| `Lightricks/LTX-2` | Text-to-Video (with Audio), Image-to-Video (with Audio) | ## Feature Matrix diff --git a/docs/source/models/visual-generation.md b/docs/source/models/visual-generation.md index 72c34ad23f75..089e535fbd25 100644 --- a/docs/source/models/visual-generation.md +++ b/docs/source/models/visual-generation.md @@ -30,7 +30,7 @@ TensorRT-LLM **VisualGen** provides a unified inference stack for diffusion mode | `Wan-AI/Wan2.1-I2V-14B-720P-Diffusers` | Image-to-Video | | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | Text-to-Video | | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | Image-to-Video | -| `Lightricks/LTX-Video` | Text-to-Video (with Audio), Image-to-Video (with Audio) | +| `Lightricks/LTX-2` | Text-to-Video (with Audio), Image-to-Video (with Audio) | Models are auto-detected from the checkpoint directory. Diffusers-format models are detected via `model_index.json`; LTX-2 monolithic safetensors checkpoints are detected via embedded metadata. The `AutoPipeline` registry selects the appropriate pipeline class automatically. From 47d74686e5f772f637c6d005aac6d47bab81db99 Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Tue, 24 Mar 2026 16:59:19 -0700 Subject: [PATCH 4/6] [None][doc] Remove precision columns from visual gen feature matrix Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/models/supported-models.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/source/models/supported-models.md b/docs/source/models/supported-models.md index b26e47c1248a..d10fe41ee6bb 100644 --- a/docs/source/models/supported-models.md +++ b/docs/source/models/supported-models.md @@ -104,12 +104,12 @@ For full documentation, see the [Visual Generation](./visual-generation.md) page ## Feature Matrix -| Model | FP8 blockwise | NVFP4 | TeaCache | CFG Parallelism | Ulysses Parallelism | Parallel VAE | CUDA Graph | torch.compile | trtllm-serve | -|---|---|---|---|---|---|---|---|---|---| -| **FLUX.1** | Yes | Yes | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | -| **FLUX.2** | Yes | Yes | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | -| **Wan 2.1** | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | -| **Wan 2.2** | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | -| **LTX-2** | Yes | Yes | No | Yes | Yes | No | No | Yes | Yes | +| Model | TeaCache | CFG Parallelism | Ulysses Parallelism | Parallel VAE | CUDA Graph | torch.compile | trtllm-serve | +|---|---|---|---|---|---|---|---| +| **FLUX.1** | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | +| **FLUX.2** | Yes | No [^vg1] | Yes | No | Yes | Yes | Yes | +| **Wan 2.1** | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +| **Wan 2.2** | No | Yes | Yes | Yes | Yes | Yes | Yes | +| **LTX-2** | No | Yes | Yes | No | No | Yes | Yes | [^vg1]: FLUX models use embedded guidance and do not have a separate negative prompt path, so CFG parallelism is not applicable. From e06bab3f65276070c3912a73e2bc1e05e28bd050 Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Mon, 30 Mar 2026 10:25:40 -0700 Subject: [PATCH 5/6] [None][doc] Promote visual generation from prototype to beta Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/commands/trtllm-serve/trtllm-serve.rst | 2 +- docs/source/models/supported-models.md | 2 +- docs/source/models/visual-generation.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/commands/trtllm-serve/trtllm-serve.rst b/docs/source/commands/trtllm-serve/trtllm-serve.rst index cdfa3cac9fc4..d4b335472a65 100644 --- a/docs/source/commands/trtllm-serve/trtllm-serve.rst +++ b/docs/source/commands/trtllm-serve/trtllm-serve.rst @@ -218,7 +218,7 @@ Visual Generation Serving ``trtllm-serve`` supports diffusion-based visual generation models (FLUX.1, FLUX.2, Wan2.1, Wan2.2) for image and video generation. When a diffusion model directory is provided (detected by the presence of ``model_index.json``), the server automatically launches in visual generation mode with dedicated endpoints. .. note:: - VisualGen is in **prototype** stage. APIs, supported models, and optimization options are actively evolving and may change in future releases. + VisualGen is in **beta** stage. APIs, supported models, and optimization options are actively evolving and may change in future releases. .. code-block:: bash diff --git a/docs/source/models/supported-models.md b/docs/source/models/supported-models.md index d10fe41ee6bb..54d01f145c23 100644 --- a/docs/source/models/supported-models.md +++ b/docs/source/models/supported-models.md @@ -85,7 +85,7 @@ Note: # Visual Generation Models -TensorRT-LLM provides prototype support for diffusion-based image and video generation. +TensorRT-LLM provides beta support for diffusion-based image and video generation. For full documentation, see the [Visual Generation](./visual-generation.md) page. ## Supported Models diff --git a/docs/source/models/visual-generation.md b/docs/source/models/visual-generation.md index 089e535fbd25..49c00e332b1a 100644 --- a/docs/source/models/visual-generation.md +++ b/docs/source/models/visual-generation.md @@ -1,7 +1,7 @@ -# Visual Generation (Prototype) +# Visual Generation (Beta) ```{note} -This feature is in **prototype** stage. APIs, supported models, and optimization options are +This feature is in **beta** stage. APIs, supported models, and optimization options are actively evolving and may change in future releases. ``` From ecae316902d9b6c2b9e2792f0e7f33294fb100b1 Mon Sep 17 00:00:00 2001 From: Chang Liu <9713593+chang-l@users.noreply.github.com> Date: Fri, 3 Apr 2026 15:51:59 -0700 Subject: [PATCH 6/6] [None][doc] Fix visual gen quickstart rendering in visual-generation.md Replace Sphinx-only literalinclude directive with a plain bash run command so the Quick Start section renders correctly on GitHub. Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com> --- docs/source/models/visual-generation.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/source/models/visual-generation.md b/docs/source/models/visual-generation.md index 49c00e332b1a..fe5ef3703255 100644 --- a/docs/source/models/visual-generation.md +++ b/docs/source/models/visual-generation.md @@ -50,9 +50,8 @@ Models are auto-detected from the checkpoint directory. Diffusers-format models Here is a simple example to generate a video with Wan 2.1: -```{literalinclude} ../../../examples/visual_gen/quickstart_example.py - :language: python - :linenos: +```bash +python examples/visual_gen/quickstart_example.py ``` To learn more about VisualGen, see [`examples/visual_gen/`](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/visual_gen) for more examples including text-to-image, image-to-video, and batch generation.