diff --git a/README.md b/README.md index e5147e8576e..7a21c8ab657 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ Easy, fast, and cheap omni-modality model serving for everyone *Latest News* 🔥 +- [2026/08] We released [0.28.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.28.0), featuring production-ready [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) serving on GPU and NPU, a unified AR/DiT paged KV cache runtime, and enhanced realtime full-duplex serving for the [MiniCPM-o series](recipes/OpenBMB/MiniCPM-o-4_5.md). - [2026/08] [VeRL-Omni](https://github.com/verl-project/verl-omni) `v0.2.0` is released: faster diffusion RL powered by vLLM-Omni (request-level/step-wise batching with FA3), rebuilt Qwen3-Omni multimodal training (DPO & GSPO), plus LTX-2.3, Qwen-Image-Edit support and more. See the [release notes](https://github.com/verl-project/verl-omni/releases/tag/v0.2.0). - [2026/08] We released [0.26.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.26.0) - aligned with the vLLM 0.26 release line, featuring [MiniMax H3](recipes/MiniMaxAI/MiniMax-H3.md) joint video/audio generation, an experimental full-duplex realtime runtime for [MiniCPM-o 4.5](recipes/OpenBMB/MiniCPM-o-4_5.md), distributed layerwise diffusion offload, and broader model, hardware, streaming, TTS, and quantization support. - [2026/07] We released [0.24.0](https://github.com/vllm-project/vllm-omni/releases/tag/v0.24.0) - aligned with the vLLM 0.24 release line, expanding production-ready coverage across TTS, speech, diffusion, image/video generation, and robot-policy serving, with major Omni stage runtime refactoring, diffusion request-level batching, async output materialization, quantization/cache/memory improvements, and broad CUDA/ROCm/XPU/NPU support. @@ -57,9 +58,9 @@ vLLM-Omni is flexible and easy to use with: vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: - **Omni-modality models** (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL) -- **TTS models** (e.g. Qwen3-TTS, VoxCPM2, Ming-Omni-TTS, CosyVoice3) -- **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, Qwen-Image, Wan2.2, FLUX) -- **Robot-policy and action models** (e.g. GR00T-N1.7, DreamZero-DROID, InternVLA-A1, Cosmos3 action policy) +- **TTS models** (e.g. Qwen3-TTS, IndexTTS 2.5, dots.tts, CosyVoice3) +- **Diffusion models** — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2) +- **Robot-policy and action models** (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1) ## Getting Started @@ -67,6 +68,7 @@ Visit our [documentation](https://vllm-omni.readthedocs.io/en/latest/) to learn - [Installation](https://vllm-omni.readthedocs.io/en/latest/getting_started/installation/) - [Quickstart](https://vllm-omni.readthedocs.io/en/latest/getting_started/quickstart/) +- [Nightly CUDA Docker images](https://hub.docker.com/r/vllm/vllm-omni/tags?name=nightly) built from the latest `main` - [List of Supported Models](https://vllm-omni.readthedocs.io/en/latest/models/supported_models/) - [Deployment Recipes](https://recipes.vllm.ai) for vLLM-Omni model serving diff --git a/docs/README.md b/docs/README.md index 39df4e6a095..7eeb0bd7060 100644 --- a/docs/README.md +++ b/docs/README.md @@ -57,9 +57,9 @@ vLLM-Omni is flexible and easy to use with: vLLM-Omni seamlessly supports most popular open-source models on HuggingFace, including: - Omni-modality models (e.g. Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, BAGEL) -- TTS models (e.g. Qwen3-TTS, VoxCPM2, Ming-Omni-TTS, CosyVoice3) -- Diffusion models — image, video, and audio generation (e.g. MiniMax H3, Qwen-Image, Wan2.2, FLUX) -- Robot-policy and action models (e.g. GR00T-N1.7, DreamZero-DROID, InternVLA-A1, Cosmos3 action policy) +- TTS models (e.g. Qwen3-TTS, IndexTTS 2.5, dots.tts, CosyVoice3) +- Diffusion models — image, video, and audio generation (e.g. MiniMax H3, LTX-2.5, SANA-Video, Wan2.2) +- Robot-policy and action models (e.g. π0, GR00T-N1.7, DreamZero-DROID, InternVLA-A1) For more information, checkout the following: diff --git a/docs/configuration/README.md b/docs/configuration/README.md index e68668e06f5..cdfb5818ace 100644 --- a/docs/configuration/README.md +++ b/docs/configuration/README.md @@ -2,7 +2,7 @@ This section lists the most common options for running vLLM-Omni. -For options within a vLLM Engine, please refer to the [vLLM 0.26 configuration guide](https://docs.vllm.ai/en/v0.26.0/configuration/index.html). +For options within a vLLM Engine, please refer to the [vLLM 0.28 configuration guide](https://docs.vllm.ai/en/v0.28.0/configuration/index.html). Each model defines fixed topology in a registered `PipelineConfig` and runtime overrides in a deploy YAML. diff --git a/docs/configuration/environment_variables.md b/docs/configuration/environment_variables.md index ace89704893..14f95bda082 100644 --- a/docs/configuration/environment_variables.md +++ b/docs/configuration/environment_variables.md @@ -151,7 +151,7 @@ their keys only. ## Inherited vLLM variables vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to -the [vLLM 0.26 environment-variable reference](https://docs.vllm.ai/en/v0.26.0/configuration/env_vars.html) +the [vLLM 0.28 environment-variable reference](https://docs.vllm.ai/en/v0.28.0/configuration/env_vars.html) for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm, XPU, ModelScope, and FlashInfer workspace settings. diff --git a/docs/getting_started/installation/gpu.md b/docs/getting_started/installation/gpu.md index 196b6a63870..7b87782967f 100644 --- a/docs/getting_started/installation/gpu.md +++ b/docs/getting_started/installation/gpu.md @@ -34,7 +34,7 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr ### Pre-built wheels -Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc1, 0.14.0rc1, 0.14.0, 0.16.0, 0.18.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, and 0.26.0; the 0.28.0 wheels are published together with the v0.28.0 release this branch produces. If you need a newer unreleased revision, please [build from source](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/#build-wheel-from-source). +Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc1, 0.14.0rc1, 0.14.0, 0.16.0, 0.18.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, and 0.28.0. If you need a newer unreleased revision, please [build from source](https://docs.vllm.ai/projects/vllm-omni/en/latest/getting_started/installation/gpu/#build-wheel-from-source). === "NVIDIA CUDA" diff --git a/docs/getting_started/installation/gpu/cuda.inc.md b/docs/getting_started/installation/gpu/cuda.inc.md index a8f795861e7..5e6cd47c9ce 100644 --- a/docs/getting_started/installation/gpu/cuda.inc.md +++ b/docs/getting_started/installation/gpu/cuda.inc.md @@ -5,7 +5,7 @@ # --8<-- [end:requirements] # --8<-- [start:set-up-using-python] -vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.26.x release line uses vLLM 0.26.x. +vLLM-Omni depends on the matching major/minor release of vLLM. The vLLM-Omni 0.28.x release line uses vLLM 0.28.x. !!! note PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See for more details. @@ -95,6 +95,13 @@ uv pip install --no-build-isolation --editable . vLLM-Omni offers an official docker image for deployment. These images are built on top of vLLM docker images and available on Docker Hub as [vllm/vllm-omni](https://hub.docker.com/r/vllm/vllm-omni/tags). The version of vLLM-Omni indicates which release of vLLM it is based on. +Nightly CUDA images built from the latest `main` are also available: + +- [`vllm/vllm-omni:nightly`](https://hub.docker.com/r/vllm/vllm-omni/tags?name=nightly) is the rolling multi-architecture tag. +- [`vllm/vllm-omni:nightly-`](https://hub.docker.com/r/vllm/vllm-omni/tags?name=nightly-) provides commit-pinned multi-architecture tags. The newest 14 are retained. + +Use a commit-pinned tag when you need a reproducible nightly environment. + Here's an example deployment command that has been verified on 2 x H100's: ```bash docker run --runtime nvidia --gpus 2 \ diff --git a/docs/getting_started/installation/npu/npu.inc.md b/docs/getting_started/installation/npu/npu.inc.md index a0d4c86b079..0019d00bdf5 100644 --- a/docs/getting_started/installation/npu/npu.inc.md +++ b/docs/getting_started/installation/npu/npu.inc.md @@ -9,8 +9,8 @@ The recommended way to use vLLM-Omni on NPU is through the vLLM-Ascend pre-built ```bash # vLLM-Ascend image -# Atlas A2: quay.io/atlas-ci/vllm-ascend:v0.26.0 -# Atlas A3: quay.io/atlas-ci/vllm-ascend:v0.26.0-a3 +# Atlas A2: quay.io/atlas-ci/vllm-ascend:v0.28.0 +# Atlas A3: quay.io/atlas-ci/vllm-ascend:v0.28.0-a3 docker run --rm \ --name vllm-omni-npu \ --shm-size=1g \ @@ -28,11 +28,11 @@ docker run --rm \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v /root/.cache:/root/.cache \ -p 8000:8000 \ - -it quay.io/atlas-ci/vllm-ascend:v0.26.0-a3 bash + -it quay.io/atlas-ci/vllm-ascend:v0.28.0-a3 bash # Inside the container, install vLLM-Omni from source cd /vllm-workspace -git clone -b v0.26.0 https://github.com/vllm-project/vllm-omni.git +git clone -b v0.28.0 https://github.com/vllm-project/vllm-omni.git cd vllm-omni pip install -v -e . --no-build-isolation # or VLLM_OMNI_TARGET_DEVICE=npu pip install -v -e . @@ -50,21 +50,11 @@ We are keeping [issue #886](https://github.com/vllm-project/vllm-omni/issues/886 # --8<-- [start:installation-main] -You can also build vLLM-Omni from the latest main branch if you want to use the latest features or bug fixes. (But sometimes it will break for a while. You can check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for the status of the latest commit of vLLM-Omni main branch on NPU.) +You can also install vLLM-Omni from the latest main branch if you want to use the latest features or bug fixes. Use the aligned vLLM-Ascend v0.28.0 image described above, and check [issue #886](https://github.com/vllm-project/vllm-omni/issues/886) for the status of the latest vLLM-Omni commit on NPU. ```bash -# Pin vLLM and vLLM-Ascend to the v0.26 release line -git clone -b v0.26.0 https://github.com/vllm-project/vllm.git -cd vllm -VLLM_TARGET_DEVICE=empty pip install -v -e . -cd .. - -git clone -b releases/v0.26.0rc https://github.com/vllm-project/vllm-ascend.git -cd vllm-ascend -pip install -v -e . -cd .. - -# Install vLLM-Omni from the latest main branch +# Inside the vLLM-Ascend v0.28.0 container, install vLLM-Omni from main +cd /vllm-workspace git clone https://github.com/vllm-project/vllm-omni.git cd vllm-omni pip install -v -e . --no-build-isolation @@ -88,8 +78,8 @@ Supported images as following. Here's an example deployment command that has been verified on 4 x NPUs: ```bash -# Atlas A2: quay.io/ascend/vllm-omni:v0.26.0 -# Atlas A3: quay.io/ascend/vllm-omni:v0.26.0-a3 +# Atlas A2: quay.io/ascend/vllm-omni:v0.28.0 +# Atlas A3: quay.io/ascend/vllm-omni:v0.28.0-a3 docker run --rm \ --name vllm-omni-a3 \ --shm-size=4g \ @@ -114,7 +104,7 @@ docker run --rm \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v ~/.cache:/root/.cache \ -p 8091:8091 \ - -it quay.io/ascend/vllm-omni:v0.26.0-a3 bash + -it quay.io/ascend/vllm-omni:v0.28.0-a3 bash ``` !!! tip