feat(vllm): add Qwen3-ASR speech recognition example

.lycheeignore

@@ -115,3 +115,7 @@
 # examples/gpu-sharing is added in the same PR as the GPU sharing docs
 # that link to it on main. Remove this rule once that PR merges.
 ^https://github\.com/ai-dynamo/dynamo/(blob|tree)/main/examples/gpu-sharing
+
+# The Qwen3-ASR DGD manifest is introduced by the same PR as its vLLM multimodal
+# guide link, so the blob/main URL returns 404 until merge. Remove this entry after merge.
+^https://github\.com/ai-dynamo/dynamo/blob/main/examples/backends/vllm/deploy/agg_asr\.yaml$

docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/vllm/multimodal.md

@@ -63,6 +63,7 @@ The main multimodal vLLM launchers in this repo are:
 | P/D | CUDA | `disagg_multimodal_p_d.sh` | Prefill/decode separation without a dedicated encoder |
 | E/PD (Encode + PD) | CUDA | `disagg_multimodal_e_pd.sh` | Separate encoder and embedding-cache workflows |
 | E/P/D (Full Disaggregation) | CUDA | `disagg_multimodal_epd.sh` | Separate encode, prefill, and decode workers |
+| Aggregated ASR | CUDA | `agg_asr.sh` | Speech recognition with Qwen3-ASR; see [Speech Recognition (ASR)](#speech-recognition-asr) |
 
 ### Custom Vision Encoders
 
@@ -130,7 +131,7 @@ This launcher sets `--dyn-chat-processor vllm`. The frontend runs vLLM's Hugging
 
 `DYNAMO_MM_TRANSFER` selects the transfer mechanism:
 
-- `shm` (default) uses shared memory for same-node frontend and worker deployments.
+- `shm` (default) uses shared memory for same-node frontend and worker deployments. The frontend and worker must share `/dev/shm`, as processes on one host do. Separate Kubernetes pods do not share it even on the same node; when the worker cannot read the segment, it serves the request without its media.
 - `nixl` uses NIXL for cross-node transfer.
 - `DYNAMO_DISABLE_NIXL_MM=1` disables processed-input transfer and makes the worker process the original media.
 
@@ -356,7 +357,7 @@ bash launch/disagg_multimodal_epd.sh --model Qwen/Qwen3-VL-2B-Instruct --single-
 
 ## Audio Serving
 
-Dynamo supports `audio_url` requests for audio-capable models. Audio is loaded by the backend worker via vLLM's `AudioMediaIO` at native sample rate — vLLM's model-specific processor handles resampling and feature extraction internally. Omni models can handle `image_url`, `video_url`, and `audio_url` in the same request.
+Dynamo supports `audio_url` requests for audio-capable models. Audio is loaded via vLLM's `AudioMediaIO` at native sample rate, and vLLM resamples it to the rate the model's feature extractor expects before extracting features. vLLM resamples through PyAV, which the Dynamo runtime images do not ship, so audio whose sample rate differs from the model's needs PyAV installed. See [Additional Media Decoders](../../../../../use-cases/multimodal-serving/additional-media-decoders.md). Omni models can handle `image_url`, `video_url`, and `audio_url` in the same request.
 
 ### Aggregated Serving
 
@@ -411,6 +412,89 @@ curl http://localhost:8000/v1/chat/completions \
     }' | jq
 ```
 
+### Speech Recognition (ASR)
+
+Qwen3-ASR checkpoints such as `Qwen/Qwen3-ASR-1.7B` and `Qwen/Qwen3-ASR-0.6B` transcribe speech sent as an `audio_url` content part to `/v1/chat/completions`. These checkpoints ship `vocab.json` and `merges.txt` but no `tokenizer.json`, which the default Rust frontend needs, so run the frontend with the [Python chat processor](#python-chat-processor) (`--dyn-chat-processor vllm`). The `agg_asr.sh` launcher does this and serves `Qwen/Qwen3-ASR-1.7B` by default:
+
+```bash
+cd $DYNAMO_HOME/examples/backends/vllm
+bash launch/agg_asr.sh
+```
+
+Pass `--model Qwen/Qwen3-ASR-0.6B` to serve the smaller checkpoint. The launcher passes any other arguments to the worker.
+
+`Qwen/Qwen3-ASR-1.7B-hf`, the Hugging Face Transformers-format checkpoint, ships a `tokenizer.json` and also runs behind the default Rust frontend, which forwards the audio to the worker:
+
+```bash
+DYN_CHAT_PROCESSOR=dynamo bash launch/agg_asr.sh --model Qwen/Qwen3-ASR-1.7B-hf
+```
+
+**Transcription request:**
+
+The request names the default checkpoint. If you serve another checkpoint, such as `Qwen/Qwen3-ASR-0.6B` or `Qwen/Qwen3-ASR-1.7B-hf`, set `model` to that checkpoint's name.
+
+```bash
+curl http://localhost:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+      "model": "Qwen/Qwen3-ASR-1.7B",
+      "messages": [
+        {
+          "role": "user",
+          "content": [
+            {
+              "type": "audio_url",
+              "audio_url": {
+                "url": "https://raw.githubusercontent.com/yuekaizhang/Triton-ASR-Client/main/datasets/mini_en/wav/1221-135766-0002.wav"
+              }
+            }
+          ]
+        }
+      ],
+      "max_tokens": 256
+    }' | jq -r '.choices[0].message.content'
+```
+
+The response names the detected language before an `<asr_text>` tag, followed by the transcript:
+
+```text
+language English<asr_text>Yet these thoughts affected Hester Prynne less with hope than apprehension.
+```
+
+Strip the text up to and including `<asr_text>` to keep only the transcript. With `"stream": true`, the prefix arrives in the first chunks.
+
+The Dynamo runtime images accept audio at the model's 16 kHz rate. Audio at any other rate fails with HTTP 500 until PyAV is installed, and the frontend or worker log shows `ImportError: Please install vllm[audio] for audio support`. With the Python chat processor, the frontend also resamples audio, so install PyAV where both the frontend and the worker run, or convert audio to 16 kHz before sending it:
+
+```bash
+pip install --no-deps 'av>=18.0.0,<19'
+```
+
+Dynamo does not serve `/v1/audio/transcriptions`. Send audio through `/v1/chat/completions` or stream it through `/v1/realtime`.
+
+#### Kubernetes Deployment
+
+[`agg_asr.yaml`](https://github.com/ai-dynamo/dynamo/blob/main/examples/backends/vllm/deploy/agg_asr.yaml) deploys the same frontend and worker as a DynamoGraphDeployment. The frontend and worker run in separate pods, so the default shared-memory transfer of processed audio cannot reach the worker, and the worker transcribes the prompt without the audio (`language None<asr_text>`). The manifest sets `DYNAMO_DISABLE_NIXL_MM=1` on the frontend so the worker loads and processes the audio itself. NIXL transfer also works across pods and spares the worker a second pass over the audio. To use it, remove `DYNAMO_DISABLE_NIXL_MM` from the frontend and set `DYNAMO_MM_TRANSFER=nixl` there. The frontend checks `DYNAMO_DISABLE_NIXL_MM` before it selects a transfer mode, so setting `DYNAMO_MM_TRANSFER=nixl` alone has no effect. With `Qwen/Qwen3-ASR-1.7B-hf` behind the default Rust frontend, the worker always loads the audio, so neither setting applies. To accept audio at other sample rates, run both components from an image with PyAV installed; see [Additional Media Decoders](../../../../../use-cases/multimodal-serving/additional-media-decoders.md#kubernetes).
+
+#### Streaming Transcription
+
+vLLM serves streaming transcription for Qwen3-ASR through its `Qwen3ASRRealtimeGeneration` architecture on `/v1/realtime`. The launcher does not pick an architecture from the model name, so select it with `--hf-overrides`. The realtime path does not tokenize in the frontend, so it runs with the default Rust frontend:
+
+```bash
+cd $DYNAMO_HOME/examples/backends/vllm
+bash launch/agg_realtime_transcription.sh --model Qwen/Qwen3-ASR-1.7B \
+  --hf-overrides '{"architectures":["Qwen3ASRRealtimeGeneration"]}'
+```
+
+Stream a sample clip from a second terminal:
+
+```bash
+cd $DYNAMO_HOME/examples/backends/vllm
+python launch/realtime_audio_client.py --session-type transcription \
+  --url ws://localhost:8000/v1/realtime --model Qwen/Qwen3-ASR-1.7B
+```
+
+The session accepts 24 kHz PCM16 input, which the worker resamples to 16 kHz without PyAV. The model transcribes each 5-second segment as it arrives. Each segment's text starts on a new line with its own `language <Language><asr_text>` prefix, and a very short final segment can repeat the previous segment's text.
+
 ## Embedding Cache
 
 Dynamo supports embedding cache in both aggregated and disaggregated settings:

docs/fern/pages/use-cases/multimodal-serving/additional-media-decoders.md

@@ -19,6 +19,7 @@ backends need the corresponding package from the table below. Installing an
 additional decoder package also covers:
 
 - **AAC and other compressed audio**, which NVDEC does not decode at all.
+- **Audio at a sample rate other than the model's** on vLLM, which resamples it through PyAV by default. Whisper-style audio encoders expect 16 kHz.
 - **H.264 and H.265 on hosts where NVDEC is unavailable** — no video decode engine on the GPU, or a container without the `video` capability.
 
 Each backend decodes such input through a specific Python package whose wheel bundles its own FFmpeg, so the support is added with a plain `pip install` — no image rebuild. **Nothing installs automatically.** There is no environment switch and no startup hook: an operator runs the install as a deliberate, visible step, so a deployment that broadens the image's codec surface says so in its Dockerfile, pod spec, or runbook.
@@ -56,7 +57,7 @@ pip install --no-deps 'opencv-python-headless>=4.13.0.92,<5'
 
 ## Install with the bundled installer
 
-Every runtime image ships an installer for the packages above as part of the `ai-dynamo` package. Run it in the worker container (not the frontend) for the backend you deploy:
+Every runtime image ships an installer for the packages above as part of the `ai-dynamo` package. Run it in the worker container for the backend you deploy. A frontend running the vLLM chat processor (`--dyn-chat-processor vllm`) also loads and resamples media, so run it in that frontend container too:
 
 ```bash
 python -m dynamo.common.utils.install_media_decoders vllm

examples/backends/vllm/deploy/README.md

@@ -65,6 +65,13 @@ Two-node aggregated deployments with a process-local KVCR tier or a resilient
 KVCR memory-service sidecar. See [`kvcr/README.md`](./kvcr/README.md) for image,
 RDMA, and lifecycle requirements.
 
+### 9. **Aggregated Speech Recognition** (`agg_asr.yaml`)
+Serves Qwen3-ASR through `/v1/chat/completions` with `audio_url` content parts. See the [vLLM multimodal guide](../../../../docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/vllm/multimodal.md#speech-recognition-asr) for request format, output format, and audio requirements.
+
+**Architecture:**
+- `Frontend`: OpenAI-compatible API server with the vLLM chat processor (`--dyn-chat-processor vllm`), because the original Qwen3-ASR checkpoints ship no `tokenizer.json`. `DYNAMO_DISABLE_NIXL_MM=1` makes the worker load the audio, because shared-memory handoff does not cross pods.
+- `worker`: Single worker with `--enable-multimodal`
+
 ## CRD Structure
 
 All templates use the **DynamoGraphDeployment** CRD:

examples/backends/vllm/deploy/agg_asr.yaml

@@ -0,0 +1,65 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Aggregated speech recognition (ASR) with Qwen3-ASR.
+#
+# - The frontend runs vLLM's chat processor because the original Qwen3-ASR
+#   checkpoints ship no tokenizer.json for the Rust preprocessor.
+# - The frontend and worker run in separate pods, so processed audio features
+#   cannot be handed over through shared memory. DYNAMO_DISABLE_NIXL_MM=1 makes
+#   the frontend forward the audio URL and the worker load and process it.
+# - The stock image accepts audio at the model's 16 kHz rate. Resampling other
+#   rates needs PyAV in both images; see the additional media decoders guide.
+apiVersion: nvidia.com/v1beta1
+kind: DynamoGraphDeployment
+metadata:
+  name: vllm-agg-asr
+spec:
+  components:
+  - name: Frontend
+    podTemplate:
+      spec:
+        containers:
+        - args:
+          - --dyn-chat-processor
+          - vllm
+          command:
+          - python3
+          - -m
+          - dynamo.frontend
+          env:
+          - name: DYNAMO_DISABLE_NIXL_MM
+            value: "1"
+          envFrom:
+          - secretRef:
+              name: hf-token-secret
+          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
+          name: main
+    replicas: 1
+    type: frontend
+  - name: worker
+    podTemplate:
+      spec:
+        containers:
+        - args:
+          - --model
+          - Qwen/Qwen3-ASR-1.7B
+          - --enable-multimodal
+          command:
+          - python3
+          - -m
+          - dynamo.vllm
+          envFrom:
+          - secretRef:
+              name: hf-token-secret
+          image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:my-tag
+          name: main
+          resources:
+            limits:
+              nvidia.com/gpu: "1"
+            requests:
+              # Increase this value for larger models.
+              ephemeral-storage: 10Gi
+          workingDir: /workspace/examples/backends/vllm
+    replicas: 1
+    type: worker

examples/backends/vllm/launch/agg_asr.sh

@@ -0,0 +1,89 @@
+#!/bin/bash
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Aggregated speech recognition (ASR) serving with Qwen3-ASR
+#
+# Architecture: Single-worker PD (Prefill-Decode)
+# - Frontend: vLLM chat processor (--dyn-chat-processor vllm). The original
+#   Qwen3-ASR checkpoints ship vocab.json and merges.txt but no tokenizer.json,
+#   which the default Rust preprocessor requires.
+# - Worker: Standard vLLM worker with --enable-multimodal
+#
+# Clients send an audio_url content part to /v1/chat/completions. The model
+# replies with "language <Language><asr_text><transcript>".
+#
+# The stock runtime image decodes WAV/FLAC audio but omits PyAV, which vLLM
+# uses to resample audio to the model's 16 kHz rate. To accept other sample
+# rates, first run: python -m dynamo.common.utils.install_media_decoders vllm
+#
+# For streaming transcription over /v1/realtime, see
+# agg_realtime_transcription.sh.
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "$SCRIPT_DIR/../../../common/gpu_utils.sh"
+source "$SCRIPT_DIR/../../../common/launch_utils.sh"
+
+MODEL_NAME="${DYN_MODEL_NAME:-Qwen/Qwen3-ASR-1.7B}"
+CHAT_PROCESSOR="${DYN_CHAT_PROCESSOR:-vllm}"
+
+EXTRA_ARGS=()
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        --model)
+            MODEL_NAME=$2
+            shift 2
+            ;;
+        -h|--help)
+            echo "Usage: $0 [OPTIONS] [EXTRA_VLLM_ARGS]"
+            echo "Options:"
+            echo "  --model <model_name>   Qwen3-ASR checkpoint to serve (default: $MODEL_NAME)"
+            echo "  -h, --help             Show this help message"
+            echo ""
+            echo "Any additional arguments are passed through to the vLLM worker."
+            echo "Example: $0 --model Qwen/Qwen3-ASR-0.6B --max-model-len 8192"
+            exit 0
+            ;;
+        *)
+            EXTRA_ARGS+=("$1")
+            shift
+            ;;
+    esac
+done
+
+trap dynamo_exit_trap EXIT
+
+HTTP_PORT="${DYN_HTTP_PORT:-8000}"
+GPU_MEM_ARGS=$(build_vllm_gpu_mem_args)
+
+print_launch_banner --no-curl "Launching Aggregated ASR Serving" "$MODEL_NAME" "$HTTP_PORT" \
+    "Chat proc:   $CHAT_PROCESSOR" \
+    "Backend:     dynamo.vllm --enable-multimodal" \
+    "Media:       audio_url (16 kHz unless PyAV is installed)"
+
+print_curl_footer <<CURL
+  curl http://localhost:${HTTP_PORT}/v1/chat/completions \\
+    -H 'Content-Type: application/json' \\
+    -d '{
+      "model": "${MODEL_NAME}",
+      "messages": [{"role": "user", "content": [
+        {"type": "audio_url", "audio_url": {"url": "https://raw.githubusercontent.com/yuekaizhang/Triton-ASR-Client/main/datasets/mini_en/wav/1221-135766-0002.wav"}}
+      ]}],
+      "max_tokens": 256
+    }'
+CURL
+
+# dynamo.frontend accepts either --http-port flag or DYN_HTTP_PORT env var (defaults to 8000)
+python -m dynamo.frontend --dyn-chat-processor "$CHAT_PROCESSOR" &
+
+# Extra args from command line come last to allow overrides
+CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-0} \
+DYN_SYSTEM_PORT=${DYN_SYSTEM_PORT:-8081} \
+    python -m dynamo.vllm --enable-multimodal --model "$MODEL_NAME" \
+    $GPU_MEM_ARGS \
+    "${EXTRA_ARGS[@]}" &
+
+# Exit on first process failure; the EXIT trap tears down the rest
+wait_any_exit