[Alex-Wengg]

Overview

Lists NVIDIA Parakeet TDT (ParakeetForTDT, e.g. nvidia/parakeet-tdt-0.6b-v3) as supported through the out-of-tree vllm-parakeet-plugin, the same route as Nemotron 3.5 ASR in #59827. Refs #23943.

Claims

• ParakeetForTDT is listed in the plugin table in supported_models.md and with the other plugin-backed encoder-decoder models in v1_guide.md.
• No vLLM code changes.

Validation

The plugin on vLLM 0.32.0, one RTX A6000:

Check	Result
Token parity vs Transformers generate(), 73 LibriSpeech clips, fp32 (scripts/parity.py)	73/73 identical
WER through vllm serve (bf16 encoder), LibriSpeech test-clean / test-other (scripts/eval_wer.py)	1.93% / 3.61% (model card: 1.93% / 3.59%)
Throughput through vllm serve, 64 concurrent requests, test-clean	RTFx 238

pre-commit run --files docs/models/supported_models.md docs/usage/v1_guide.md passes.

Details

• Not a duplicate of [WIP][Model]Add parakeet tdt 0.6b v3 model #41708 (with MRV2 ModelState work), the first Parakeet TDT integration, which has been waiting since September on a decision about native support. That PR decodes each transcript at prefill and replays it as forced tokens. The plugin instead decodes step by step, batched across requests, and adds no model code to vLLM.
• The plugin follows the layout of vllm-nemotron-asr-plugin: a vllm.general_plugins entry point, a VerifyAndUpdateConfig hook, an MRV2 ModelState that keeps each request's encoder frames, the decode loop inside forward, and one-hot compute_logits. It differs in two ways. The TDT loop advances all requests together in GPU tensors and applies TDT's frame-skip durations. The plugin also captures its own CUDA graphs, so it runs with plain vllm serve and needs no --enforce-eager or VLLM_USE_V2_MODEL_RUNNER=1.
• Unlike [Model] Add Nemotron 3.5 ASR transcription support #59827, this adds no _OOT_SUPPORTED_MODELS entry. Transformers defines ParakeetForTDT, so without the plugin, vLLM resolves the model to the Transformers backend before it checks that table, and the entry would never be used.
• AI assistance (Claude Code) was used to write the plugin and this PR. I reviewed the changed lines and ran the validation above.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[mergify]

Documentation preview: https://vllm--65005.org.readthedocs.build/en/65005/

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[DarkLight1337]

Thank you!

[DarkLight1337]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99078 for commit 9b4ced1de818.

ℹ️ This PR is 26 commits behind upstream main. Merged upstream main into this branch as 9b4ced1de818 before starting CI. Pull the branch before pushing more changes.

[mergify]

Hi @Alex-Wengg, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[Alex-Wengg]

/ci run

[github-actions]

❌ CI already ran on this commit: https://buildkite.com/vllm/ci/builds/99078. No new CI build was started. Push new changes to run CI again, use /ci retry to rerun failed jobs, or ask a reviewer with write access to rerun it.

[Alex-Wengg]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99199 for commit 70fa36d00353.