Export Parakeet TDT models to QNN - #3719
Conversation
dosubot
Bot
added
the
size:L
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces configuration generation and ONNX export scripts for the Parakeet TDT model, including enabling decoder quantization in test scripts. Feedback on these changes includes wrapping the pip package name in quotes within the shell script to prevent globbing issues, and using the length of the vocabulary instead of a loop variable to safely write the blank token index.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| set -ex | ||
|
|
||
| pip install \ | ||
| nemo_toolkit['asr'] \ |
There was a problem hiding this comment.
Unquoted square brackets in shell scripts can trigger globbing/pattern matching in some shells (like zsh), leading to errors like no matches found. It is safer to wrap the package name in double quotes.
| nemo_toolkit['asr'] \ | |
| "nemo_toolkit[asr]" \ |
| with open("./tokens.txt", "w", encoding="utf-8") as f: | ||
| for i, s in enumerate(asr_model.joint.vocabulary): | ||
| f.write(f"{s} {i}\n") | ||
| f.write(f"<blk> {i+1}\n") |
There was a problem hiding this comment.
Relying on the loop variable i outside of the for loop is a code smell and can raise a NameError if the vocabulary is empty. Using len(asr_model.joint.vocabulary) is safer and more explicit.
| f.write(f"<blk> {i+1}\n") | |
| f.write(f"<blk> {len(asr_model.joint.vocabulary)}\n") |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughAdds a new Parakeet-TDT ONNX export wrapper, a run script, a symlinked test entrypoint, and a build-matrix generator script, plus a new GitHub Actions workflow that exports ONNX models and converts them to QNN artifacts across multiple SoCs. Also makes minor edits to shared transducer test/wrapper code. ChangesParakeet-TDT export and QNN workflow
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant CI as GitHub Actions Trigger
participant OnnxJob as onnx Job
participant MatrixJob as generate_build_matrix Job
participant QnnJob as qnn Job
participant QnnSdk as QNN Toolkit
CI->>OnnxJob: Run wrapper.py export (encoder/decoder/joiner)
OnnxJob->>OnnxJob: Package ONNX artifacts into tarball
OnnxJob-->>CI: Upload ONNX artifact
CI->>MatrixJob: Run generate_parakeet_tdt.py
MatrixJob-->>CI: Export build matrix JSON
CI->>QnnJob: Start with matrix + depends on onnx, generate_build_matrix
QnnJob->>QnnJob: Download ONNX artifact
QnnJob->>QnnSdk: Convert encoder/decoder/joiner via qnn-onnx-converter
QnnSdk-->>QnnJob: Quantized model libs
QnnJob->>QnnSdk: Build context binaries
QnnSdk-->>QnnJob: .bin outputs
QnnJob->>QnnJob: Package per-SOC tarballs
QnnJob-->>CI: Upload quant JSON + optional release assets
Possibly related PRs
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
Specifically, the following two models are supported:
Other models may also be converted to QNN using scripts in this PR.
C++ runtime will be added in a separate pull request.
Please see
for exported QNN models.
Summary by CodeRabbit
New Features
Bug Fixes