Add C++ runtime for Parakeet CTC with QNN. - #3688
Conversation
dosubot
Bot
added
the
size:XL
📝 WalkthroughWalkthroughAdds QNN backend support for NeMo CTC (Parakeet) offline speech recognition. A new ChangesQNN Parakeet CTC Offline Recognizer
Sequence Diagram(s)sequenceDiagram
participant App
participant OfflineRecognizerImpl
participant OfflineRecognizerParakeetCtcQnnImpl
participant OfflineParakeetCtcModelQnn
participant OfflineCtcGreedySearchDecoderRknn
App->>OfflineRecognizerImpl: Create(config, provider="qnn")
OfflineRecognizerImpl->>OfflineRecognizerParakeetCtcQnnImpl: construct(config)
OfflineRecognizerParakeetCtcQnnImpl->>OfflineParakeetCtcModelQnn: construct(model_config)
OfflineParakeetCtcModelQnn-->>OfflineRecognizerParakeetCtcQnnImpl: model ready
App->>OfflineRecognizerParakeetCtcQnnImpl: DecodeStreams(streams)
OfflineRecognizerParakeetCtcQnnImpl->>OfflineRecognizerParakeetCtcQnnImpl: normalize features
OfflineRecognizerParakeetCtcQnnImpl->>OfflineParakeetCtcModelQnn: Run(features) → log_probs
OfflineParakeetCtcModelQnn-->>OfflineRecognizerParakeetCtcQnnImpl: log_probs
OfflineRecognizerParakeetCtcQnnImpl->>OfflineCtcGreedySearchDecoderRknn: Decode(log_probs) → CtcResult
OfflineCtcGreedySearchDecoderRknn-->>OfflineRecognizerParakeetCtcQnnImpl: CtcResult
OfflineRecognizerParakeetCtcQnnImpl->>OfflineRecognizerParakeetCtcQnnImpl: Convert + ITN + HomophoneReplacer
OfflineRecognizerParakeetCtcQnnImpl-->>App: OfflineRecognitionResult written to stream
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for Qualcomm Neural Network (QNN) offline Parakeet CTC models, including the implementation of the QNN model wrapper, the offline recognizer implementation, and integration into the existing offline recognizer factory. Feedback focuses on resolving a member variable shadowing issue in OfflineRecognizerParakeetCtcQnnImpl that could lead to out-of-sync configurations, and adding input/output tensor shape validation in OfflineParakeetCtcModelQnn to prevent potential division-by-zero errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| explicit OfflineRecognizerParakeetCtcQnnImpl( | ||
| const OfflineRecognizerConfig &config) | ||
| : OfflineRecognizerImpl(config), | ||
| config_(config), | ||
| symbol_table_(config_.model_config.tokens), | ||
| model_(std::make_unique<OfflineParakeetCtcModelQnn>( | ||
| config.model_config)) { | ||
| Init(); | ||
| } | ||
|
|
||
| template <typename Manager> | ||
| OfflineRecognizerParakeetCtcQnnImpl(Manager *mgr, | ||
| const OfflineRecognizerConfig &config) | ||
| : OfflineRecognizerImpl(mgr, config), | ||
| config_(config), | ||
| symbol_table_(mgr, config_.model_config.tokens), | ||
| model_(std::make_unique<OfflineParakeetCtcModelQnn>( | ||
| mgr, config.model_config)) { | ||
| Init(); | ||
| } |
There was a problem hiding this comment.
The member variable config_ shadows the base class member OfflineRecognizerImpl::config_. This shadowing causes OfflineRecognizerImpl::SetConfig to only update the base class's config_, leaving the derived class's config_ out of sync and breaking functionality when SetConfig is called.
Please remove config_(config) from the initializer list and use the passed-in config parameter directly to initialize other members.
explicit OfflineRecognizerParakeetCtcQnnImpl(
const OfflineRecognizerConfig &config)
: OfflineRecognizerImpl(config),
symbol_table_(config.model_config.tokens),
model_(std::make_unique<OfflineParakeetCtcModelQnn>(
config.model_config)) {
Init();
}
template <typename Manager>
OfflineRecognizerParakeetCtcQnnImpl(Manager *mgr,
const OfflineRecognizerConfig &config)
: OfflineRecognizerImpl(mgr, config),
symbol_table_(mgr, config.model_config.tokens),
model_(std::make_unique<OfflineParakeetCtcModelQnn>(
mgr, config.model_config)) {
Init();
}| OfflineRecognizerConfig config_; | ||
| SymbolTable symbol_table_; |
There was a problem hiding this comment.
Remove the shadowed config_ member variable to ensure that the base class's config_ is used consistently throughout the class.
| OfflineRecognizerConfig config_; | |
| SymbolTable symbol_table_; | |
| SymbolTable symbol_table_; |
| max_num_frames_ = x_shape[1]; | ||
| feat_dim_ = x_shape[2]; |
There was a problem hiding this comment.
We should validate that max_num_frames_ and feat_dim_ are positive integers to prevent potential division-by-zero or other undefined behaviors in Run().
max_num_frames_ = x_shape[1];
feat_dim_ = x_shape[2];
if (max_num_frames_ <= 0 || feat_dim_ <= 0) {
SHERPA_ONNX_LOGE("Invalid model input shape: max_num_frames=%d, feat_dim=%d",
max_num_frames_, feat_dim_);
SHERPA_ONNX_EXIT(-1);
}| auto out_shape = model_->TensorShape("log_probs"); | ||
| vocab_size_ = out_shape[2]; | ||
|
|
||
| subsampling_factor_ = max_num_frames_ / out_shape[1]; |
There was a problem hiding this comment.
We should validate the output tensor shape of log_probs to ensure it is 3-dimensional and has positive dimensions. This prevents potential division-by-zero when calculating subsampling_factor_ and ensures vocab_size_ is valid.
auto out_shape = model_->TensorShape("log_probs");
if (out_shape.size() != 3 || out_shape[1] <= 0 || out_shape[2] <= 0) {
SHERPA_ONNX_LOGE("Invalid output shape for log_probs");
SHERPA_ONNX_EXIT(-1);
}
vocab_size_ = out_shape[2];
subsampling_factor_ = max_num_frames_ / out_shape[1];![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sherpa-onnx/csrc/offline-nemo-enc-dec-ctc-model-config.cc`:
- Around line 16-17: Update the help text in the po->Register call for
"nemo-ctc-model" in offline-nemo-enc-dec-ctc-model-config.cc to reflect that the
option now accepts compiled artifacts (such as libmodel.so) for QNN usage, not
just model.onnx files. Replace the current description that specifically
mentions "model.onnx" with a more generic description that accurately describes
the supported formats and clarifies that it works with both ONNX and compiled
artifacts in the QNN code path.
In `@sherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.cc`:
- Around line 198-201: The code accesses out_shape[2] and out_shape[1] without
validating that the tensor has sufficient rank or that the values are positive,
which can cause out-of-bounds access or division by zero errors. Add validation
checks after calling model_->TensorShape("log_probs") to ensure out_shape has at
least 3 elements and that out_shape[1] is greater than zero before using it in
the subsampling_factor_ calculation. If validation fails, log an appropriate
error and return early.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b644fc7a-9a21-4b87-97f6-faa4c16886fb
📒 Files selected for processing (8)
sherpa-onnx/csrc/CMakeLists.txtsherpa-onnx/csrc/offline-nemo-enc-dec-ctc-model-config.ccsherpa-onnx/csrc/offline-nemo-enc-dec-ctc-model-config.hsherpa-onnx/csrc/offline-recognizer-impl.ccsherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.ccsherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.hsherpa-onnx/csrc/qnn/offline-recognizer-parakeet-ctc-qnn-impl.hsherpa-onnx/csrc/rknn/offline-ctc-greedy-search-decoder-rknn.cc
…, update help text
There was a problem hiding this comment.
🧹 Nitpick comments (1)
sherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.cc (1)
82-99: 💤 Low valueConsider validating that feature vector size is an exact multiple of
feat_dim_.Integer division at line 83 silently truncates if
features.size() % feat_dim_ != 0. The subsequentresize()preserves orphan floats, causing frame misalignment. While callers should pass properly aligned data, a defensive check would catch upstream bugs early.💡 Optional defensive check
std::vector<float> Run(std::vector<float> features) { + if (features.size() % feat_dim_ != 0) { + SHERPA_ONNX_LOGE( + "Feature vector size %d is not a multiple of feat_dim %d", + static_cast<int32_t>(features.size()), feat_dim_); + } int32_t num_frames = features.size() / feat_dim_;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.cc` around lines 82 - 99, The Run method performs integer division of features.size() by feat_dim_ without validating that the feature vector size is an exact multiple of feat_dim_, which can cause silent truncation and frame misalignment. Add a defensive check immediately after entering the Run method to validate that features.size() modulo feat_dim_ equals zero, and log an error or throw an exception if this validation fails to catch upstream bugs early.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@sherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.cc`:
- Around line 82-99: The Run method performs integer division of features.size()
by feat_dim_ without validating that the feature vector size is an exact
multiple of feat_dim_, which can cause silent truncation and frame misalignment.
Add a defensive check immediately after entering the Run method to validate that
features.size() modulo feat_dim_ equals zero, and log an error or throw an
exception if this validation fails to catch upstream bugs early.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: c7bd9f7d-4a8d-4fdf-8718-9c9dd5c0bace
📒 Files selected for processing (3)
sherpa-onnx/csrc/offline-nemo-enc-dec-ctc-model-config.ccsherpa-onnx/csrc/qnn/offline-parakeet-ctc-model-qnn.ccsherpa-onnx/csrc/qnn/offline-recognizer-parakeet-ctc-qnn-impl.h
🚧 Files skipped from review as they are similar to previous changes (2)
- sherpa-onnx/csrc/qnn/offline-recognizer-parakeet-ctc-qnn-impl.h
- sherpa-onnx/csrc/offline-nemo-enc-dec-ctc-model-config.cc
1 participant
See https://k2-fsa.github.io/sherpa/onnx/qnn/run-executables-on-your-phone-binary.html
for how to run it.
Usage:
Logs:
Summary by CodeRabbit
Release Notes
New Features
Improvements