Fix greedy search decoding for Nemo streaming transducer - #3785
Merged
Merged
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
dosubot
Bot
added
the
size:M
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe PR adds persistent NeMo decoder output storage and accessors, updates greedy decoding to reuse and save decoder output/state pairs, clears stored output on reset, and replaces hardcoded waveform padding with validated configurable left- and right-padding options. ChangesNeMo decoder flow
Configurable stream padding
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant DecodeOne
participant OnlineStream
participant RunDecoder
DecodeOne->>OnlineStream: read stored decoder output and states
DecodeOne->>RunDecoder: run decoder when no stored output exists
RunDecoder-->>DecodeOne: return decoder output and states
DecodeOne->>OnlineStream: store decoder output and states
OnlineStream->>OnlineStream: clear decoder output during reset
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sherpa-onnx/csrc/online-transducer-greedy-search-nemo-decoder.cc`:
- Around line 46-58: Update the NeMo decoder reset path to clear both the cached
decoder output and decoder states together, ensuring the next decode treats the
state as uninitialized and invokes RunDecoder. Add a regression test that
decodes after an endpoint reset and verifies the fresh decoder initialization
path is used.
In `@sherpa-onnx/csrc/sherpa-onnx.cc`:
- Around line 86-98: Validate left_padding_second and right_padding_second after
CLI parsing and before any conversion or vector allocation: reject non-finite or
negative values, then ensure each seconds-to-samples calculation does not exceed
INT32_MAX before converting to int or passing it to AcceptWaveform. Emit the
specified error and return -1 on invalid input, covering both padding
registration paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a3cbee41-3d60-44b2-9050-a1fc7baa6f30
📒 Files selected for processing (4)
sherpa-onnx/csrc/online-stream.ccsherpa-onnx/csrc/online-stream.hsherpa-onnx/csrc/online-transducer-greedy-search-nemo-decoder.ccsherpa-onnx/csrc/sherpa-onnx.cc
There was a problem hiding this comment.
Pull request overview
This PR targets more stable and efficient NeMo streaming transducer greedy decoding by persisting decoder context across streaming steps, while also making the CLI example’s audio padding configurable.
Changes:
- Add
--left-padding/--right-paddingoptions to thesherpa-onnxCLI example and apply them when feeding audio to streams. - Cache NeMo decoder output (
Ort::Value) alongside decoder states inOnlineStream, and reuse it across decoding steps. - Update NeMo greedy-search decoding to reuse cached decoder output/state and to persist updated decoder context after each chunk.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
sherpa-onnx/csrc/sherpa-onnx.cc |
Adds configurable left/right padding and applies padding when feeding wave samples. |
sherpa-onnx/csrc/online-transducer-greedy-search-nemo-decoder.cc |
Reuses cached decoder output/state across chunks and persists updated decoder context each call. |
sherpa-onnx/csrc/online-stream.h |
Adds SetNeMoDecoderOut() / GetNeMoDecoderOut() API for caching NeMo decoder output. |
sherpa-onnx/csrc/online-stream.cc |
Implements storage for cached NeMo decoder output and renames/organizes NeMo decoder state members. |
Comments suppressed due to low confidence (1)
sherpa-onnx/csrc/sherpa-onnx.cc:155
right_padding_secondis user-configurable; if it is negative,static_cast<int>(right_padding_second * sampling_rate)can become a hugesize_tand attempt an enormous allocation. Clamp/validate the computed sample count before sizing the padding vector.
std::vector<float> tail_paddings(
static_cast<int>(right_padding_second * sampling_rate), 0);
// Note: We can call AcceptWaveform() multiple times.
s->AcceptWaveform(sampling_rate, tail_paddings.data(),
tail_paddings.size());
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
126
to
128
| void SetNeMoDecoderStates(std::vector<Ort::Value> decoder_states) { | ||
| decoder_states_ = std::move(decoder_states); | ||
| nemo_decoder_states_ = std::move(decoder_states); | ||
| } |
Comment on lines
+100
to
+101
| s->SetNeMoDecoderOut(std::move(decoder_output_pair.first)); | ||
| s->SetNeMoDecoderStates(std::move(decoder_output_pair.second)); |
Comment on lines
+144
to
147
| std::vector<float> left_paddings( | ||
| static_cast<int>(left_padding_second * sampling_rate), 0); | ||
| s->AcceptWaveform(sampling_rate, left_paddings.data(), | ||
| left_paddings.size()); |
dosubot
Bot
added
size:L
This was referenced Aug 12, 2026
This was referenced Sep 15, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit