Fix OHOS APIs for TTS and ASR - #3356
Conversation
dosubot
Bot
added
the
size:L
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the capabilities of the SherpaOnnx HarmonyOS module by integrating support for a wider range of ASR and TTS models. It introduces new configuration options for both streaming and non-streaming speech processing, providing developers with more flexibility and control over model selection and speech generation parameters. These updates aim to enhance the robustness and versatility of speech-related functionalities within HarmonyOS applications. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughThe PR extends the public API surface of the sherpa-onnx HarmonyOS binding by adding new ASR model configuration classes (FireRed, FunASR Nano, Omnilingual, MedASR), TTS model configuration classes (Pocket, Supertonic, Zipvoice), generation config types, and new TTS generation methods with configuration support. Additional fields are added to existing config classes for in-memory buffers and provider selection. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces new ASR and TTS model configurations and corresponding API exports across several files. New model configurations for OfflineFireRedAsrCtcModelConfig, OfflineFireRedAsrModelConfig, OfflineFunASRNanoModelConfig, OfflineMedAsrCtcModelConfig, OfflineOmnilingualAsrCtcModelConfig, OfflineTtsPocketModelConfig, OfflineTtsSupertonicModelConfig, and OfflineTtsZipvoiceModelConfig have been added. Additionally, TtsGenerationConfig and TtsInputWithConfig are introduced for more flexible TTS generation. The VadConfig and OnlineModelConfig classes also received minor updates to include new properties. Overall, the changes are well-structured and extend the functionality of the SherpaOnnxHar library.
| public temperature: number = 0; | ||
| public topP: number = 0; |
There was a problem hiding this comment.
The default values for temperature and topP are set to 0. For language models, temperature typically controls the randomness of predictions (a value of 0 usually means deterministic output), and topP is a nucleus sampling parameter (a value of 0 might disable it or lead to unexpected behavior). It's generally recommended to use small positive values for temperature (e.g., 0.7-1.0) and values between 0 and 1 for topP (e.g., 0.8-0.95) to allow for more natural and diverse text generation. Please consider adjusting these defaults to more appropriate values for typical LLM usage.
There was a problem hiding this comment.
Pull request overview
Updates HarmonyOS ETS bindings/configs to match newer Sherpa-ONNX OHOS APIs for TTS and ASR, adding missing configuration fields and exposing new generation functions.
Changes:
- Add provider support to VAD config.
- Extend streaming ASR configs with token/hotword buffer fields.
- Expand offline TTS/ASR model configs and expose “generate with config” APIs + type/export updates.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/ets/components/Vad.ets | Adds provider to VadConfig and threads it through constructor. |
| harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/ets/components/StreamingAsr.ets | Adds buffer + size fields to model/recognizer configs. |
| harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/ets/components/NonStreamingTts.ets | Adds new TTS model configs and new “with config” generation inputs/APIs. |
| harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/ets/components/NonStreamingAsr.ets | Adds new offline ASR model config types/fields. |
| harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/types/libsherpa_onnx/Index.d.ts | Exposes new native TTS functions in TS typings. |
| harmony-os/SherpaOnnxHar/sherpa_onnx/Index.ets | Re-exports newly added config types and inputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| public sampleRate: number; | ||
| public debug: boolean; | ||
| public numThreads: number; | ||
| public provider: string = 'cpu'; | ||
|
|
||
| public constructor(sileroVad: SileroVadConfig, tenVad: TenVadConfig, sampleRate: number, debug: boolean, numThreads: number) { | ||
| public constructor(sileroVad: SileroVadConfig, tenVad: TenVadConfig, sampleRate: number, debug: boolean, | ||
| numThreads: number, provider: string = 'cpu') { | ||
| this.sileroVad = sileroVad; | ||
| this.tenVad = tenVad; | ||
| this.sampleRate = sampleRate; | ||
| this.debug = debug; | ||
| this.numThreads = numThreads; | ||
| this.provider = provider; | ||
| } |
| @@ -67,6 +67,8 @@ export class OnlineModelConfig { | |||
| public modelType: string = ''; | |||
| public modelingUnit: string = "cjkchar"; | |||
| public bpeVocab: string = ''; | |||
| public hotwordsBuf: string = ''; | ||
| public hotwordsBufSize: number = 0; | ||
| public hr: HomophoneReplacerConfig = new HomophoneReplacerConfig(); |
| public referenceSampleRate: number = 0; | ||
| public referenceText: string = ''; | ||
| public numSteps: number = 5; | ||
| public extra: object = {}; |
| public enableExternalBuffer: boolean = true; | ||
| public callback?: (data: TtsCallbackData) => number; | ||
| } | ||
|
|
||
| export class TtsInputWithConfig { | ||
| public text: string = ''; | ||
| public generationConfig: TtsGenerationConfig = new TtsGenerationConfig(); | ||
| public enableExternalBuffer: boolean = true; |
| } | ||
|
|
||
| generateWithConfig(input: TtsInputWithConfig): TtsOutput { | ||
| return offlineTtsGenerateWithConfig(this.handle, input) as TtsOutput; |
Summary by CodeRabbit
Release Notes