Add JavaScript (WebAssembly) example for ZipVoice TTS - #3341
Conversation
dosubot
Bot
added
the
size:M
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds offline ZipVoice and several Kokoro/Matcha TTS test steps to CI, a new Node.js offline ZipVoice TTS example script, and README documentation describing the ZipVoice offline TTS test and run commands. Changes
Sequence Diagram(s)sequenceDiagram
participant CI as CI runner
participant Remote as Model host (storage)
participant FS as Filesystem
participant Node as Node.js test
participant Lib as sherpa-onnx runtime
CI->>Remote: download model tarballs & ONNX files
Remote-->>CI: tarballs/onnx files
CI->>FS: extract tarballs, place models
CI->>Node: invoke test script (node test-offline-tts-zipvoice-zh-en.js)
Node->>Lib: initialize offline TTS with model paths
Node->>Lib: generate audio (generationConfig with ref wave/text)
Lib-->>Node: generated WAV bytes
Node->>FS: write WAV file(s)
CI->>FS: list WAVs, then remove extracted models/onnx
CI-->>Remote: (cleanup complete)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a new JavaScript example for ZipVoice Text-to-Speech, enabling users to perform zero-shot TTS in both Chinese and English within a Node.js environment. The changes include the addition of the example script, its integration into the CI/CD testing pipeline, and comprehensive documentation to guide users on its execution and model setup. This expands the available TTS demonstrations and showcases the capabilities of ZipVoice within the sherpa-onnx ecosystem. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Pull request overview
Adds a new Node.js (WebAssembly-backed) example demonstrating ZipVoice zero-shot TTS for Chinese/English, and wires it into the Node.js examples documentation and CI smoke test flow.
Changes:
- Add
test-offline-tts-zipvoice-zh-en.jsexample that generates audio usinggenerateWithConfig()with ZipVoice reference audio/text. - Document download + run steps for the new example in
nodejs-examples/README.md. - Extend GitHub CI Node.js npm test script to download the required ZipVoice/vocoder assets and execute the example.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| nodejs-examples/test-offline-tts-zipvoice-zh-en.js | New ZipVoice zh-en zero-shot TTS example using a prompt wav + transcript and generateWithConfig(). |
| nodejs-examples/README.md | Adds a section describing how to run the new ZipVoice TTS example and download model artifacts. |
| .github/scripts/test-nodejs-npm.sh | CI: downloads ZipVoice model + vocoder, runs the new example, then cleans up downloaded artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new JavaScript example for ZipVoice TTS, complete with the necessary test script modifications and documentation updates. My review focuses on enhancing the code quality of the new example file. I've provided a suggestion to use const for variables that are not reassigned, which is a JavaScript best practice for improving code robustness and clarity.
| let zipvoice = { | ||
| encoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/encoder.int8.onnx', | ||
| decoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/decoder.int8.onnx', | ||
| tokens: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/tokens.txt', | ||
| lexicon: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/lexicon.txt', | ||
| dataDir: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/espeak-ng-data', | ||
| vocoder: './vocos_24khz.onnx', | ||
| }; | ||
|
|
||
| let offlineTtsModelConfig = { | ||
| offlineTtsZipVoiceModelConfig: zipvoice, | ||
| numThreads: 1, | ||
| debug: 1, // set it to 1 to see verbose logs; 0 to disable logs | ||
| provider: 'cpu', | ||
| }; | ||
|
|
||
| let offlineTtsConfig = { | ||
| offlineTtsModelConfig: offlineTtsModelConfig, | ||
| maxNumSentences: 1, | ||
| }; |
There was a problem hiding this comment.
The variables zipvoice, offlineTtsModelConfig, and offlineTtsConfig are not reassigned within this function. It is a best practice in modern JavaScript to use const for variables that do not change, as it prevents accidental reassignment and makes the code easier to reason about.
| let zipvoice = { | |
| encoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/encoder.int8.onnx', | |
| decoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/decoder.int8.onnx', | |
| tokens: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/tokens.txt', | |
| lexicon: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/lexicon.txt', | |
| dataDir: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/espeak-ng-data', | |
| vocoder: './vocos_24khz.onnx', | |
| }; | |
| let offlineTtsModelConfig = { | |
| offlineTtsZipVoiceModelConfig: zipvoice, | |
| numThreads: 1, | |
| debug: 1, // set it to 1 to see verbose logs; 0 to disable logs | |
| provider: 'cpu', | |
| }; | |
| let offlineTtsConfig = { | |
| offlineTtsModelConfig: offlineTtsModelConfig, | |
| maxNumSentences: 1, | |
| }; | |
| const zipvoice = { | |
| encoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/encoder.int8.onnx', | |
| decoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/decoder.int8.onnx', | |
| tokens: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/tokens.txt', | |
| lexicon: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/lexicon.txt', | |
| dataDir: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/espeak-ng-data', | |
| vocoder: './vocos_24khz.onnx', | |
| }; | |
| const offlineTtsModelConfig = { | |
| offlineTtsZipVoiceModelConfig: zipvoice, | |
| numThreads: 1, | |
| debug: 1, // set it to 1 to see verbose logs; 0 to disable logs | |
| provider: 'cpu', | |
| }; | |
| const offlineTtsConfig = { | |
| offlineTtsModelConfig: offlineTtsModelConfig, | |
| maxNumSentences: 1, | |
| }; |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
nodejs-examples/test-offline-tts-zipvoice-zh-en.js (1)
48-52: Guardtts.free()withfinallyfor failure-safe cleanup.If
generateWithConfig()orsave()throws, cleanup is skipped. Wrap lifecycle intry/finally.♻️ Proposed fix
-const tts = createOfflineTts(); -const audio = tts.generateWithConfig(text, generationConfig); -tts.save('./test-zipvoice-zh-en.wav', audio); -console.log('Saved to test-zipvoice-zh-en.wav successfully.'); -tts.free(); +let tts; +try { + tts = createOfflineTts(); + const audio = tts.generateWithConfig(text, generationConfig); + tts.save('./test-zipvoice-zh-en.wav', audio); + console.log('Saved to test-zipvoice-zh-en.wav successfully.'); +} finally { + tts?.free(); +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@nodejs-examples/test-offline-tts-zipvoice-zh-en.js` around lines 48 - 52, The code calls createOfflineTts(), then generateWithConfig() and save(), but tts.free() is unconditional and skipped if an exception occurs; wrap the lifecycle of the TTS instance in a try/finally so that createOfflineTts() -> try { const audio = tts.generateWithConfig(text, generationConfig); tts.save(...); } finally { tts.free(); } ensuring the TTS instance (tts) is freed even when generateWithConfig or save throws; reference createOfflineTts, generateWithConfig, save, and free to locate and modify the block.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@nodejs-examples/test-offline-tts-zipvoice-zh-en.js`:
- Around line 48-52: The code calls createOfflineTts(), then
generateWithConfig() and save(), but tts.free() is unconditional and skipped if
an exception occurs; wrap the lifecycle of the TTS instance in a try/finally so
that createOfflineTts() -> try { const audio = tts.generateWithConfig(text,
generationConfig); tts.save(...); } finally { tts.free(); } ensuring the TTS
instance (tts) is freed even when generateWithConfig or save throws; reference
createOfflineTts, generateWithConfig, save, and free to locate and modify the
block.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 56f7483f-b9ab-4dff-a745-e7611e6eed5b
📒 Files selected for processing (3)
.github/scripts/test-nodejs-npm.shnodejs-examples/README.mdnodejs-examples/test-offline-tts-zipvoice-zh-en.js
Summary by CodeRabbit
New Features
Tests
Documentation