Repository navigation
Add JavaScript (WebAssembly) API for ten-vad - #2382
Conversation
WalkthroughThis update introduces support for the TEN VAD model alongside Silero VAD in the WebAssembly VAD demo, adds a new GitHub Actions workflow for automated building and publishing of the TEN VAD wasm artifact, enhances the UI and documentation, and updates configuration, build scripts, and model selection logic to accommodate dual VAD models. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Browser
participant JS_Module
participant WASM_Module
User->>Browser: Load VAD demo page
Browser->>JS_Module: Initialize Module (onRuntimeInitialized)
JS_Module->>JS_Module: initVad()
JS_Module->>WASM_Module: fileExists("ten-vad.onnx")
alt ten-vad.onnx exists
JS_Module->>JS_Module: Set TEN VAD config/model path
else silero_vad.onnx exists
JS_Module->>JS_Module: Set Silero VAD config/model path
else
JS_Module->>Browser: Show error (no model)
end
JS_Module->>WASM_Module: createVad(config)
JS_Module->>Browser: Ready for user interaction
Possibly related PRs
Poem
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Pull Request Overview
Add support for the ten-vad model alongside silero-vad in the WebAssembly VAD API, update the UI and documentation, and extend examples and CI workflows to handle the new model.
- Extend C++ bindings and printing logic to include ten-vad configuration.
- Implement JavaScript init/free routines and runtime detection for ten-vad.
- Update UI layout, README assets, example scripts, and CI workflow to support ten-vad.
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| wasm/vad/sherpa-onnx-wasm-main-vad.cc | Updated static_assert and MyPrint to include ten_vad config. |
| wasm/vad/sherpa-onnx-vad.js | Added initSherpaOnnxTenVadModelConfig and free logic for ten-vad. |
| wasm/vad/index.html | Enhanced layout, loading states, and added footer with ten-vad details. |
| wasm/vad/assets/README.md | Added download instructions for ten-vad.onnx. |
| wasm/vad/app-vad.js | Detects presence of ten-vad.onnx and selects appropriate config. |
| wasm/vad/CMakeLists.txt | Updated asset existence check to require at least one VAD model. |
| wasm/vad-asr/app-vad-asr.js | Removed unused placeholder function. |
| wasm/asr/assets/README.md | Corrected sed command to target sherpa-onnx-asr.js. |
| nodejs-examples/test-vad-with-non-streaming-asr-moonshine.js | Updated example to show how to switch to ten-vad and adjusted windowSize logic. |
| .github/workflows/wasm-simd-hf-space-ten-vad.yaml | Added GitHub Actions workflow for building and publishing the ten-vad variant. |
Comments suppressed due to low confidence (2)
wasm/vad/sherpa-onnx-vad.js:67
- The variable
offsetis assigned without alet/constdeclaration, which pollutes the global scope and may lead to unexpected behavior. Declare it locally (e.g.,let offset = 0;).
offset = 0;
wasm/vad/sherpa-onnx-vad.js:146
- The
buffervariable is not defined in this scope, causing a ReferenceError at runtime. Ensure you either declarebufferor return the correct property (e.g.,sileroVad.bufferortenVad.buffer).
buffer: buffer, ptr: ptr, len: len, sileroVad: sileroVad, tenVad: tenVad
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
wasm/vad/assets/README.md (1)
6-6: Fix markdown formatting issues for better documentation quality.Address the static analysis findings to improve markdown compliance:
- Bare URLs: Wrap URLs in angle brackets or use proper markdown link format
- Code blocks: Add language specifiers to fenced code blocks
Apply these formatting improvements:
-https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx +<https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx> -https://github.com/k2-fsa/sherpa-onnx/blob/master/.github/workflows/wasm-simd-hf-space-silero-vad.yaml +<https://github.com/k2-fsa/sherpa-onnx/blob/master/.github/workflows/wasm-simd-hf-space-silero-vad.yaml> -``` +```bash -https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/ten-vad.onnx +<https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/ten-vad.onnx> -https://github.com/k2-fsa/sherpa-onnx/blob/master/.github/workflows/wasm-simd-hf-space-ten-vad.yaml +<https://github.com/k2-fsa/sherpa-onnx/blob/master/.github/workflows/wasm-simd-hf-space-ten-vad.yaml> -``` +```bashAlso applies to: 10-10, 12-15, 20-20, 24-24, 26-32
wasm/vad/sherpa-onnx-vad.js (1)
55-89: Consider refactoring to reduce code duplication.The
initSherpaOnnxTenVadModelConfigfunction is nearly identical toinitSherpaOnnxSileroVadModelConfigexcept for the default window size. Consider extracting a common function to reduce duplication.You could refactor to a common function:
+function initSherpaOnnxVadModelConfigBase(config, Module, defaultWindowSize) { + const modelLen = Module.lengthBytesUTF8(config.model || '') + 1; + const n = modelLen; + const buffer = Module._malloc(n); + const len = 6 * 4; + const ptr = Module._malloc(len); + Module.stringToUTF8(config.model || '', buffer, modelLen); + + let offset = 0; + Module.setValue(ptr, buffer, 'i8*'); + offset += 4; + Module.setValue(ptr + offset, config.threshold || 0.5, 'float'); + offset += 4; + Module.setValue(ptr + offset, config.minSilenceDuration || 0.5, 'float'); + offset += 4; + Module.setValue(ptr + offset, config.minSpeechDuration || 0.25, 'float'); + offset += 4; + Module.setValue(ptr + offset, config.windowSize || defaultWindowSize, 'i32'); + offset += 4; + Module.setValue(ptr + offset, config.maxSpeechDuration || 20, 'float'); + offset += 4; + + return { buffer: buffer, ptr: ptr, len: len }; +} function initSherpaOnnxSileroVadModelConfig(config, Module) { - // ... existing implementation + return initSherpaOnnxVadModelConfigBase(config, Module, 512); } function initSherpaOnnxTenVadModelConfig(config, Module) { - // ... existing implementation + return initSherpaOnnxVadModelConfigBase(config, Module, 256); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
.github/workflows/wasm-simd-hf-space-ten-vad.yaml(1 hunks)nodejs-examples/test-vad-with-non-streaming-asr-moonshine.js(3 hunks)wasm/asr/assets/README.md(1 hunks)wasm/vad-asr/app-vad-asr.js(0 hunks)wasm/vad/CMakeLists.txt(2 hunks)wasm/vad/app-vad.js(1 hunks)wasm/vad/assets/README.md(1 hunks)wasm/vad/index.html(1 hunks)wasm/vad/sherpa-onnx-vad.js(5 hunks)wasm/vad/sherpa-onnx-wasm-main-vad.cc(2 hunks)
💤 Files with no reviewable changes (1)
- wasm/vad-asr/app-vad-asr.js
🧰 Additional context used
🧬 Code Graph Analysis (1)
nodejs-examples/test-vad-with-non-streaming-asr-moonshine.js (2)
nodejs-addon-examples/test_vad_asr_non_streaming_moonshine_microphone.js (2)
windowSize(77-77)vad(57-57)nodejs-addon-examples/test_vad_with_non_streaming_asr_moonshine.js (2)
windowSize(70-70)vad(55-55)
🪛 markdownlint-cli2 (0.17.2)
wasm/vad/assets/README.md
6-6: Bare URL used
(MD034, no-bare-urls)
10-10: Bare URL used
(MD034, no-bare-urls)
12-12: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
20-20: Bare URL used
(MD034, no-bare-urls)
24-24: Bare URL used
(MD034, no-bare-urls)
26-26: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (21)
wasm/asr/assets/README.md (1)
62-62: LGTM: File name correction in sed command.The update correctly targets
./sherpa-onnx-asr.jsinstead of./sherpa-onnx.js, ensuring the sed command operates on the intended ASR-specific JavaScript file.wasm/vad/CMakeLists.txt (2)
5-7: LGTM: Enhanced asset check for dual VAD model support.The updated condition correctly allows either
silero_vad.onnxorten-vad.onnxto be present, enabling flexible VAD model selection while maintaining the requirement that at least one model must be available.
33-33: LGTM: Export SherpaOnnxFileExists for runtime model detection.Adding
SherpaOnnxFileExiststo the exported functions enables JavaScript code to dynamically detect which VAD model files are available at runtime, supporting the flexible model selection mechanism.wasm/vad/assets/README.md (1)
3-33: Excellent documentation structure for dual VAD model support.The updated README provides clear, comprehensive instructions for both silero-vad and ten-vad models, including download URLs, placement instructions, and usage examples. The parallel structure makes it easy for users to understand how to configure either model.
nodejs-examples/test-vad-with-non-streaming-asr-moonshine.js (3)
29-37: Excellent documentation for VAD model switching.The comments clearly explain how to configure and switch between the two VAD models, providing users with straightforward instructions for using ten-vad as an alternative to silero-vad.
47-55: Well-structured tenVad configuration.The tenVad configuration object follows the same structure as sileroVad with appropriate default values. The commented-out model path and empty string default provide a clear pattern for users to enable ten-vad when needed.
82-85: Correct window size selection logic.The conditional logic properly prioritizes tenVad window size when its model is configured (non-empty path), while maintaining backward compatibility by defaulting to sileroVad. This ensures the appropriate window size is used based on the active VAD model.
wasm/vad/sherpa-onnx-wasm-main-vad.cc (3)
17-17: LGTM: Correct static assertion for SherpaOnnxTenVadModelConfig.The static assertion ensures the ten-vad configuration structure maintains the expected 24-byte size (6 * 4), providing compile-time verification of memory layout compatibility.
19-22: LGTM: Updated size assertion for extended VAD config.The updated assertion correctly accounts for the addition of the ten_vad member to the overall VAD model configuration, ensuring the total structure size remains consistent with expectations.
25-25: LGTM: Consistent ten-vad configuration printing.The ten_vad pointer initialization and printing logic perfectly mirror the existing silero_vad implementation, maintaining code consistency and providing users with comprehensive configuration visibility for both VAD models.
Also applies to: 38-45
wasm/vad/index.html (3)
14-16: LGTM! Good CSS pattern for loading states.The
.loadingclass provides a clean way to hide content during initialization, which integrates well with the dynamic status management in the JavaScript code.
20-20: Excellent modern styling improvements.The flexbox layout with centered content and modern typography significantly improves the user experience compared to the basic styling.
47-74: Comprehensive footer enhances user understanding.The footer section provides valuable information about the demo's capabilities, privacy considerations, and contact options. This helps users understand the local processing benefits and how to get support.
wasm/vad/app-vad.js (3)
46-50: LGTM! Good Emscripten integration pattern.The
Module.locateFilefunction follows Emscripten's recommended pattern for customizing file path resolution.
52-73: Well-designed status management system.The
Module.setStatusfunction elegantly handles both status display and loading state management by toggling CSS classes. The integration with the HTML loading states is well thought out.
87-97: Proper memory management in native function call.The
fileExists()function correctly handles UTF-8 string conversion and memory allocation/deallocation when calling the native_SherpaOnnxFileExistsfunction.wasm/vad/sherpa-onnx-vad.js (2)
10-12: LGTM! Consistent cleanup pattern.The recursive cleanup of
tenVadconfiguration follows the same pattern assileroVad, ensuring proper memory management.
119-119: Memory layout calculation and data copying look correct.The memory layout properly accounts for both silero and ten VAD configurations, and the data copying using
Module._CopyHeapis implemented correctly.Also applies to: 142-143
.github/workflows/wasm-simd-hf-space-ten-vad.yaml (3)
3-15: Well-configured workflow triggers and concurrency.The workflow properly handles multiple trigger types (push, tags, manual dispatch) and includes concurrency control to prevent conflicting runs.
83-83: Good security practice with repository owner check.The conditional release creation properly restricts access to authorized repository owners, preventing unauthorized releases.
94-98: Excellent retry mechanism for external services.The use of
nick-fields/retryaction with reasonable timeout and retry settings provides robustness against temporary network or service issues when publishing to external repositories.Also applies to: 131-135
WebAssembly for VAD
Screenshots
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Refactor
Style