Add JavaScript (WebAssembly) API for Omnilingual ASR CTC models - #2781
Conversation
dosubot
Bot
added
the
size:L
WalkthroughThis pull request adds Omnilingual ASR CTC model support to Sherpa-ONNX across multiple layers. Changes include a new Node.js example script, updated test automation, WASM configuration handling, and C++ binding updates to support the omnilingual model configuration alongside existing ASR models. Changes
Sequence Diagram(s)sequenceDiagram
participant App as Application
participant JS as sherpa-onnx-asr.js
participant Config as Model Config
participant WASM as WASM Module
App->>JS: initSherpaOnnxOfflineModelConfig(config)
JS->>Config: Create omnilingual config
Config-->>JS: omnilingual object
JS->>WASM: Allocate memory buffer
WASM-->>JS: Buffer with all model configs
JS->>JS: Copy omnilingual data to buffer
JS-->>App: Return config with omnilingual field
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Possibly related PRs
Suggested labels
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (5)
🧰 Additional context used🧬 Code graph analysis (2)wasm/nodejs/sherpa-onnx-wasm-nodejs.cc (1)
wasm/asr/sherpa-onnx-asr.js (2)
🔇 Additional comments (11)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @csukuangfj, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly expands the capabilities of the JavaScript (WebAssembly) API by integrating support for Omnilingual ASR CTC models. This enhancement allows developers to perform offline speech recognition across a wide range of languages directly within Node.js environments, complete with a new example and updated testing infrastructure to ensure functionality. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for Omnilingual ASR CTC models to the JavaScript/WebAssembly API. The changes are well-structured and consistently follow the existing patterns in the codebase, including updates to the test scripts, documentation, and the core JavaScript wrapper. My review includes a few suggestions to improve code maintainability and documentation clarity, such as using variables for long repeated filenames and removing unused code.
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
| tar xvf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
| rm sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
|
|
||
| node ./test-offline-omnilingual-asr-ctc.js | ||
|
|
||
| rm -rf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12 |
There was a problem hiding this comment.
The script repeats a long model filename and directory name, which makes it harder to read and maintain. Using a variable for these repeated strings would make the script cleaner and less error-prone. Additionally, the v flag in tar xvf is verbose for CI logs; xf is generally sufficient.
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| tar xvf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| rm sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| node ./test-offline-omnilingual-asr-ctc.js | |
| rm -rf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12 | |
| MODEL_NAME="sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12" | |
| curl -SL -O "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/${MODEL_NAME}.tar.bz2" | |
| tar xf "${MODEL_NAME}.tar.bz2" | |
| rm "${MODEL_NAME}.tar.bz2" | |
| node ./test-offline-omnilingual-asr-ctc.js | |
| rm -rf "${MODEL_NAME}" |
| ## ./test-offline-omnilingual-asr-ctc.js | ||
|
|
||
| [./test-offline-omnilingual-asr-ctc.js](./test-offline-omnilingual-asr-ctc.js) demonstrates | ||
| how to decode a file with a Omnilingual ASR CTC model. In the code we use |
There was a problem hiding this comment.
There is a minor grammatical error. It should be "an Omnilingual" instead of "a Omnilingual".
| how to decode a file with a Omnilingual ASR CTC model. In the code we use | |
| how to decode a file with an Omnilingual ASR CTC model. In the code we use |
| wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
| tar xvf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
| rm sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | ||
|
|
||
| node ./test-offline-omnilingual-asr-ctc.js |
There was a problem hiding this comment.
The example commands repeat a very long filename, which can be hard to read and is prone to typos. Using a shell variable would make the example clearer and easier for users to adapt. Also, tar xvf is verbose; xf is sufficient.
| wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| tar xvf sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| rm sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12.tar.bz2 | |
| node ./test-offline-omnilingual-asr-ctc.js | |
| MODEL_NAME="sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12" | |
| wget "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/${MODEL_NAME}.tar.bz2" | |
| tar xf "${MODEL_NAME}.tar.bz2" | |
| rm "${MODEL_NAME}.tar.bz2" | |
| node ./test-offline-omnilingual-asr-ctc.js |
| const fs = require('fs'); | ||
| const {Readable} = require('stream'); | ||
| const wav = require('wav'); |
There was a problem hiding this comment.
The modules fs, stream (Readable), and wav are imported but are not used in this file. These unused imports should be removed to keep the code clean.
| modelConfig: { | ||
| omnilingual: { | ||
| model: | ||
| './sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12/model.int8.onnx', |
There was a problem hiding this comment.
The directory path './sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12' is hardcoded here and in other places in the file (lines 17 and 28). To improve maintainability, consider defining this path as a constant at the top of the file and reusing it. For example: const modelDir = './sherpa-onnx-omnilingual-asr-1600-languages-300M-ctc-int8-2025-11-12';
1 participant
Summary by CodeRabbit
New Features
Tests
Documentation