Skip to content

Add JavaScript (node-addon) API for ten-vad #2383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
csukuangfj merged 2 commits into from
Jul 12, 2025
Merged

Add JavaScript (node-addon) API for ten-vad #2383

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d0628e1
Add JavaScript (node-addon) API for ten-vad
csukuangfj Jul 12, 2025
b8a17f6
add max speech duration to vad config
csukuangfj Jul 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion harmony-os/SherpaOnnxHar/sherpa_onnx/Index.ets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export { listRawfileDir, readWave, readWaveFromBinary, } from "libsherpa_onnx.so";

export { CircularBuffer, SileroVadConfig, SpeechSegment, Vad, VadConfig, } from './src/main/ets/components/Vad';
export { CircularBuffer, SileroVadConfig, TenVadConfig, SpeechSegment, Vad, VadConfig, } from './src/main/ets/components/Vad';


export { Samples,
Expand Down
21 changes: 21 additions & 0 deletions harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/cpp/vad.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,25 @@ static SherpaOnnxSileroVadModelConfig GetSileroVadConfig(
return c;
}

static SherpaOnnxTenVadModelConfig GetTenVadConfig(const Napi::Object &obj) {
SherpaOnnxTenVadModelConfig c;
memset(&c, 0, sizeof(c));

if (!obj.Has("tenVad") || !obj.Get("tenVad").IsObject()) {
return c;
}

Napi::Object o = obj.Get("tenVad").As<Napi::Object>();
SHERPA_ONNX_ASSIGN_ATTR_STR(model, model);
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(threshold, threshold);
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(min_silence_duration, minSilenceDuration);
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(min_speech_duration, minSpeechDuration);
SHERPA_ONNX_ASSIGN_ATTR_INT32(window_size, windowSize);
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(max_speech_duration, maxSpeechDuration);

Comment on lines +311 to +312

Copilot AI Jul 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TenVadConfig does not have a maxSpeechDuration property, but this line maps max_speech_duration. Either add maxSpeechDuration to TenVadConfig or remove this mapping to match the class definition.

Suggested change
SHERPA_ONNX_ASSIGN_ATTR_FLOAT(max_speech_duration, maxSpeechDuration);

Copilot uses AI. Check for mistakes.
return c;
}

static Napi::External<SherpaOnnxVoiceActivityDetector>
CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
Napi::Env env = info.Env();
Expand Down Expand Up @@ -339,6 +358,7 @@ CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
SherpaOnnxVadModelConfig c;
memset(&c, 0, sizeof(c));
c.silero_vad = GetSileroVadConfig(o);
c.ten_vad = GetTenVadConfig(o);

SHERPA_ONNX_ASSIGN_ATTR_INT32(sample_rate, sampleRate);
SHERPA_ONNX_ASSIGN_ATTR_INT32(num_threads, numThreads);
Expand Down Expand Up @@ -369,6 +389,7 @@ CreateVoiceActivityDetectorWrapper(const Napi::CallbackInfo &info) {
SherpaOnnxCreateVoiceActivityDetector(&c, buffer_size_in_seconds);
#endif
SHERPA_ONNX_DELETE_C_STR(c.silero_vad.model);
SHERPA_ONNX_DELETE_C_STR(c.ten_vad.model);
SHERPA_ONNX_DELETE_C_STR(c.provider);

return Napi::External<SherpaOnnxVoiceActivityDetector>::New(
Expand Down
29 changes: 26 additions & 3 deletions harmony-os/SherpaOnnxHar/sherpa_onnx/src/main/ets/components/Vad.ets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,25 +23,48 @@ export class SileroVadConfig {
public minSpeechDuration: number;
public minSilenceDuration: number;
public windowSize: number;
public maxSpeechDuration: number;

public constructor(model: string, threshold: number, minSpeechDuration: number, minSilenceDuration: number,
windowSize: number) {
windowSize: number, maxSpeechDuration: number = 20) {
this.model = model;
this.threshold = threshold;
this.minSpeechDuration = minSpeechDuration;
this.minSilenceDuration = minSilenceDuration;
this.windowSize = windowSize;
this.maxSpeechDuration = maxSpeechDuration
}
}

export class TenVadConfig {
public model: string;
public threshold: number;
public minSpeechDuration: number;
public minSilenceDuration: number;
public windowSize: number;
public maxSpeechDuration: number;

public constructor(model: string, threshold: number, minSpeechDuration: number, minSilenceDuration: number,
windowSize: number, maxSpeechDuration: number = 20) {
this.model = model;
this.threshold = threshold;
this.minSpeechDuration = minSpeechDuration;
this.minSilenceDuration = minSilenceDuration;
this.windowSize = windowSize;
this.maxSpeechDuration = maxSpeechDuration
}
}

export class VadConfig {
public sileroVad: SileroVadConfig;
public tenVad: TenVadConfig;
public sampleRate: number;
public debug: boolean;
public numThreads: number;

public constructor(sileroVad: SileroVadConfig, sampleRate: number, debug: boolean, numThreads: number) {
public constructor(sileroVad: SileroVadConfig, tenVad: TenVadConfig, sampleRate: number, debug: boolean, numThreads: number) {
this.sileroVad = sileroVad;
this.tenVad = tenVad;
this.sampleRate = sampleRate;
this.debug = debug;
this.numThreads = numThreads;
Expand Down Expand Up @@ -127,4 +150,4 @@ export class Vad {
flush(): void {
voiceActivityDetectorFlush(this.handle);
}
}
}
22 changes: 20 additions & 2 deletions harmony-os/SherpaOnnxVadAsr/entry/src/main/ets/workers/NonStreamingAsrWithVadWorker.ets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
OnlineRecognizerResult,
readWaveFromBinary,
SileroVadConfig,
TenVadConfig,
SpeechSegment,
Vad,
VadConfig,
Expand All @@ -31,6 +32,13 @@ function initVad(context: Context): Vad {
0.5,
512,
),
new TenVadConfig(
'', // set it to ten-vad.onnx to use ten-vad
0.5,
0.25,
0.5,
256,
),
16000,
true,
1,
Expand Down Expand Up @@ -93,7 +101,12 @@ function decodeFile(filename: string): string {
console.log(`samples length ${wave.samples.length}`);
const resultList: string[] = [];

const windowSize: number = vad.config.sileroVad.windowSize;
let windowSize: number = vad.config.sileroVad.windowSize;

if (vad.config.tenVad.model != '') {
windowSize = vad.config.tenVad.windowSize;
}

for (let i = 0; i < wave.samples.length; i += windowSize) {
const thisWindow: Float32Array = wave.samples.subarray(i, i + windowSize)
vad.acceptWaveform(thisWindow);
Expand Down Expand Up @@ -138,7 +151,12 @@ function decodeFile(filename: string): string {
function decodeMic(samples: Float32Array) {
const resultList: string[] = [];

const windowSize: number = vad.config.sileroVad.windowSize;
let windowSize: number = vad.config.sileroVad.windowSize;

if (vad.config.tenVad.model != '') {
windowSize = vad.config.tenVad.windowSize;
}

for (let i = 0; i < samples.length; i += windowSize) {
const thisWindow: Float32Array = samples.subarray(i, i + windowSize)
vad.acceptWaveform(thisWindow);
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_moonshine_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_nemo_ctc_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_paraformer_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_sense_voice_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,9 +95,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_transducer_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_whisper_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_asr_non_streaming_zipformer_ctc_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,11 @@ ai.on('data', data => {
console.log(`${index}: ${text}`);

const filename = `${index}-${text}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
26 changes: 22 additions & 4 deletions nodejs-addon-examples/test_vad_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,28 @@ const sherpa_onnx = require('sherpa-onnx-node');
function createVad() {
// please download silero_vad.onnx from
// https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/silero_vad.onnx
//
// OR
//
// please download ten-vad.onnx from
// https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/ten-vad.onnx
const config = {
sileroVad: {
// model: '',
model: './silero_vad.onnx',
threshold: 0.5,
minSpeechDuration: 0.25,
minSilenceDuration: 0.5,
windowSize: 512,
},
tenVad: {
model: '',
// model: './ten-vad.onnx',
threshold: 0.5,
minSpeechDuration: 0.25,
minSilenceDuration: 0.5,
windowSize: 256,
},
sampleRate: 16000,
debug: true,
numThreads: 1,
Expand Down Expand Up @@ -47,7 +61,10 @@ const ai = new portAudio.AudioIO({
let printed = false;
let index = 0;
ai.on('data', data => {
const windowSize = vad.config.sileroVad.windowSize;
const windowSize = vad.config.sileroVad.model != '' ?
vad.config.sileroVad.windowSize :
vad.config.tenVad.windowSize;
Comment on lines +64 to +66

Copilot AI Jul 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition checks sileroVad.model, which is always non-empty by default, causing tenVad.windowSize never to be used. Consider switching the check to vad.config.tenVad.model != '' so that tenVad.windowSize is selected when its model is set.

Suggested change
const windowSize = vad.config.sileroVad.model != '' ?
vad.config.sileroVad.windowSize :
vad.config.tenVad.windowSize;
const windowSize = vad.config.tenVad.model != '' ?
vad.config.tenVad.windowSize :
vad.config.sileroVad.windowSize;

Copilot uses AI. Check for mistakes.
Comment on lines +64 to +66

Copilot AI Jul 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition checks sileroVad.model to choose the window size but should instead check tenVad.model to properly select the Ten VAD configuration when provided.

Suggested change
const windowSize = vad.config.sileroVad.model != '' ?
vad.config.sileroVad.windowSize :
vad.config.tenVad.windowSize;
const windowSize = vad.config.tenVad.model != '' ?
vad.config.tenVad.windowSize :
vad.config.sileroVad.windowSize;

Copilot uses AI. Check for mistakes.

buffer.push(new Float32Array(data.buffer));
while (buffer.size() > windowSize) {
const samples = buffer.get(buffer.head(), windowSize);
Expand All @@ -66,9 +83,10 @@ ai.on('data', data => {
const segment = vad.front();
vad.pop();
const filename = `${index}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');
sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down
8 changes: 5 additions & 3 deletions nodejs-addon-examples/test_vad_spoken_language_identification_microphone.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,11 @@ ai.on('data', data => {
const fullLang = display.of(lang);

const filename = `${index}-${fullLang}-${
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`;
new Date()
.toLocaleTimeString('en-US', {hour12: false})
.split(' ')[0]}.wav`
.replace(/:/g, '-');
Comment on lines 93 to +97

Copilot AI Jul 12, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The filename sanitization logic is duplicated across multiple example scripts. Consider extracting a helper function (e.g., sanitizeFilename) to improve reuse and reduce duplication.

Copilot uses AI. Check for mistakes.

sherpa_onnx.writeWave(
filename,
{samples: segment.samples, sampleRate: vad.config.sampleRate});
Expand Down