[BCI] QVAC-17057 feat: add bci-whispercpp POC with batch transcription by sharmaraju352 · Pull Request #1583 · tetherto/qvac

sharmaraju352 · 2026-04-15T07:09:26Z

Summary

Adds new @qvac/bci-whispercpp package for brain-computer interface neural signal transcription using a modified whisper.cpp backend
Includes C++ native addon (NeuralProcessor, BCIModel, BCIConfig), CMake + vcpkg build system with BCI-specific whisper.cpp fork (tetherto/qvac-ext-lib-whisper.cpp v1.8.4)
JavaScript API exposes BCIWhispercpp class (using createJobHandler + exclusiveRunQueue from @qvac/infer-base) with batch transcribeFile/transcribe returning QvacResponse
Integration tests for load/destroy, batch transcription, and full WER measurement across 5 neural signal test samples
POC demonstrates 6.0% average WER

Results

Sample	Ground Truth	Native Output	WER
0	"You can see the code at this point as well."	"You can see the good at this point as well."	10.0%
1	"How does it keep the cost down?"	"How does it keep the cost down?"	0.0%
2	"Not too controversial."	"Not too controversial."	0.0%
3	"The jury and a judge work together on it."	"The jury and a judge work together on it."	0.0%
4	"Were quite vocal about it."	"We're quite vocal about it."	20.0%
Average			6.0%

Test plan

npm run build succeeds (native addon compiles with whisper-cpp v1.8.4 from tetherto fork)
bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin produces transcription output
bare examples/transcribe-neural.js --batch — all 5 samples transcribed, 6.0% average WER
npm run test:integration passes (3/3 tests, 10/10 assertions — load/destroy + batch transcription + WER measurement)
npm run test:cpp passes (7/7 GoogleTest tests including mel layout validation)
npm run lint passes (standardjs)

Made with Cursor

github-actions · 2026-04-15T07:11:30Z

Tier-based Approval Status

**PR Tier:** TIER1

**Current Status:** ✅ APPROVED

**Requirements:**
- 1 Team Member approval ✅ (1/1)
- 1 Team Lead OR Management approval ✅ (1/1)



---
*This comment is automatically updated when reviews change.*

GustavoA1604

Need to run JS lint as well

…n, fix Linux linkage - Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue from @qvac/infer-base instead of manual promise plumbing, matching the TranscriptionWhispercpp / LlmLlamacpp addon pattern - Constructor now takes { files: { model }, logger, opts } (was { modelPath }) - transcribe/transcribeFile/transcribeStream return QvacResponse - Add unload(), getState(), exclusiveRunQueue-serialized destroy() - Add @qvac/infer-base dependency Address all review feedback from Gustavo (PR #1583): - Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms) - Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED - Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp - Add bounds validation for all BCIConfig numeric params - Throw on unknown config keys (was silently ignored) - Consume gpu_device in context params - Collect whisper timings in runtimeStats() - Trim unused BCIErrors enum values, map codes to distinct names - Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js - Fix _activeJobId race: set after native acceptance - Remove unimplemented bciConfig params from JS whitelist + index.d.ts - Promote hardcoded kernel-trim threshold to named constant - Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs) - Rename bci-addon.test.js → addon.test.js - Replace t.skip() with proper assertions - Fix day_idx handling in tests/examples (group by day, pass to config) - Generate comprehensive NOTICE file - Update vcpkg overlay to v1.8.4 description Fix Linux C++ test linkage: - Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++ - Add linux-clang toolchain (clang-19) - Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds Made-with: Cursor

…nscription Add a new @qvac/bci-whispercpp addon that transcribes brain-computer interface neural signals into text using a modified whisper.cpp backend. This POC includes: - C++ native addon with BCI model inference (NeuralProcessor, BCIModel, BCIConfig) built on the qvac addon-cpp framework - CMake + vcpkg build system with whisper-cpp overlay ports carrying BCI-specific patches (variable conv1 kernel, windowed attention) - JavaScript API: BCIWhispercpp class with batch transcribeFile/transcribe - Integration tests for load/destroy and batch transcription - Example script and model conversion tooling - WER utility for accuracy measurement Streaming transcription will be added in a follow-up PR (QVAC-17062). Made-with: Cursor

…n, fix Linux linkage - Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue from @qvac/infer-base instead of manual promise plumbing, matching the TranscriptionWhispercpp / LlmLlamacpp addon pattern - Constructor now takes { files: { model }, logger, opts } (was { modelPath }) - transcribe/transcribeFile return QvacResponse - Add unload(), getState(), exclusiveRunQueue-serialized destroy() - Add @qvac/infer-base dependency Address all review feedback from Gustavo (PR #1583): - Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms) - Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED - Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp - Add bounds validation for all BCIConfig numeric params - Throw on unknown config keys (was silently ignored) - Consume gpu_device in context params - Collect whisper timings in runtimeStats() - Trim unused BCIErrors enum values, map codes to distinct names - Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js - Fix _activeJobId race: set after native acceptance - Remove unimplemented bciConfig params from JS whitelist + index.d.ts - Promote hardcoded kernel-trim threshold to named constant - Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs) - Rename bci-addon.test.js → addon.test.js - Replace t.skip() with proper assertions - Fix day_idx handling in tests/examples (group by day, pass to config) - Generate comprehensive NOTICE file - Update vcpkg overlay to v1.8.4 description Fix Linux C++ test linkage: - Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++ - Add linux-clang toolchain (clang-19) - Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds Made-with: Cursor

…ayer flash attn Update whisper-cpp overlay to 5645ad60 which includes: - Cached window_mask recompute for exp_n_audio_ctx overrides - Per-layer flash attention (upper encoder layers use FA even with BCI) - std::abs instead of C abs in mask computation Made-with: Cursor

Update overlay to tetherto/qvac-ext-lib-whisper.cpp@3e91e3a4 which addresses jpgaribotti's review on PR #10: 1. Extract compute_window_mask() helper to eliminate duplicated O(n_ctx^2) mask fill logic 2. Guard encode-time mask block with hparams.is_bci 3. Add is_bci to graph builder window_mask guard 4. Validate BCI hparams (conv1_kernel > 0, window_size >= 0) 5. Document n_mels > 256 threshold convention Bump port-version to 3. Made-with: Cursor

Address Gustavo's review feedback: test fixtures (neural_sample_*.bin) are gitignored but the PR had no way for developers to obtain them. Rewrite download-models.sh to fetch both models and test fixtures from the bci-test-assets-v0.1.0 GitHub release. Supports --models, --fixtures, or both (default). Made-with: Cursor

…cleanup - Bump whisper-cpp override in vcpkg.json from 1.7.5.1 to 1.8.4 to match the overlay port version - Move gtest to a vcpkg "tests" feature so it is only pulled when BUILD_TESTING=ON - Fix PaddedFramesAreZero test: use mel-major indexing (data[bin * n_frames + frame]) matching the actual processToMel layout - Remove four unused overlay patch files (0001–0004) now that portfile.cmake fetches from the tetherto fork with patches baked in - Add TODO comment in download-models.sh noting the temporary personal fork for release assets Made-with: Cursor

…docs accuracy - Wrap transcribe() in exclusiveRunQueue to prevent race between inference and unload/destroy - Use find_last_of("/\\") in loadEmbedderIfNeeded for Windows compat - Add empty-buffer guard in bci.js append() before end-of-job - Update download-models.sh to use tetherto/qvac release repo - Add transformers to NOTICE and README model conversion prerequisites - Fix README WER table to match actual live test results (6.0% avg) - Fix BCI_V184_COMPAT.md stale test filename and overlay ref - Remove unused bci_wer_vs_expected field from manifest.json - Update whisper.cpp patches section to reflect fork-based overlay Made-with: Cursor

- Fix unload/destroy race: call destroyInstance() before _job.fail() so the native side stops before the JS job is failed, and remove redundant cancel() call (destroyInstance already cancels internally) - Wrap BCIInterface construction in try/catch so a native init failure sets addon=null and throws a structured QvacErrorAddonBCI - Change JSAdapter loadContextParams/loadMiscParams/loadBCIParams to return void (callers already mutate via reference, return was dead) - Add dayIdx bounds-check warning in BCIModel::process when the value falls outside [0, numDays-1] before silent clamping - Promote hardcoded gaussian smoothing params (std=2.0, kernel=100) to named constants K_SMOOTH_KERNEL_STD / K_SMOOTH_KERNEL_SIZE - Add NeuralProcessor::getNumDays() accessor for the bounds check - Remove [key: string]: unknown escape hatch from WhisperConfig in index.d.ts; enumerate all valid keys explicitly - Fix test:cpp:run script to use direct path instead of cd && chain Made-with: Cursor

ogad-tether

I found two blocking issues and left inline comments with details: 1) transcribe() currently releases the exclusive-run slot before the returned response settles, which can stale an in-flight response and still trip JOB_ALREADY_RUNNING on the native side under concurrent calls; and 2) the published exports map does not match the low-level subpaths documented in the README, so those imports will fail after publish.

Address ogad-tether review feedback on PR #1583: 1. Inference queue: transcribe() now holds its slot until the response settles via _enqueueInference(), matching the pattern from TranscriptionWhispercpp._enqueueExclusiveRunResponse(). Previously the exclusiveRunQueue released the slot as soon as runJob() was accepted, allowing a second concurrent transcribe() to race in and either clobber the first response or get rejected by the native side. 2. Exports map: add ./bci, ./bci.js, and ./binding subpath exports so the low-level BCIInterface API documented in the README is accessible after publish. The exports map previously only exposed ./binding.js, blocking require('@qvac/bci-whispercpp/bci'). Made-with: Cursor

…s review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor

GustavoA1604

Please keep the package name bci-whispercpp. We are going to rename the other qvac-lib-infer later

… validation Move the addon package back to packages/bci-whispercpp, remove unneeded overlay/docs files requested in review, and tighten JS/C++ lifecycle/config safety checks to prevent invalid-state and malformed-input issues. Made-with: Cursor

… config - Replace cd && chain in test:cpp:run with direct path (CLAUDE.md compliance) - Route whisper_log_set through addon-cpp logger instead of silencing with once_flag, preventing inter-addon log handler clobber when BCI and transcription-whispercpp coexist in the same process - Fix stats heuristic in bci.js _addonOutputCallback to match actual BCIModel::runtimeStats keys (tokensPerSecond/totalWallMs, not the audio-addon keys audioDurationMs/totalSamples) - Drain _inferenceQueueWaiter in unload()/destroy() before calling destroyInstance(), closing the race where destroy could fire while process() is mid-execution on the native thread - Remove auto-load in BCIModel::process — throw immediately if context is null instead of lazy-loading outside the controlled lifecycle - Remove dead set_weights_for_file snake_case stub and unused <span> - Add qvac-lint-cpp to vcpkg.json dependencies (matches all peer addons) - Remove empty qvac-lint-cpp overlay directory (per Gustavo review) - Remove stale bci_wer/bci_transcription from manifest.json - Stop gitignoring package-lock.json (match monorepo convention) - Move computeWER into BCIWhispercpp namespace in index.d.ts - Downgrade @types/node to ^22.15.3, remove bare-fs from devDeps - Fix PR template code blocks from typescript to javascript Made-with: Cursor

…red errors, lifecycle safety Align bci-whispercpp with monorepo conventions and fix code quality issues found during thorough review of the POC implementation. Build/config: - .gitignore aligned with peer addons (package-lock.json, .npmrc, IDE files, vcpkg cache, generated test bundles) - vcpkg.json: use "version" instead of deprecated "version-string" - package.json: replace $(find) in lint-cpp with explicit file list, remove unused bare-stream/bare-tty deps, add bare-fs to production deps - CHANGELOG.md: add date per Keep a Changelog format JS fixes: - Move fs.existsSync model check from constructor to _load(), matching TranscriptionWhispercpp lifecycle pattern - Remove dead PAUSED/STOPPED state enum values from bci.js - Add explicit event name matching alongside heuristic fallback in _addonOutputCallback (matches peer whisper.js pattern with BCI stat keys) - Add miscConfig.caption_enabled boolean type validation in configChecker - Extract duplicated flattenSegments into shared lib/util.js - Fix index.d.ts import from fragile internal path to stable @qvac/infer-base C++ fixes: - Guard whisper_log_set with std::once_flag to prevent clobbering log handlers from coexisting whisper-based addons in the same process - Replace std::runtime_error with structured StatusError/bci_error::makeStatus in BCIModel::load() and loadEmbedderIfNeeded() for proper JS error mapping - Use std::move in process(const std::any&) to avoid copying multi-MB neural signal buffers on every inference call Made-with: Cursor

- Add qvac-lint-cpp configure_file block to CMakeLists.txt (copies .clang-format, .clang-tidy, .valgrind.supp from vcpkg into build tree, matching qvac-lib-infer-whispercpp pattern) - Extend lint-cpp script to cover all .hpp header files - Match peer index.d.ts QvacResponse import path (deep import from @qvac/infer-base/src/QvacResponse) - Replace brittle string-matching in _isConfigurationError with structured error detection (TypeError, ERR_ASSERTION code checks) - Remove stale configChecker comments about unimplemented BCI params (smooth_kernel_std, smooth_kernel_size, sample_rate) - Remove unused error codes: FAILED_TO_GET_STATUS, FAILED_TO_RESET, FAILED_TO_PAUSE and their addCodes registrations - Remove unused K_SAMPLES_PER_SECOND constant from BCIModel.cpp - Remove unused <span> include from AddonJs.hpp - Add qvac-lib-inference-addon-cpp to NOTICE C++ dependencies - Add cpp-test-results.xml to .gitignore Made-with: Cursor

The BCI patches (variable conv1 kernel, windowed attention) are now merged into tetherto/qvac-ext-lib-whisper.cpp master and tagged as v1.8.4.2. The local overlay that pinned a specific fork commit is no longer needed. - Delete vcpkg-overlays/whisper-cpp/ (portfile.cmake + vcpkg.json) - Remove VCPKG_OVERLAY_PORTS from CMakeLists.txt - Bump whisper-cpp override from 1.8.4 to 1.8.4.2 - Point vcpkg-configuration.json at personal fork registry (sharmaraju352/qvac-registry-vcpkg) temporarily until tetherto/qvac-registry-vcpkg#125 merges, then swap back - Update README whisper.cpp patches section Verified: clean build from scratch + 18/18 C++ tests + 3/3 integration tests (10/10 asserts, 6.0% avg WER) + batch example all pass. Made-with: Cursor

Registry PR tetherto/qvac-registry-vcpkg#125 has been merged. Swap vcpkg-configuration.json from the personal fork back to the upstream tetherto/qvac-registry-vcpkg and update the baseline to the merge commit. Verified: clean build from scratch + all tests pass on both bci-whispercpp (18/18 C++, 3/3 integration, 6.0% WER) and transcription-whispercpp (106/106 C++, 28/28 unit, 10/10 integration, all extended suites). Made-with: Cursor

- Reset is_warmed_up_ in BCIModel::unload() so re-load triggers warmup - Add FailedToLoadModel and EmbedderWeightsNotFound error codes to BCIErrors.hpp; use them instead of InvalidNeuralSignal for context init failure (BCIModel.cpp:116) and missing embedder (BCIModel.cpp:90) - Wrap addon.activate() in try-catch in index.js _load(), throwing FAILED_TO_ACTIVATE with structured error on failure - Make all JS error codes sequential (26001-26013, no gaps) Made-with: Cursor

…s review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor

#1583) * QVAC-17057 feat: add bci-whispercpp package for BCI neural signal transcription Add a new @qvac/bci-whispercpp addon that transcribes brain-computer interface neural signals into text using a modified whisper.cpp backend. This POC includes: - C++ native addon with BCI model inference (NeuralProcessor, BCIModel, BCIConfig) built on the qvac addon-cpp framework - CMake + vcpkg build system with whisper-cpp overlay ports carrying BCI-specific patches (variable conv1 kernel, windowed attention) - JavaScript API: BCIWhispercpp class with batch transcribeFile/transcribe - Integration tests for load/destroy and batch transcription - Example script and model conversion tooling - WER utility for accuracy measurement Streaming transcription will be added in a follow-up PR (QVAC-17062). Made-with: Cursor * fix[api](bci): address review feedback, refactor to infer-base pattern, fix Linux linkage - Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue from @qvac/infer-base instead of manual promise plumbing, matching the TranscriptionWhispercpp / LlmLlamacpp addon pattern - Constructor now takes { files: { model }, logger, opts } (was { modelPath }) - transcribe/transcribeFile return QvacResponse - Add unload(), getState(), exclusiveRunQueue-serialized destroy() - Add @qvac/infer-base dependency Address all review feedback from Gustavo (PR #1583): - Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms) - Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED - Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp - Add bounds validation for all BCIConfig numeric params - Throw on unknown config keys (was silently ignored) - Consume gpu_device in context params - Collect whisper timings in runtimeStats() - Trim unused BCIErrors enum values, map codes to distinct names - Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js - Fix _activeJobId race: set after native acceptance - Remove unimplemented bciConfig params from JS whitelist + index.d.ts - Promote hardcoded kernel-trim threshold to named constant - Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs) - Rename bci-addon.test.js → addon.test.js - Replace t.skip() with proper assertions - Fix day_idx handling in tests/examples (group by day, pass to config) - Generate comprehensive NOTICE file - Update vcpkg overlay to v1.8.4 description Fix Linux C++ test linkage: - Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++ - Add linux-clang toolchain (clang-19) - Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds Made-with: Cursor * perf(bci): bump whisper-cpp overlay to include mask caching and per-layer flash attn Update whisper-cpp overlay to 5645ad60 which includes: - Cached window_mask recompute for exp_n_audio_ctx overrides - Per-layer flash attention (upper encoder layers use FA even with BCI) - std::abs instead of C abs in mask computation Made-with: Cursor * chore(bci): bump whisper-cpp overlay to include jpgaribotti review fixes Update overlay to tetherto/qvac-ext-lib-whisper.cpp@3e91e3a4 which addresses jpgaribotti's review on PR #10: 1. Extract compute_window_mask() helper to eliminate duplicated O(n_ctx^2) mask fill logic 2. Guard encode-time mask block with hparams.is_bci 3. Add is_bci to graph builder window_mask guard 4. Validate BCI hparams (conv1_kernel > 0, window_size >= 0) 5. Document n_mels > 256 threshold convention Bump port-version to 3. Made-with: Cursor * fix(bci): add test fixture download to download-models.sh Address Gustavo's review feedback: test fixtures (neural_sample_*.bin) are gitignored but the PR had no way for developers to obtain them. Rewrite download-models.sh to fetch both models and test fixtures from the bci-test-assets-v0.1.0 GitHub release. Supports --models, --fixtures, or both (default). Made-with: Cursor * fix(bci): address review findings — version mismatch, test indexing, cleanup - Bump whisper-cpp override in vcpkg.json from 1.7.5.1 to 1.8.4 to match the overlay port version - Move gtest to a vcpkg "tests" feature so it is only pulled when BUILD_TESTING=ON - Fix PaddedFramesAreZero test: use mel-major indexing (data[bin * n_frames + frame]) matching the actual processToMel layout - Remove four unused overlay patch files (0001–0004) now that portfile.cmake fetches from the tetherto fork with patches baked in - Add TODO comment in download-models.sh noting the temporary personal fork for release assets Made-with: Cursor * fix(bci): address review findings — race guard, cross-platform path, docs accuracy - Wrap transcribe() in exclusiveRunQueue to prevent race between inference and unload/destroy - Use find_last_of("/\\") in loadEmbedderIfNeeded for Windows compat - Add empty-buffer guard in bci.js append() before end-of-job - Update download-models.sh to use tetherto/qvac release repo - Add transformers to NOTICE and README model conversion prerequisites - Fix README WER table to match actual live test results (6.0% avg) - Fix BCI_V184_COMPAT.md stale test filename and overlay ref - Remove unused bci_wer_vs_expected field from manifest.json - Update whisper.cpp patches section to reflect fork-based overlay Made-with: Cursor * fix(bci): harden lifecycle, type safety, and C++ code quality - Fix unload/destroy race: call destroyInstance() before _job.fail() so the native side stops before the JS job is failed, and remove redundant cancel() call (destroyInstance already cancels internally) - Wrap BCIInterface construction in try/catch so a native init failure sets addon=null and throws a structured QvacErrorAddonBCI - Change JSAdapter loadContextParams/loadMiscParams/loadBCIParams to return void (callers already mutate via reference, return was dead) - Add dayIdx bounds-check warning in BCIModel::process when the value falls outside [0, numDays-1] before silent clamping - Promote hardcoded gaussian smoothing params (std=2.0, kernel=100) to named constants K_SMOOTH_KERNEL_STD / K_SMOOTH_KERNEL_SIZE - Add NeuralProcessor::getNumDays() accessor for the bounds check - Remove [key: string]: unknown escape hatch from WhisperConfig in index.d.ts; enumerate all valid keys explicitly - Fix test:cpp:run script to use direct path instead of cd && chain Made-with: Cursor * chore(bci): point whisper-cpp overlay to merged master (2b1e04f) qvac-ext-lib-whisper.cpp PR #10 has been merged. Update the overlay to reference the merge commit on master instead of the feature branch commit, so the overlay remains valid if the branch is deleted. Bump port-version to 4. Made-with: Cursor * fix(bci): serialize inference lifetime and export low-level subpaths Address ogad-tether review feedback on PR #1583: 1. Inference queue: transcribe() now holds its slot until the response settles via _enqueueInference(), matching the pattern from TranscriptionWhispercpp._enqueueExclusiveRunResponse(). Previously the exclusiveRunQueue released the slot as soon as runJob() was accepted, allowing a second concurrent transcribe() to race in and either clobber the first response or get rejected by the native side. 2. Exports map: add ./bci, ./bci.js, and ./binding subpath exports so the low-level BCIInterface API documented in the README is accessible after publish. The exports map previously only exposed ./binding.js, blocking require('@qvac/bci-whispercpp/bci'). Made-with: Cursor * refactor[bc](bci): rename to qvac-lib-infer-bci-whispercpp and address review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor * fix[api](bci): restore bci-whispercpp package path and harden runtime validation Move the addon package back to packages/bci-whispercpp, remove unneeded overlay/docs files requested in review, and tighten JS/C++ lifecycle/config safety checks to prevent invalid-state and malformed-input issues. Made-with: Cursor * fix[api](bci): address code review findings across JS, C++, and build config - Replace cd && chain in test:cpp:run with direct path (CLAUDE.md compliance) - Route whisper_log_set through addon-cpp logger instead of silencing with once_flag, preventing inter-addon log handler clobber when BCI and transcription-whispercpp coexist in the same process - Fix stats heuristic in bci.js _addonOutputCallback to match actual BCIModel::runtimeStats keys (tokensPerSecond/totalWallMs, not the audio-addon keys audioDurationMs/totalSamples) - Drain _inferenceQueueWaiter in unload()/destroy() before calling destroyInstance(), closing the race where destroy could fire while process() is mid-execution on the native thread - Remove auto-load in BCIModel::process — throw immediately if context is null instead of lazy-loading outside the controlled lifecycle - Remove dead set_weights_for_file snake_case stub and unused <span> - Add qvac-lint-cpp to vcpkg.json dependencies (matches all peer addons) - Remove empty qvac-lint-cpp overlay directory (per Gustavo review) - Remove stale bci_wer/bci_transcription from manifest.json - Stop gitignoring package-lock.json (match monorepo convention) - Move computeWER into BCIWhispercpp namespace in index.d.ts - Downgrade @types/node to ^22.15.3, remove bare-fs from devDeps - Fix PR template code blocks from typescript to javascript Made-with: Cursor * fix[api](bci): address review findings — standards alignment, structured errors, lifecycle safety Align bci-whispercpp with monorepo conventions and fix code quality issues found during thorough review of the POC implementation. Build/config: - .gitignore aligned with peer addons (package-lock.json, .npmrc, IDE files, vcpkg cache, generated test bundles) - vcpkg.json: use "version" instead of deprecated "version-string" - package.json: replace $(find) in lint-cpp with explicit file list, remove unused bare-stream/bare-tty deps, add bare-fs to production deps - CHANGELOG.md: add date per Keep a Changelog format JS fixes: - Move fs.existsSync model check from constructor to _load(), matching TranscriptionWhispercpp lifecycle pattern - Remove dead PAUSED/STOPPED state enum values from bci.js - Add explicit event name matching alongside heuristic fallback in _addonOutputCallback (matches peer whisper.js pattern with BCI stat keys) - Add miscConfig.caption_enabled boolean type validation in configChecker - Extract duplicated flattenSegments into shared lib/util.js - Fix index.d.ts import from fragile internal path to stable @qvac/infer-base C++ fixes: - Guard whisper_log_set with std::once_flag to prevent clobbering log handlers from coexisting whisper-based addons in the same process - Replace std::runtime_error with structured StatusError/bci_error::makeStatus in BCIModel::load() and loadEmbedderIfNeeded() for proper JS error mapping - Use std::move in process(const std::any&) to avoid copying multi-MB neural signal buffers on every inference call Made-with: Cursor * fix[api](bci): align with peer addon standards and remove unused code - Add qvac-lint-cpp configure_file block to CMakeLists.txt (copies .clang-format, .clang-tidy, .valgrind.supp from vcpkg into build tree, matching qvac-lib-infer-whispercpp pattern) - Extend lint-cpp script to cover all .hpp header files - Match peer index.d.ts QvacResponse import path (deep import from @qvac/infer-base/src/QvacResponse) - Replace brittle string-matching in _isConfigurationError with structured error detection (TypeError, ERR_ASSERTION code checks) - Remove stale configChecker comments about unimplemented BCI params (smooth_kernel_std, smooth_kernel_size, sample_rate) - Remove unused error codes: FAILED_TO_GET_STATUS, FAILED_TO_RESET, FAILED_TO_PAUSE and their addCodes registrations - Remove unused K_SAMPLES_PER_SECOND constant from BCIModel.cpp - Remove unused <span> include from AddonJs.hpp - Add qvac-lib-inference-addon-cpp to NOTICE C++ dependencies - Add cpp-test-results.xml to .gitignore Made-with: Cursor * chore(bci): remove whisper-cpp overlay, consume v1.8.4.2 from registry The BCI patches (variable conv1 kernel, windowed attention) are now merged into tetherto/qvac-ext-lib-whisper.cpp master and tagged as v1.8.4.2. The local overlay that pinned a specific fork commit is no longer needed. - Delete vcpkg-overlays/whisper-cpp/ (portfile.cmake + vcpkg.json) - Remove VCPKG_OVERLAY_PORTS from CMakeLists.txt - Bump whisper-cpp override from 1.8.4 to 1.8.4.2 - Point vcpkg-configuration.json at personal fork registry (sharmaraju352/qvac-registry-vcpkg) temporarily until tetherto/qvac-registry-vcpkg#125 merges, then swap back - Update README whisper.cpp patches section Verified: clean build from scratch + 18/18 C++ tests + 3/3 integration tests (10/10 asserts, 6.0% avg WER) + batch example all pass. Made-with: Cursor * chore(bci): point vcpkg registry back to tetherto upstream Registry PR tetherto/qvac-registry-vcpkg#125 has been merged. Swap vcpkg-configuration.json from the personal fork back to the upstream tetherto/qvac-registry-vcpkg and update the baseline to the merge commit. Verified: clean build from scratch + all tests pass on both bci-whispercpp (18/18 C++, 3/3 integration, 6.0% WER) and transcription-whispercpp (106/106 C++, 28/28 unit, 10/10 integration, all extended suites). Made-with: Cursor * fix(bci): address Gustavo review — error types, lifecycle, error codes - Reset is_warmed_up_ in BCIModel::unload() so re-load triggers warmup - Add FailedToLoadModel and EmbedderWeightsNotFound error codes to BCIErrors.hpp; use them instead of InvalidNeuralSignal for context init failure (BCIModel.cpp:116) and missing embedder (BCIModel.cpp:90) - Wrap addon.activate() in try-catch in index.js _load(), throwing FAILED_TO_ACTIVATE with structured error on failure - Make all JS error codes sequential (26001-26013, no gaps) Made-with: Cursor * Remove date from changelog --------- Co-authored-by: Raju <raju.sharma> Co-authored-by: Ishan Vohra <ishanvohra2@gmail.com> Co-authored-by: GustavoA1604 <54457676+GustavoA1604@users.noreply.github.com>

sharmaraju352 requested review from a team as code owners April 15, 2026 07:09

sharmaraju352 mentioned this pull request Apr 15, 2026

[BCI] QVAC-17071 chore: switch whisper-cpp overlay to tetherto fork #1584

Closed

3 tasks

github-code-quality Bot found potential problems Apr 15, 2026

View reviewed changes

Comment thread packages/bci-whispercpp/index.js Fixed

sharmaraju352 mentioned this pull request Apr 15, 2026

[BCI] QVAC-17072 chore: prepare for registry-based whisper-cpp consumption #1585

Closed

3 tasks

sharmaraju352 added verify tier1 labels Apr 15, 2026

sharmaraju352 changed the title ~~QVAC-17057 feat[notask]: add bci-whispercpp POC with batch transcription~~ QVAC-17057 feat[BCI]: add bci-whispercpp POC with batch transcription Apr 15, 2026

sharmaraju352 mentioned this pull request Apr 15, 2026

[BCI] QVAC-17071 feat: add BCI patches to whisper-cpp port tetherto/qvac-registry-vcpkg#122

Closed

3 tasks

sharmaraju352 changed the title ~~QVAC-17057 feat[BCI]: add bci-whispercpp POC with batch transcription~~ [BCI] QVAC-17057 feat: add bci-whispercpp POC with batch transcription Apr 15, 2026

GustavoA1604 requested changes Apr 15, 2026

View reviewed changes

Raju added 3 commits April 20, 2026 16:25

sharmaraju352 force-pushed the feat/bci-whispercpp-1-poc branch from d2d08ed to 0eec4b4 Compare April 20, 2026 11:05

Raju and others added 7 commits April 20, 2026 16:46

Merge branch 'main' into feat/bci-whispercpp-1-poc

b27b2e7

Merge branch 'main' into feat/bci-whispercpp-1-poc

ab28489

ogad-tether reviewed Apr 20, 2026

View reviewed changes

Comment thread packages/bci-whispercpp/index.js Outdated

ogad-tether requested changes Apr 20, 2026

View reviewed changes

sharmaraju352 and others added 3 commits April 21, 2026 11:21

Merge branch 'main' into feat/bci-whispercpp-1-poc

3030209

GustavoA1604 requested changes Apr 21, 2026

View reviewed changes

Comment thread packages/qvac-lib-infer-bci-whispercpp/vcpkg-overlays/qvac-lint-cpp/vcpkg.json Outdated

Comment thread packages/bci-whispercpp/vcpkg-overlays/whisper-cpp/portfile.cmake Outdated

Comment thread packages/qvac-lib-infer-bci-whispercpp/docs/BCI_V184_COMPAT.md Outdated

GustavoA1604 requested changes Apr 21, 2026

View reviewed changes

Raju added 4 commits April 21, 2026 14:09

sharmaraju352 mentioned this pull request Apr 21, 2026

chore: bump whisper-cpp port to 1.8.4.2 (BCI patches) tetherto/qvac-registry-vcpkg#125

Merged

Raju added 2 commits April 21, 2026 17:18

ogad-tether previously approved these changes Apr 21, 2026

View reviewed changes

ishanvohra2 previously approved these changes Apr 21, 2026

View reviewed changes

GustavoA1604 requested changes Apr 21, 2026

View reviewed changes

sharmaraju352 dismissed stale reviews from ishanvohra2 and ogad-tether via 8513287 April 21, 2026 13:32

Merge branch 'main' into feat/bci-whispercpp-1-poc

a068153

GustavoA1604 requested changes Apr 22, 2026

View reviewed changes

Comment thread packages/bci-whispercpp/CHANGELOG.md Outdated

ishanvohra2 and others added 2 commits April 22, 2026 19:14

Remove date from changelog

26ef7b7

Merge branch 'main' into feat/bci-whispercpp-1-poc

2ec1597

GustavoA1604 approved these changes Apr 22, 2026

View reviewed changes

ishanvohra2 approved these changes Apr 22, 2026

View reviewed changes

GustavoA1604 merged commit dadaf7d into main Apr 22, 2026
10 checks passed

GustavoA1604 deleted the feat/bci-whispercpp-1-poc branch April 22, 2026 13:49

Conversation

sharmaraju352 commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Results

Test plan

Uh oh!

Uh oh!

github-actions Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tier-based Approval Status

Uh oh!

GustavoA1604 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ogad-tether left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GustavoA1604 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sharmaraju352 commented Apr 15, 2026 •

edited

Loading

github-actions Bot commented Apr 15, 2026 •

edited

Loading