[BCI] QVAC-17062 feat: add streaming transcription and integration tests#1586
Closed
sharmaraju352 wants to merge 15 commits into
Closed
[BCI] QVAC-17062 feat: add streaming transcription and integration tests#1586sharmaraju352 wants to merge 15 commits into
sharmaraju352 wants to merge 15 commits into
Conversation
This was referenced Apr 15, 2026
sharmaraju352
force-pushed
the
feat/bci-whispercpp-3-registry
branch
from
e1af991 to
18c2db2
Compare
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
993ddd9 to
1c7a316
Compare
sharmaraju352
changed the title
This was referenced Apr 15, 2026
sharmaraju352
changed the title
added 3 commits
April 20, 2026 16:25
…nscription Add a new @qvac/bci-whispercpp addon that transcribes brain-computer interface neural signals into text using a modified whisper.cpp backend. This POC includes: - C++ native addon with BCI model inference (NeuralProcessor, BCIModel, BCIConfig) built on the qvac addon-cpp framework - CMake + vcpkg build system with whisper-cpp overlay ports carrying BCI-specific patches (variable conv1 kernel, windowed attention) - JavaScript API: BCIWhispercpp class with batch transcribeFile/transcribe - Integration tests for load/destroy and batch transcription - Example script and model conversion tooling - WER utility for accuracy measurement Streaming transcription will be added in a follow-up PR (QVAC-17062). Made-with: Cursor
…n, fix Linux linkage
- Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue
from @qvac/infer-base instead of manual promise plumbing, matching
the TranscriptionWhispercpp / LlmLlamacpp addon pattern
- Constructor now takes { files: { model }, logger, opts } (was { modelPath })
- transcribe/transcribeFile return QvacResponse
- Add unload(), getState(), exclusiveRunQueue-serialized destroy()
- Add @qvac/infer-base dependency
Address all review feedback from Gustavo (PR #1583):
- Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms)
- Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED
- Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp
- Add bounds validation for all BCIConfig numeric params
- Throw on unknown config keys (was silently ignored)
- Consume gpu_device in context params
- Collect whisper timings in runtimeStats()
- Trim unused BCIErrors enum values, map codes to distinct names
- Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js
- Fix _activeJobId race: set after native acceptance
- Remove unimplemented bciConfig params from JS whitelist + index.d.ts
- Promote hardcoded kernel-trim threshold to named constant
- Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs)
- Rename bci-addon.test.js → addon.test.js
- Replace t.skip() with proper assertions
- Fix day_idx handling in tests/examples (group by day, pass to config)
- Generate comprehensive NOTICE file
- Update vcpkg overlay to v1.8.4 description
Fix Linux C++ test linkage:
- Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++
- Add linux-clang toolchain (clang-19)
- Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds
Made-with: Cursor
…ayer flash attn Update whisper-cpp overlay to 5645ad60 which includes: - Cached window_mask recompute for exp_n_audio_ctx overrides - Per-layer flash attention (upper encoder layers use FA even with BCI) - std::abs instead of C abs in mask computation Made-with: Cursor
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
3db0a11 to
1375a1b
Compare
sharmaraju352
changed the base branch from
feat/bci-whispercpp-3-registry
to
feat/bci-whispercpp-1-poc
Update overlay to tetherto/qvac-ext-lib-whisper.cpp@3e91e3a4 which addresses jpgaribotti's review on PR #10: 1. Extract compute_window_mask() helper to eliminate duplicated O(n_ctx^2) mask fill logic 2. Guard encode-time mask block with hparams.is_bci 3. Add is_bci to graph builder window_mask guard 4. Validate BCI hparams (conv1_kernel > 0, window_size >= 0) 5. Document n_mels > 256 threshold convention Bump port-version to 3. Made-with: Cursor
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
1375a1b to
950b3b5
Compare
Address Gustavo's review feedback: test fixtures (neural_sample_*.bin) are gitignored but the PR had no way for developers to obtain them. Rewrite download-models.sh to fetch both models and test fixtures from the bci-test-assets-v0.1.0 GitHub release. Supports --models, --fixtures, or both (default). Made-with: Cursor
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
950b3b5 to
e9b44a0
Compare
…cleanup - Bump whisper-cpp override in vcpkg.json from 1.7.5.1 to 1.8.4 to match the overlay port version - Move gtest to a vcpkg "tests" feature so it is only pulled when BUILD_TESTING=ON - Fix PaddedFramesAreZero test: use mel-major indexing (data[bin * n_frames + frame]) matching the actual processToMel layout - Remove four unused overlay patch files (0001–0004) now that portfile.cmake fetches from the tetherto fork with patches baked in - Add TODO comment in download-models.sh noting the temporary personal fork for release assets Made-with: Cursor
…docs accuracy
- Wrap transcribe() in exclusiveRunQueue to prevent race between
inference and unload/destroy
- Use find_last_of("/\\") in loadEmbedderIfNeeded for Windows compat
- Add empty-buffer guard in bci.js append() before end-of-job
- Update download-models.sh to use tetherto/qvac release repo
- Add transformers to NOTICE and README model conversion prerequisites
- Fix README WER table to match actual live test results (6.0% avg)
- Fix BCI_V184_COMPAT.md stale test filename and overlay ref
- Remove unused bci_wer_vs_expected field from manifest.json
- Update whisper.cpp patches section to reflect fork-based overlay
Made-with: Cursor
- Fix unload/destroy race: call destroyInstance() before _job.fail() so the native side stops before the JS job is failed, and remove redundant cancel() call (destroyInstance already cancels internally) - Wrap BCIInterface construction in try/catch so a native init failure sets addon=null and throws a structured QvacErrorAddonBCI - Change JSAdapter loadContextParams/loadMiscParams/loadBCIParams to return void (callers already mutate via reference, return was dead) - Add dayIdx bounds-check warning in BCIModel::process when the value falls outside [0, numDays-1] before silent clamping - Promote hardcoded gaussian smoothing params (std=2.0, kernel=100) to named constants K_SMOOTH_KERNEL_STD / K_SMOOTH_KERNEL_SIZE - Add NeuralProcessor::getNumDays() accessor for the bounds check - Remove [key: string]: unknown escape hatch from WhisperConfig in index.d.ts; enumerate all valid keys explicitly - Fix test:cpp:run script to use direct path instead of cd && chain Made-with: Cursor
qvac-ext-lib-whisper.cpp PR #10 has been merged. Update the overlay to reference the merge commit on master instead of the feature branch commit, so the overlay remains valid if the branch is deleted. Bump port-version to 4. Made-with: Cursor
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
e9b44a0 to
4145458
Compare
Address ogad-tether review feedback on PR #1583: 1. Inference queue: transcribe() now holds its slot until the response settles via _enqueueInference(), matching the pattern from TranscriptionWhispercpp._enqueueExclusiveRunResponse(). Previously the exclusiveRunQueue released the slot as soon as runJob() was accepted, allowing a second concurrent transcribe() to race in and either clobber the first response or get rejected by the native side. 2. Exports map: add ./bci, ./bci.js, and ./binding subpath exports so the low-level BCIInterface API documented in the README is accessible after publish. The exports map previously only exposed ./binding.js, blocking require('@qvac/bci-whispercpp/bci'). Made-with: Cursor
…n tests Add transcribeStream method to BCIWhispercpp that accepts an async iterable of neural signal chunks, buffers them, and triggers inference on end-of-stream. - transcribeStream() uses _enqueueInference for proper slot serialization - Add _handleSignalStream() for async iteration + append plumbing - Add streaming integration test with chunked neural signal input - Add transcribeStream type declaration to index.d.ts Made-with: Cursor
sharmaraju352
force-pushed
the
feat/bci-whispercpp-4-streaming
branch
from
4145458 to
1d691f8
Compare
Contributor
|
This draft PR is stale because it has been open 21 days and the author has not commented since opening. It is flagged for removal. Remove the stale label or comment on the PR or this will be closed in one day. |
Contributor
|
This draft PR was closed because it has been stalled for 22 days with no author comment since opening. You can reopen this PR later if it is still necessary. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
transcribeStream(signalStream)method toBCIWhispercppthat accepts an async iterable ofUint8Arrayneural signal chunks, buffers them via the low-levelappendAPI, and triggers inference on end-of-streamtranscribeStreamtype signature toindex.d.tsStacked PR
This is PR 4 of 6 in the BCI addon stack (based on PR 3: #1585):
Test plan
npm run test:integrationpasses all 4 tests (load/destroy, batch, streaming, WER report)Made with Cursor