feat(tts-ggml): Android dynamic ggml backends#2168
Merged
Conversation
olyasir
approved these changes
May 21, 2026
Contributor
Tier-based Approval Status |
pratiknarola-t
approved these changes
May 21, 2026
Contributor
Author
|
/review |
Proletter
pushed a commit
that referenced
this pull request
May 24, 2026
* Add dynamic backend loading for android and model download in integration tests * Remove gguf bundling from mobile integration test * Add missing registry-client dependency * Remove non-working GPUs * Fix failing test * Point to tetherto repo * Remove redundant comments * Update readme
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
@qvac/tts-ggml: Android builds needlibqvac-speech-ggml-{vulkan,opencl,cpu-android_armv*_*.so}next to the.baremodule soggml_backend_load_all_from_path()can discover backends at runtime. The addon had noBACKENDS_SUBDIRstaging, nobackendsDir/openclCacheDirJS surface, and no forwarding intotts_cpp::EngineOptions— unlike@qvac/transcription-parakeetand@qvac/llm-llamacpp.EngineOptions::backends_dir, Adreno tier policy,init_gpu_backend()). This package must pintts-cpp >= 2026-05-20and depend on qvac-registry-vcpkg#159 (ggml-speech#4+tts-cppport bump).useGPU: truedefault OOMs on Android: GPU backends mirror large f16 S3Gen weights (~1 GB) on top of the mmap'd CPU copy; 8 GB devices hitlmkdSIGKILL. Default should be CPU with explicit opt-in on capable hosts.How does it solve it?
CMake / prebuilds (
packages/tts-ggml/CMakeLists.txt)find_package(ggml)forGGML_AVAILABLE_BACKENDS+ loose.soglob from vcpkglib/.bare_target+bare_module_target→BACKENDS_SUBDIR(android-arm64/qvac__tts-ggml, …).add_bare_module(... EXPORTS INSTALL TARGET ggml::<backend>)+install(FILES libqvac-speech-ggml-*.so)for Vulkan/OpenCL MODULE backends not exposed as IMPORTED targets.-Wl,-z,max-page-size=16384) — same as parakeet (Pixel 9 class devices).force_loadfor@available→__isPlatformVersionAtLeast(iOS Metal stability on unload/reload).JS + C++ wiring
backendsDir/openclCacheDiron constructor options andTTSGgmlRuntimeConfig; defaultbackendsDir→path.join(__dirname, 'prebuilds').ChatterboxModel/SupertonicModelcomposebackendsDir / BACKENDS_SUBDIRintoopts.backends_dir; forwardopencl_cache_dir.JSAdapterreads both fields for Chatterbox and Supertonic configs.GPU policy
useGPUdefaults tofalsefor Chatterbox (wastruein 0.1.1). Opt in withconfig: { useGPU: true }on Metal / Vulkan / OpenCL hosts.#ifdef __ANDROID__inloadLocked(): forcesn_gpu_layers = 0and logs a warning if the host requested GPU — Vulkan (Mali) and OpenCL (Adreno) paths for Chatterbox/Supertonic graphs are not validated yet. Dynamic backend.sostaging still matters on Android for per-arch CPU dlopen even while GPU stays off.Tests & CI
@qvac/registry-clientdevDependency +downloadModel.jsregistry fetch (q4_0 Chatterbox T3, f16 S3Gen, Supertonic q4_0) with min/max size bands to reject stale caches.resolveRefWavPath()— mobileglobal.assetPaths/Library/Caches/jfk.wavbefore in-bundle path (fixes iOSModelFileNotFound).Docs & version
useGPUtable,backendsDir/openclCacheDirknobs (mirrors parakeet).0.1.2, package.json0.1.2.tts-cpp >= 2026-05-20.Breaking changes
useGPUdefaulttrue→falseconfig: { useGPU: true }where you previously relied on the implicit GPU default (macOS Metal, CUDA desktop, etc.).Supertonic remains CPU-only at construction time when
useGPU: trueis passed.How was it tested
Manual CI run here