tts-cpp: Add dynamic backend selection for Android#29
Merged
Conversation
This was referenced May 20, 2026
gianni-cor
pushed a commit
that referenced
this pull request
May 28, 2026
tts-cpp: Add dynamic backend selection for Android
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
.sofiles (libspeech-ggml-vulkan.so,libspeech-ggml-opencl.so, per-archlibspeech-ggml-cpu-android_armv*_*.so) withGGML_BACKEND_DL=ON. The old tts-cpp init path calledggml_backend_{cuda,metal,vk,opencl,cpu}_init()directly — those symbols live in dlopen'd libraries and are not linkable fromlibtts-cpp.parakeet-cppandllm-llamacppalready use registry-based backend discovery plus an Adreno-tier GPU policy (OpenCL on 7xx/8xx, Vulkan elsewhere, 6xx OpenCL skipped). tts-cpp still used compile-time#ifdef GGML_USE_<X>cascades, so Android prebuilds could silently compile out GPU paths or fail at link/runtime.backends_diror an OpenCL program-binary cache directory before the first Engine init — both are process-singleton state in ggml.How does it solve it?
Registry-only backend selection (
src/backend_selection.{h,cpp})ensure_backends_loaded()— one-timeggml_backend_load_all()orggml_backend_load_all_from_path(backends_dir)(idempotent per process).init_gpu_backend()— walksggml_backend_dev_*with the same tier policy as parakeet-cpp:Adreno X<n>/ Snapdragon X Elite) → prefer OpenCLTTS_CPP_ALLOW_ADRENO_6XX=1.init_cpu_backend()/init_blas_backend()— registry helpers for CPU and BLAS accel devices.set_backends_directory()/set_opencl_cache_dir()— first-Engine-wins overrides with one-shot stderr warnings on conflict (mirrors parakeet-cpp).backend_util.hbackend_is_cpu/backend_is_metal/backend_set_n_threads— route throughggml_backend_get_device+ggml_backend_reg_get_proc_addressso they work underGGML_BACKEND_DL=ON(replaces directggml_backend_is_cpu/ggml_backend_cpu_set_n_threadscalls that fail to link on Android).Init site refactor
main.cpp,chatterbox_tts.cpp(s3gen_init_backend),supertonic_gguf.cpp, and related paths withinit_gpu_backend+init_cpu_backend.voice_encoder.cpp,campplus_forward.inc,mel_extract_stft.cpp.Public API (
EngineOptions)backends_dir— forwarded toset_backends_directory()before first backend init (Chatterbox + Supertonic engines).opencl_cache_dir— sets$GGML_OPENCL_CACHE_DIRon__ANDROID__before first init (reduces coldclBuildProgramlatency).CMake (
tts-cpp/CMakeLists.txt)GGML_BACKEND_DL=ON,GGML_CPU_ALL_VARIANTS=ON,GGML_CPU_REPACK=ON,GGML_VULKAN=ON,GGML_OPENCL=ON, coopmat/coopmat2 disabled — aligned with parakeet-cpp and llamacpp Android prebuilds.tts-cpp-backend-defs— kept as an empty INTERFACE target for export compatibility;GGML_USE_*compile defines removed (they falsely implied static backend linkage on Android).test-metal-ops—GGML_USE_METALdefined locally on that target only (it exercises Metal directly, not the registry path).Behavior changes (non-Android)
Desktop / dev builds no longer pick GPUs via compile-order cascade (CUDA → Metal → Vulkan → OpenCL). Selection now follows the Adreno-aware registry policy; on multi-GPU machines the chosen device follows registry enumeration order within each bucket, not
#ifdeflink order. This matches parakeet-cpp and llamacpp.Breaking changes
None for API consumers that only pass existing
EngineOptionsfields. New optional fields (backends_dir,opencl_cache_dir) are additive.Android prebuilds will emit additional backend
.soartifacts (Vulkan, OpenCL, per-arch CPU) that must be staged next to the host module — same layout as parakeet / llamacpp.