Skip to content

tts-cpp: Add dynamic backend selection for Android#29

Merged
GustavoA1604 merged 1 commit into
tetherto:masterfrom
GustavoA1604:master
May 20, 2026
Merged

tts-cpp: Add dynamic backend selection for Android#29
GustavoA1604 merged 1 commit into
tetherto:masterfrom
GustavoA1604:master

Conversation

@GustavoA1604

@GustavoA1604 GustavoA1604 commented May 20, 2026

Copy link
Copy Markdown

What problem does this PR solve?

  • Android dynamic loading: Embedded host apps ship ggml backends as separate .so files (libspeech-ggml-vulkan.so, libspeech-ggml-opencl.so, per-arch libspeech-ggml-cpu-android_armv*_*.so) with GGML_BACKEND_DL=ON. The old tts-cpp init path called ggml_backend_{cuda,metal,vk,opencl,cpu}_init() directly — those symbols live in dlopen'd libraries and are not linkable from libtts-cpp.
  • Inconsistent stack behavior: parakeet-cpp and llm-llamacpp already use registry-based backend discovery plus an Adreno-tier GPU policy (OpenCL on 7xx/8xx, Vulkan elsewhere, 6xx OpenCL skipped). tts-cpp still used compile-time #ifdef GGML_USE_<X> cascades, so Android prebuilds could silently compile out GPU paths or fail at link/runtime.
  • Missing host knobs: No way for QVAC addons to pass backends_dir or an OpenCL program-binary cache directory before the first Engine init — both are process-singleton state in ggml.

How does it solve it?

Registry-only backend selection (src/backend_selection.{h,cpp})

  • ensure_backends_loaded() — one-time ggml_backend_load_all() or ggml_backend_load_all_from_path(backends_dir) (idempotent per process).
  • init_gpu_backend() — walks ggml_backend_dev_* with the same tier policy as parakeet-cpp:
    1. Adreno 700+ (incl. synthetic 800 for Adreno X<n> / Snapdragon X Elite) → prefer OpenCL
    2. Other GPUs (Vulkan on non-Adreno Android, Metal, CUDA, Mali, …) → non-OpenCL backends
    3. Fallback → other OpenCL devices (desktop, etc.)
    • Adreno 6xx OpenCL is skipped (known broken kernels) unless TTS_CPP_ALLOW_ADRENO_6XX=1.
  • init_cpu_backend() / init_blas_backend() — registry helpers for CPU and BLAS accel devices.
  • set_backends_directory() / set_opencl_cache_dir() — first-Engine-wins overrides with one-shot stderr warnings on conflict (mirrors parakeet-cpp).

backend_util.h

  • backend_is_cpu / backend_is_metal / backend_set_n_threads — route through ggml_backend_get_device + ggml_backend_reg_get_proc_address so they work under GGML_BACKEND_DL=ON (replaces direct ggml_backend_is_cpu / ggml_backend_cpu_set_n_threads calls that fail to link on Android).

Init site refactor

  • Replaces duplicated ladders in main.cpp, chatterbox_tts.cpp (s3gen_init_backend), supertonic_gguf.cpp, and related paths with init_gpu_backend + init_cpu_backend.
  • Updates auxiliary CPU fallbacks in voice_encoder.cpp, campplus_forward.inc, mel_extract_stft.cpp.

Public API (EngineOptions)

  • backends_dir — forwarded to set_backends_directory() before first backend init (Chatterbox + Supertonic engines).
  • opencl_cache_dir — sets $GGML_OPENCL_CACHE_DIR on __ANDROID__ before first init (reduces cold clBuildProgram latency).

CMake (tts-cpp/CMakeLists.txt)

  • Android defaults (when not already in cache): GGML_BACKEND_DL=ON, GGML_CPU_ALL_VARIANTS=ON, GGML_CPU_REPACK=ON, GGML_VULKAN=ON, GGML_OPENCL=ON, coopmat/coopmat2 disabled — aligned with parakeet-cpp and llamacpp Android prebuilds.
  • tts-cpp-backend-defs — kept as an empty INTERFACE target for export compatibility; GGML_USE_* compile defines removed (they falsely implied static backend linkage on Android).
  • test-metal-opsGGML_USE_METAL defined locally on that target only (it exercises Metal directly, not the registry path).

Behavior changes (non-Android)

Desktop / dev builds no longer pick GPUs via compile-order cascade (CUDA → Metal → Vulkan → OpenCL). Selection now follows the Adreno-aware registry policy; on multi-GPU machines the chosen device follows registry enumeration order within each bucket, not #ifdef link order. This matches parakeet-cpp and llamacpp.

Breaking changes

None for API consumers that only pass existing EngineOptions fields. New optional fields (backends_dir, opencl_cache_dir) are additive.

Android prebuilds will emit additional backend .so artifacts (Vulkan, OpenCL, per-arch CPU) that must be staged next to the host module — same layout as parakeet / llamacpp.

@GustavoA1604 GustavoA1604 requested review from a team as code owners May 20, 2026 20:22
@GustavoA1604 GustavoA1604 changed the title tts-cpp: Add dynamic backend selection for android tts-cpp: Add dynamic backend selection for Android May 20, 2026
@GustavoA1604 GustavoA1604 merged commit 60dc150 into tetherto:master May 20, 2026
57 of 65 checks passed
gianni-cor pushed a commit that referenced this pull request May 28, 2026
tts-cpp: Add dynamic backend selection for Android
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant