UPSTREAM PR #19230: cuda: skip CUDA_SCALE_LAUNCH_QUEUES scaling on Jetson/Tegra

[loci-dev]

Note

Source pull request: ggml-org/llama.cpp#19230

Summary

On Jetson Orin AGX (SM87, unified memory), the CUDA_SCALE_LAUNCH_QUEUES=4x environment variable introduced in #19042 causes llama-server to deadlock during warmup decode. This affects all MoE models tested (GLM-4.7-Flash, Qwen3-Coder-30B-A3B).

This PR detects Tegra/Jetson systems at runtime via /sys/devices/soc0/family (which returns "Tegra" on all Jetson devices) and skips the launch queue scaling. This preserves the multi-GPU pipeline parallelism optimization on desktop systems while avoiding the deadlock on Jetson unified memory.

Alternative to full revert in #19227.

Environment tested

Component	Version
Device	Jetson AGX Orin 64GB
JetPack / L4T	R36.4.7
Kernel	5.15.148-tegra
NVIDIA Driver	540.4.0
CUDA Toolkit	12.6.2 (V12.6.77)
cuDNN	9.4.0.58
GCC/G++	11.4.0
CMake	4.1.0

Build flags

cmake -B build -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=87 \
  -DGGML_CUDA_FA_ALL_QUANTS=ON \
  -DGGML_CUDA_F16=ON \
  -DCMAKE_BUILD_TYPE=Release

Verification

• Without fix: server hangs indefinitely on warmup decode
• With fix: 20.4 tok/s generation, works perfectly
• Bisect: 9-step bisect confirmed a83c73a18 as the single breaking commit

How the fix works

Before any CUDA API call, reads /sys/devices/soc0/family which is a sysfs node available on all Linux systems. On Tegra/Jetson it returns "Tegra", on desktop/server systems the file either doesn't exist or returns a different value. No CUDA calls needed for detection.

Fixes #19219

[loci-review]

No meaningful performance changes were detected across 115327 analyzed functions in the following binaries: build.bin.libllama.so, build.bin.llama-cvector-generator, build.bin.llama-tts, build.bin.libmtmd.so, build.bin.llama-tokenize, build.bin.llama-quantize, build.bin.llama-qwen2vl-cli, build.bin.llama-gemma3-cli, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli, build.bin.llama-bench, build.bin.libggml-base.so, build.bin.libggml-cpu.so, build.bin.libggml.so.

🔎 Full breakdown: Loci Inspector.
💬 Questions? Tag @loci-dev.

[loci-review]

No meaningful performance changes were detected across 115327 analyzed functions in the following binaries: build.bin.llama-cvector-generator, build.bin.libmtmd.so, build.bin.llama-tts, build.bin.libllama.so, build.bin.llama-bench, build.bin.llama-gguf-split, build.bin.llama-llava-cli, build.bin.llama-minicpmv-cli, build.bin.llama-quantize, build.bin.llama-gemma3-cli, build.bin.libggml-cpu.so, build.bin.libggml.so, build.bin.libggml-base.so, build.bin.llama-tokenize, build.bin.llama-qwen2vl-cli.

🔎 Full breakdown: Loci Inspector.
💬 Questions? Tag @loci-dev.