Skip to content

UPSTREAM PR #17044: Add MoE dynamic routing with expert caching - #101

Open
DajanaV wants to merge 1 commit into
mainfrom
upstream-PR17044-branch_jmangold23-feature/moe_dynamic_routing
Open

DajanaV wants to merge 1 commit into
mainfrom
upstream-PR17044-branch_jmangold23-feature/moe_dynamic_routing

Add MoE dynamic routing with expert caching

8978c86
Select commit
Loading
Failed to load commit list.
This check has been archived and is scheduled for deletion. Learn more about checks retention
LOCI Review / Performance Review #101 succeeded Nov 6, 2025 in 27m 2s

Performance varied across binaries, overall acceptable

0 binaries improved · 12 binaries unchanged · 4 binaries stable ~ within threshold · 0 binaries degraded ~ beyond threshold

Binary Δ % Response Δ % Throughput Performance (based on response time)
build.bin.libggml-base.so 0.00 0.00 unchanged
build.bin.libggml-cpu.so 0.00 0.00 unchanged
build.bin.libggml.so 0.00 0.00 unchanged
build.bin.libllama.so 0.10 0.00 stable
build.bin.libmtmd.so 0.00 0.00 unchanged
build.bin.llama-bench 0.00 0.00 unchanged
build.bin.llama-cvector-generator 0.10 0.00 stable
build.bin.llama-gemma3-cli 0.00 0.00 unchanged
build.bin.llama-gguf-split 0.00 0.00 unchanged
build.bin.llama-llava-cli 0.00 0.00 unchanged
build.bin.llama-minicpmv-cli 0.00 0.00 unchanged
build.bin.llama-quantize 0.00 0.00 unchanged
build.bin.llama-qwen2vl-cli 0.00 0.00 unchanged
build.bin.llama-run 0.10 -0.04 stable
build.bin.llama-tokenize 0.00 0.00 unchanged
build.bin.llama-tts 0.10 0.00 stable

Performance threshold: 30%
Default configuration used.
Note: Performance status is evaluated only from Δ% Response. Throughput is displayed for reference.

Access the complete analysis in the LOCI Dashboard.
Open the Pull Request linked to this check-run.