UPSTREAM PR #17579: Add PagedAttention support (experimental, CUDA only) - #352
Open
loci-dev wants to merge 1 commit into
Open
UPSTREAM PR #17579: Add PagedAttention support (experimental, CUDA only)#352loci-dev wants to merge 1 commit into
loci-dev wants to merge 1 commit into
LOCI Review / Performance Review #352
succeeded
Dec 1, 2025 in 28m 22s
Performance varied across binaries, overall acceptable
2 binaries improved · 6 binaries unchanged · 8 binaries stable ~ within threshold · 0 binaries degraded ~ beyond threshold
| Binary | Δ % Response | Δ % Throughput | Performance (based on response time) |
|---|---|---|---|
build.bin.libggml-base.so |
0.3 | 0.13 | stable |
build.bin.libggml-cpu.so |
0.09 | -0.15 | stable |
build.bin.libggml.so |
0 | 0 | unchanged |
build.bin.libllama.so |
0.52 | 2.57 | stable |
build.bin.libmtmd.so |
0 | 0 | unchanged |
build.bin.llama-bench |
0.01 | 0.04 | stable |
build.bin.llama-cvector-generator |
0.27 | -0.38 | stable |
build.bin.llama-gemma3-cli |
0 | 0 | unchanged |
build.bin.llama-gguf-split |
-0.06 | -0.08 | improved |
build.bin.llama-llava-cli |
0 | 0 | unchanged |
build.bin.llama-minicpmv-cli |
0 | 0 | unchanged |
build.bin.llama-quantize |
0.01 | 0.07 | stable |
build.bin.llama-qwen2vl-cli |
0 | 0 | unchanged |
build.bin.llama-run |
-0.37 | -0.1 | improved |
build.bin.llama-tokenize |
0.01 | -0.02 | stable |
build.bin.llama-tts |
0.39 | -0.19 | stable |
Performance threshold: 30%
Default configuration used.
Note: Performance status is evaluated only from Δ% Response. Throughput is displayed for reference.
Explore the complete analysis inside the Version Insights.
Loading