Skip to content

UPSTREAM PR #18816: HIP: tune mmq/rocblas switching for RDNA4#908

Open
loci-dev wants to merge 4 commits intomainfrom
upstream-PR18816-branch_jiachengjason-fix/jiachengjason/rocm7.x_regression
Open

UPSTREAM PR #18816: HIP: tune mmq/rocblas switching for RDNA4#908
loci-dev wants to merge 4 commits intomainfrom
upstream-PR18816-branch_jiachengjason-fix/jiachengjason/rocm7.x_regression

Conversation

@loci-dev
Copy link

@loci-dev loci-dev commented Jan 13, 2026

Mirrored from ggml-org/llama.cpp#18816

Following similar approach to ggml-org/llama.cpp#18537 for tuning mmq/rocblas switching for RDNA4

Testing set up:

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" cmake -S . -B build   -DGGML_HIP=ON   -DGGML_CUDA_FORCE_MMQ=OFF   -DGGML_HIP_UMA=OFF   -DGGML_HIP_ROCWMMA_FATTN=ON   -DGPU_TARGETS="gfx1201"   -DGGML_HIP_GRAPHS=OFF   -DLLAMA_CURL=OFF   -DGGML_CUDA_FORCE_CUBLAS=OFF  -DCMAKE_BUILD_TYPE=Release && cmake --build build --config Release -- -j 32

for q in q4_0 q4_1 q5_1 q8_0 q2_k_s q3_k_s q4_k_s q5_k_s q6_k iq1_s iq2_xxs iq2_xs iq2_s iq3_xxs iq3_xs iq3_s iq3_m iq4_nl iq4_xs; do echo $q; HIP_VISIBLE_DEVICES=0 ./build/bin/llama-bench --model /mnt/nas_share/models/gguf/llama-8b/llama-8b${model_name}-${q}.gguf -r 1 -fa 0 -n 0 -p 2048 -ub "1-2048*2" --progress -o sql|sqlite3 llama-bench.sqlite; sleep 10; done

python3 scripts/compare-llama-bench.py -s gpu_info,model_type,n_ubatch -i llama-bench.sqlite -b 96892952 -c fix/jiachengjason/rocm7.x_regression | tee benchout.txt
Performance result for llama-bench
GPU Model Microbatch size Test t/s 9689295 t/s fix/jiachengjason/rocm7.x_regression Speedup
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 1 pp2048 116.65 116.50 1.00
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 2 pp2048 185.36 184.40 0.99
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 4 pp2048 283.15 282.20 1.00
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 8 pp2048 422.11 420.30 1.00
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 16 pp2048 870.34 868.22 1.00
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 32 pp2048 46.48 47.01 1.01
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 64 pp2048 92.75 94.30 1.02
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 128 pp2048 186.10 188.11 1.01
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 256 pp2048 368.71 371.24 1.01
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 512 pp2048 721.78 714.58 0.99
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 1024 pp2048 1324.23 1280.97 0.97
AI PRO R9700 llama 8B IQ1_S - 1.5625 bpw 2048 pp2048 1893.95 2023.37 1.07
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 1 pp2048 91.42 91.29 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 2 pp2048 147.35 147.37 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 4 pp2048 232.21 232.57 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 8 pp2048 393.74 393.63 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 16 pp2048 522.11 521.15 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 32 pp2048 46.80 46.55 0.99
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 64 pp2048 93.90 93.71 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 128 pp2048 187.08 187.00 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 256 pp2048 364.09 368.31 1.01
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 512 pp2048 714.88 714.25 1.00
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 1024 pp2048 1311.81 1298.76 0.99
AI PRO R9700 llama 8B IQ2_S - 2.5 bpw 2048 pp2048 1725.67 2017.03 1.17
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 1 pp2048 94.55 93.95 0.99
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 2 pp2048 150.40 149.59 0.99
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 4 pp2048 233.35 232.76 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 8 pp2048 394.11 392.86 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 16 pp2048 504.50 505.21 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 32 pp2048 46.71 46.81 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 64 pp2048 93.44 93.35 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 128 pp2048 186.18 186.93 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 256 pp2048 368.95 366.11 0.99
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 512 pp2048 720.57 710.28 0.99
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 1024 pp2048 1312.73 1316.40 1.00
AI PRO R9700 llama 8B IQ2_XS - 2.3125 bpw 2048 pp2048 1693.83 2045.13 1.21
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 1 pp2048 81.06 80.39 0.99
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 2 pp2048 136.39 135.89 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 4 pp2048 228.45 227.82 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 8 pp2048 347.37 346.65 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 16 pp2048 733.56 733.28 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 32 pp2048 46.56 46.39 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 64 pp2048 93.17 93.24 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 128 pp2048 186.34 186.10 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 256 pp2048 368.54 366.71 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 512 pp2048 713.87 713.57 1.00
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 1024 pp2048 1319.65 1308.83 0.99
AI PRO R9700 llama 8B IQ2_XXS - 2.0625 bpw 2048 pp2048 1923.32 2031.18 1.06
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 1 pp2048 76.14 75.56 0.99
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 2 pp2048 132.14 131.65 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 4 pp2048 224.40 224.15 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 8 pp2048 347.60 347.22 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 16 pp2048 701.41 700.71 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 32 pp2048 46.24 46.64 1.01
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 64 pp2048 92.74 93.54 1.01
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 128 pp2048 184.58 185.75 1.01
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 256 pp2048 363.12 364.21 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 512 pp2048 704.46 704.03 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 1024 pp2048 1298.12 1296.16 1.00
AI PRO R9700 llama 8B IQ3_S - 3.4375 bpw 2048 pp2048 1895.07 2005.77 1.06
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 1 pp2048 76.49 76.20 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 2 pp2048 132.45 132.40 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 4 pp2048 220.96 220.99 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 8 pp2048 335.52 335.50 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 16 pp2048 716.89 716.88 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 32 pp2048 46.89 46.59 0.99
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 64 pp2048 94.11 93.48 0.99
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 128 pp2048 187.99 187.78 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 256 pp2048 369.27 368.33 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 512 pp2048 714.30 713.09 1.00
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 1024 pp2048 1330.33 1284.09 0.97
AI PRO R9700 llama 8B IQ3_S mix - 3.66 bpw 2048 pp2048 1897.23 2005.97 1.06
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 1 pp2048 82.80 82.52 1.00
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 2 pp2048 138.71 138.31 1.00
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 4 pp2048 227.96 227.63 1.00
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 8 pp2048 355.77 354.59 1.00
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 16 pp2048 747.16 745.87 1.00
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 32 pp2048 46.83 45.19 0.96
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 64 pp2048 93.83 91.19 0.97
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 128 pp2048 187.14 181.89 0.97
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 256 pp2048 367.46 360.60 0.98
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 512 pp2048 711.98 689.69 0.97
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 1024 pp2048 1311.59 1273.18 0.97
AI PRO R9700 llama 8B IQ3_XS - 3.3 bpw 2048 pp2048 1958.27 1982.27 1.01
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 1 pp2048 89.64 89.18 0.99
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 2 pp2048 143.99 143.59 1.00
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 4 pp2048 229.68 229.46 1.00
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 8 pp2048 358.04 358.44 1.00
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 16 pp2048 697.98 698.62 1.00
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 32 pp2048 46.01 46.77 1.02
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 64 pp2048 93.16 93.66 1.01
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 128 pp2048 186.75 186.44 1.00
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 256 pp2048 363.16 366.73 1.01
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 512 pp2048 700.07 713.70 1.02
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 1024 pp2048 1314.70 1303.03 0.99
AI PRO R9700 llama 8B IQ3_XXS - 3.0625 bpw 2048 pp2048 1935.46 2010.21 1.04
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 1 pp2048 89.46 89.21 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 2 pp2048 154.44 154.90 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 4 pp2048 271.10 270.84 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 8 pp2048 409.31 408.60 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 16 pp2048 927.92 925.17 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 32 pp2048 46.81 46.99 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 64 pp2048 93.94 93.71 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 128 pp2048 187.22 186.69 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 256 pp2048 368.17 370.66 1.01
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 512 pp2048 717.49 717.24 1.00
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 1024 pp2048 1332.59 1318.79 0.99
AI PRO R9700 llama 8B IQ4_NL - 4.5 bpw 2048 pp2048 2081.83 2032.09 0.98
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 1 pp2048 92.86 92.58 1.00
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 2 pp2048 158.79 158.52 1.00
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 4 pp2048 281.73 281.27 1.00
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 8 pp2048 446.83 446.40 1.00
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 16 pp2048 970.48 968.82 1.00
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 32 pp2048 46.93 45.93 0.98
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 64 pp2048 93.97 91.82 0.98
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 128 pp2048 187.83 184.35 0.98
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 256 pp2048 368.45 362.06 0.98
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 512 pp2048 720.72 701.18 0.97
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 1024 pp2048 1333.53 1315.88 0.99
AI PRO R9700 llama 8B IQ4_XS - 4.25 bpw 2048 pp2048 2087.39 2017.86 0.97
AI PRO R9700 llama 8B Q2_K_S 1 pp2048 107.84 107.95 1.00
AI PRO R9700 llama 8B Q2_K_S 2 pp2048 153.49 153.50 1.00
AI PRO R9700 llama 8B Q2_K_S 4 pp2048 204.03 204.85 1.00
AI PRO R9700 llama 8B Q2_K_S 8 pp2048 261.28 262.60 1.01
AI PRO R9700 llama 8B Q2_K_S 16 pp2048 519.99 519.97 1.00
AI PRO R9700 llama 8B Q2_K_S 32 pp2048 46.60 46.60 1.00
AI PRO R9700 llama 8B Q2_K_S 64 pp2048 93.29 93.55 1.00
AI PRO R9700 llama 8B Q2_K_S 128 pp2048 186.06 185.58 1.00
AI PRO R9700 llama 8B Q2_K_S 256 pp2048 369.62 366.51 0.99
AI PRO R9700 llama 8B Q2_K_S 512 pp2048 708.37 716.96 1.01
AI PRO R9700 llama 8B Q2_K_S 1024 pp2048 1012.71 1302.57 1.29
AI PRO R9700 llama 8B Q2_K_S 2048 pp2048 1272.54 2028.48 1.59
AI PRO R9700 llama 8B Q3_K_S 1 pp2048 82.56 81.97 0.99
AI PRO R9700 llama 8B Q3_K_S 2 pp2048 132.91 132.66 1.00
AI PRO R9700 llama 8B Q3_K_S 4 pp2048 192.73 192.25 1.00
AI PRO R9700 llama 8B Q3_K_S 8 pp2048 265.34 264.94 1.00
AI PRO R9700 llama 8B Q3_K_S 16 pp2048 733.96 734.08 1.00
AI PRO R9700 llama 8B Q3_K_S 32 pp2048 46.56 46.54 1.00
AI PRO R9700 llama 8B Q3_K_S 64 pp2048 92.69 93.19 1.01
AI PRO R9700 llama 8B Q3_K_S 128 pp2048 185.55 186.21 1.00
AI PRO R9700 llama 8B Q3_K_S 256 pp2048 363.70 366.18 1.01
AI PRO R9700 llama 8B Q3_K_S 512 pp2048 714.34 711.59 1.00
AI PRO R9700 llama 8B Q3_K_S 1024 pp2048 1342.24 1330.40 0.99
AI PRO R9700 llama 8B Q3_K_S 2048 pp2048 1829.42 2043.11 1.12
AI PRO R9700 llama 8B Q4_0 1 pp2048 90.26 90.04 1.00
AI PRO R9700 llama 8B Q4_0 2 pp2048 155.38 155.29 1.00
AI PRO R9700 llama 8B Q4_0 4 pp2048 277.20 276.81 1.00
AI PRO R9700 llama 8B Q4_0 8 pp2048 424.85 423.88 1.00
AI PRO R9700 llama 8B Q4_0 16 pp2048 902.01 898.74 1.00
AI PRO R9700 llama 8B Q4_0 32 pp2048 46.43 47.11 1.01
AI PRO R9700 llama 8B Q4_0 64 pp2048 93.46 94.40 1.01
AI PRO R9700 llama 8B Q4_0 128 pp2048 184.74 188.02 1.02
AI PRO R9700 llama 8B Q4_0 256 pp2048 366.57 369.24 1.01
AI PRO R9700 llama 8B Q4_0 512 pp2048 707.99 719.00 1.02
AI PRO R9700 llama 8B Q4_0 1024 pp2048 1332.03 1350.90 1.01
AI PRO R9700 llama 8B Q4_0 2048 pp2048 2071.24 2077.98 1.00
AI PRO R9700 llama 8B Q4_1 1 pp2048 85.28 85.10 1.00
AI PRO R9700 llama 8B Q4_1 2 pp2048 149.30 148.87 1.00
AI PRO R9700 llama 8B Q4_1 4 pp2048 265.48 265.04 1.00
AI PRO R9700 llama 8B Q4_1 8 pp2048 449.57 448.75 1.00
AI PRO R9700 llama 8B Q4_1 16 pp2048 917.92 916.55 1.00
AI PRO R9700 llama 8B Q4_1 32 pp2048 46.77 46.86 1.00
AI PRO R9700 llama 8B Q4_1 64 pp2048 93.72 93.65 1.00
AI PRO R9700 llama 8B Q4_1 128 pp2048 187.93 186.49 0.99
AI PRO R9700 llama 8B Q4_1 256 pp2048 368.83 366.56 0.99
AI PRO R9700 llama 8B Q4_1 512 pp2048 713.95 712.48 1.00
AI PRO R9700 llama 8B Q4_1 1024 pp2048 1326.69 1315.70 0.99
AI PRO R9700 llama 8B Q4_1 2048 pp2048 1887.01 1878.97 1.00
AI PRO R9700 llama 8B Q4_K_S 1 pp2048 87.43 87.25 1.00
AI PRO R9700 llama 8B Q4_K_S 2 pp2048 144.02 143.54 1.00
AI PRO R9700 llama 8B Q4_K_S 4 pp2048 201.26 201.96 1.00
AI PRO R9700 llama 8B Q4_K_S 8 pp2048 268.37 269.04 1.00
AI PRO R9700 llama 8B Q4_K_S 16 pp2048 875.16 874.46 1.00
AI PRO R9700 llama 8B Q4_K_S 32 pp2048 46.56 46.96 1.01
AI PRO R9700 llama 8B Q4_K_S 64 pp2048 93.52 93.97 1.00
AI PRO R9700 llama 8B Q4_K_S 128 pp2048 187.49 187.93 1.00
AI PRO R9700 llama 8B Q4_K_S 256 pp2048 369.26 369.16 1.00
AI PRO R9700 llama 8B Q4_K_S 512 pp2048 719.30 715.78 1.00
AI PRO R9700 llama 8B Q4_K_S 1024 pp2048 1333.21 1320.57 0.99
AI PRO R9700 llama 8B Q4_K_S 2048 pp2048 1928.76 2034.29 1.05
AI PRO R9700 llama 8B Q5_1 1 pp2048 78.26 78.38 1.00
AI PRO R9700 llama 8B Q5_1 2 pp2048 136.10 135.96 1.00
AI PRO R9700 llama 8B Q5_1 4 pp2048 246.88 246.80 1.00
AI PRO R9700 llama 8B Q5_1 8 pp2048 473.78 471.09 0.99
AI PRO R9700 llama 8B Q5_1 16 pp2048 689.87 689.10 1.00
AI PRO R9700 llama 8B Q5_1 32 pp2048 47.19 46.79 0.99
AI PRO R9700 llama 8B Q5_1 64 pp2048 94.29 93.89 1.00
AI PRO R9700 llama 8B Q5_1 128 pp2048 188.33 186.94 0.99
AI PRO R9700 llama 8B Q5_1 256 pp2048 370.61 368.40 0.99
AI PRO R9700 llama 8B Q5_1 512 pp2048 719.56 717.00 1.00
AI PRO R9700 llama 8B Q5_1 1024 pp2048 1347.15 1326.14 0.98
AI PRO R9700 llama 8B Q5_1 2048 pp2048 1866.44 1863.49 1.00
AI PRO R9700 llama 8B Q5_K_S 1 pp2048 78.52 78.36 1.00
AI PRO R9700 llama 8B Q5_K_S 2 pp2048 134.23 134.40 1.00
AI PRO R9700 llama 8B Q5_K_S 4 pp2048 196.35 196.71 1.00
AI PRO R9700 llama 8B Q5_K_S 8 pp2048 263.89 264.87 1.00
AI PRO R9700 llama 8B Q5_K_S 16 pp2048 865.49 868.00 1.00
AI PRO R9700 llama 8B Q5_K_S 32 pp2048 46.77 46.77 1.00
AI PRO R9700 llama 8B Q5_K_S 64 pp2048 93.94 93.84 1.00
AI PRO R9700 llama 8B Q5_K_S 128 pp2048 187.08 187.22 1.00
AI PRO R9700 llama 8B Q5_K_S 256 pp2048 367.31 366.52 1.00
AI PRO R9700 llama 8B Q5_K_S 512 pp2048 714.78 695.45 0.97
AI PRO R9700 llama 8B Q5_K_S 1024 pp2048 1331.90 1315.78 0.99
AI PRO R9700 llama 8B Q5_K_S 2048 pp2048 1892.93 1998.50 1.06
AI PRO R9700 llama 8B Q6_K 1 pp2048 71.76 71.68 1.00
AI PRO R9700 llama 8B Q6_K 2 pp2048 124.34 123.91 1.00
AI PRO R9700 llama 8B Q6_K 4 pp2048 199.76 199.31 1.00
AI PRO R9700 llama 8B Q6_K 8 pp2048 296.67 295.99 1.00
AI PRO R9700 llama 8B Q6_K 16 pp2048 659.90 657.28 1.00
AI PRO R9700 llama 8B Q6_K 32 pp2048 45.74 46.55 1.02
AI PRO R9700 llama 8B Q6_K 64 pp2048 91.53 93.49 1.02
AI PRO R9700 llama 8B Q6_K 128 pp2048 182.62 184.96 1.01
AI PRO R9700 llama 8B Q6_K 256 pp2048 356.95 366.90 1.03
AI PRO R9700 llama 8B Q6_K 512 pp2048 707.43 711.97 1.01
AI PRO R9700 llama 8B Q6_K 1024 pp2048 1118.18 1321.42 1.18
AI PRO R9700 llama 8B Q6_K 2048 pp2048 1412.75 2032.63 1.44
AI PRO R9700 llama 8B Q8_0 1 pp2048 62.74 62.76 1.00
AI PRO R9700 llama 8B Q8_0 2 pp2048 113.28 113.88 1.01
AI PRO R9700 llama 8B Q8_0 4 pp2048 209.16 209.69 1.00
AI PRO R9700 llama 8B Q8_0 8 pp2048 392.25 391.79 1.00
AI PRO R9700 llama 8B Q8_0 16 pp2048 747.38 747.35 1.00
AI PRO R9700 llama 8B Q8_0 32 pp2048 46.35 46.24 1.00
AI PRO R9700 llama 8B Q8_0 64 pp2048 92.61 93.48 1.01
AI PRO R9700 llama 8B Q8_0 128 pp2048 186.43 186.71 1.00
AI PRO R9700 llama 8B Q8_0 256 pp2048 365.10 366.09 1.00
AI PRO R9700 llama 8B Q8_0 512 pp2048 711.16 715.29 1.01
AI PRO R9700 llama 8B Q8_0 1024 pp2048 1309.86 1325.03 1.01
AI PRO R9700 llama 8B Q8_0 2048 pp2048 2048.52 2044.60 1.00

@loci-review
Copy link

loci-review bot commented Jan 13, 2026

Explore the complete analysis inside the Version Insights

Based on the analysis, no functions were identified with measurable performance changes between the base and target versions. This indicates no meaningful performance impact from the code changes.

@loci-dev loci-dev force-pushed the main branch 27 times, most recently from 839190f to f1b080b Compare January 18, 2026 02:54
@loci-dev loci-dev force-pushed the main branch 18 times, most recently from 4f9b49b to 30f9ba9 Compare January 23, 2026 17:12
@loci-review
Copy link

loci-review bot commented Jan 23, 2026

Performance Review Report

Summary

No functions were identified for performance analysis between the base and target versions. This indicates that no meaningful performance changes occurred in this code revision.

The analysis examined both response time and throughput time metrics across all functions in the binary, and found no significant variations that would warrant detailed investigation. The code changes between versions appear to be performance-neutral from a static analysis perspective.

See the complete breakdown in Version Insights
Have questions? Tag @loci-dev to ask about this PR.

@loci-dev loci-dev force-pushed the main branch 8 times, most recently from 5481840 to b98376c Compare January 25, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@loci-dev @jiachengjason