vulkan: GEMM/Group GEMM optimizations on Intel Xe (3/3, Xe-LPG Plus/Xe2/Xe3) by fish-jiang · Pull Request #24407 · ggml-org/llama.cpp

fish-jiang · 2026-06-10T09:34:42Z

Overview

PR 3/3 of the Intel Xe optimization series — see #24408 (mega PR, draft) for the full feature set.

Target platforms: Xe-LPG Plus, Xe2, Xe3

This PR adds GEMM/Group GEMM kernel optimizations and load-time weight compression for the Intel quick MoE path. Dependency: builds on top of #24404 (Xe-LPG Plus coopmat1 enable). Independent of #24406 (FA).

GEMM kernel optimizations (Intel Xe)

LOAD_A_OPT path: SLM-based A-matrix layout optimization for coopmat1
MXFP4, Q4_K, Q5_K dequant via bitfieldExtract optimization
Alt pipeline (l_alt/a_l_alt, BM=128 warptile) for runtime selection when problem dimensions are small
f32→f16 activation conversion for Intel coopmat GEMM, scoped to Intel devices only
vulkan-shaders-gen.cpp: registers all new pipeline variants

MoE optimizations (Intel Xe)

mul_mm.comp shader optimization for MUL_MAT_ID: reduces unnecessary memory loads and matrix core operations for MoE models
Separate warptile tuning for MoE expert GEMM
Gemma4 MoE router: fuse rms_norm + mul into a single RMS_NORM_MUL kernel dispatch for the expert gate input calculation

Performance (Panther Lake B390 + Windows OS)

BEFORE:
C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64>llama-bench.exe -p 8192 -n 0 -r 3 -fa 0,1 --delay 10 -ngl 99 -m C:\Users\dungeon\Desktop\models\Qwen3.5-35B-A3B-Q4_K_M\Qwen3.5-35B-A3B-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gpt-oss-20b-Q4_K_M\gpt-oss-20b-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gemma-4-26B-A4B-it-UD-Q4_K_M\gemma-4-26B-A4B-it-UD-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
load_backend: loaded RPC backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-rpc.dll
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(TM) B390 GPU (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-vulkan.dll
load_backend: loaded CPU backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-cpu-alderlake.dll
| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   0 |          pp8192 |       523.86 ± 21.17 |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   1 |          pp8192 |        439.07 ± 2.28 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   0 |          pp8192 |        496.90 ± 5.37 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   1 |          pp8192 |        561.50 ± 1.79 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   0 |          pp8192 |        615.07 ± 7.95 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   1 |          pp8192 |        414.96 ± 0.91 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   0 |          pp8192 |        330.11 ± 0.45 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   1 |          pp8192 |        220.38 ± 0.36 |

build: 3571fa543 (9490)

C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64>

AFTER:
C:\upsteaming_build\subPR2_GEMM_CW\Release>llama-bench.exe -p 8192 -n 0 -r 3 -fa 0,1 --delay 10 -ngl 99 -m C:\Users\dungeon\Desktop\models\Qwen3.5-35B-A3B-Q4_K_M\Qwen3.5-35B-A3B-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gpt-oss-20b-Q4_K_M\gpt-oss-20b-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gemma-4-26B-A4B-it-UD-Q4_K_M\gemma-4-26B-A4B-it-UD-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(TM) B390 GPU (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: KHR_coopmat
| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   0 |          pp8192 |       819.43 ± 47.14 |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   1 |          pp8192 |        689.30 ± 4.62 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   0 |          pp8192 |        727.97 ± 5.98 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   1 |          pp8192 |       724.32 ± 14.87 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   0 |          pp8192 |       848.26 ± 10.87 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   1 |          pp8192 |        493.29 ± 4.36 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   0 |          pp8192 |        469.47 ± 0.74 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   1 |          pp8192 |        229.64 ± 3.29 |

build: 84de82817 (9492)

C:\upsteaming_build\subPR2_GEMM_CW\Release>

BEFORE:
C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64>llama-bench.exe -p 0 -n 128 -d 8192 -r 3 -fa 0,1 --delay 10 -ngl 99 -m C:\Users\dungeon\Desktop\models\Qwen3.5-35B-A3B-Q4_K_M\Qwen3.5-35B-A3B-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gpt-oss-20b-Q4_K_M\gpt-oss-20b-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gemma-4-26B-A4B-it-UD-Q4_K_M\gemma-4-26B-A4B-it-UD-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
load_backend: loaded RPC backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-rpc.dll
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(TM) B390 GPU (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-vulkan.dll
load_backend: loaded CPU backend from C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64\ggml-cpu-alderlake.dll
| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         27.68 ± 0.08 |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         25.68 ± 0.04 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         30.60 ± 0.49 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         25.77 ± 0.11 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         22.76 ± 0.07 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         20.80 ± 0.07 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         26.63 ± 0.49 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         19.04 ± 0.02 |

build: 3571fa543 (9490)

C:\Users\dungeon\Downloads\llama-b9490-bin-win-vulkan-x64>

AFTER:
C:\upsteaming_build\subPR2_GEMM_CW\Release>llama-bench.exe -p 0 -n 128 -d 8192 -r 3 -fa 0,1 --delay 10 -ngl 99 -m C:\Users\dungeon\Desktop\models\Qwen3.5-35B-A3B-Q4_K_M\Qwen3.5-35B-A3B-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gpt-oss-20b-Q4_K_M\gpt-oss-20b-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\gemma-4-26B-A4B-it-UD-Q4_K_M\gemma-4-26B-A4B-it-UD-Q4_K_M.gguf,C:\Users\dungeon\Desktop\models\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M\Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Intel(R) Arc(TM) B390 GPU (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: KHR_coopmat
| model                          |       size |     params | backend    | ngl |  fa |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         36.15 ± 0.71 |
| qwen35moe 35B.A3B Q4_K - Medium |  20.49 GiB |    34.66 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         31.76 ± 0.38 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         30.69 ± 0.13 |
| gpt-oss 20B Q4_K - Medium      |  10.81 GiB |    20.91 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         25.68 ± 0.05 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         28.96 ± 0.14 |
| gemma4 26B.A4B Q4_K - Medium   |  15.70 GiB |    25.23 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         24.74 ± 1.05 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   0 |   tg128 @ d8192 |         27.08 ± 1.07 |
| qwen3moe 30B.A3B Q4_K - Medium |  17.28 GiB |    30.53 B | Vulkan     |  99 |   1 |   tg128 @ d8192 |         17.24 ± 0.04 |

build: 84de82817 (9492)

Requirements

I have read and agree with the contributing guidelines
AI usage disclosure: YES, used claude code, then lots of manual review/tweaking.

ggml-gh-bot · 2026-06-10T09:39:09Z

Hi @fish-jiang, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 4 open PRs.
AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

fish-jiang · 2026-06-15T08:49:15Z

Hi @0cc4m, would you be able to review this PR as well? Looking forward to your feedback.

0cc4m · 2026-06-15T09:23:43Z

I prefer to do them one step at a time. But I can already tell you that the load-time compression won't be accepted. The right way to do that would be providing "Intel-optimized" GGUF files.

fish-jiang · 2026-06-16T08:56:15Z

I prefer to do them one step at a time. But I can already tell you that the load-time compression won't be accepted. The right way to do that would be providing "Intel-optimized" GGUF files.

Thanks, removed the load-time compression.

jeffbolznv · 2026-06-16T13:06:12Z


-#ifdef COOPMAT
+#if defined(COOPMAT) && defined(LOAD_A_OPT)
+#define SHMEM_STRIDE (BK / 2)


It looks like the LOAD_A_OPT define is hardcoded based on shader name. This isn't OK, it will cause bank conflicts on most devices. We also can't define another set of shader variants, see #24663 (comment). So if you need a different stride, you should make it a spec constant (see conv2d_SHMEM_PAD for example).

Good point. Added a new spec constant for SHMEM_STRIDE_PAD, set to 0 on Intel GPU and decoupled from LOAD_A_OPT. No impact for other vendors.

Hi @jeffbolznv! We added a change for a new spec constant, would this be ok now?

jeffbolznv · 2026-06-16T13:09:08Z

            const uint idx = pos_a + col * p.stride_a / LOAD_VEC_A + row;
+#ifndef LOAD_A_OPT
            const uint buf_idx = col * SHMEM_STRIDE + row * LOAD_VEC_A / 2;
+#else


It's not clear to me why any of the ifdefs in this file are needed.

It's not clear to me why any of the ifdefs in this file are needed.

With LOAD_A_OPT macro, the SLM of matrix A (dtype=f16vec2) reshaped from [BM, BK / 2] to [BK / TK, BM, TK / 2], thus matrix A dequant goes with different code path under ifdefs

I assume this is to make a BM x TK slice of matrices contiguous in shared memory. Is this helping the stores or the loads or both? Can you go into any more detail?

I assume this is to make a BM x TK slice of matrices contiguous in shared memory. Is this helping the stores or the loads or both? Can you go into any more detail?

Exactly, on Intel GPU, contiguous BM x TK slice for matrix A can help to optimize the SLM load instruction count by 8x, while SLM store instruction count almost flat.
As you suggested, we were also looking for possible method to avoid #ifdefs in this file.

Hi @jeffbolznv thanks for review on this, any concern? :)

Rename LOAD_A_OPT to APPLY_SLM_A_RESHAPE, and add it as one spec constant, no hardcoded based on shader name now.

Using a spec constant is definitely an improvement. I'll try to look at this change in more detail today or tomorrow, but here's some "gut reaction" feedback:

It would be nice if we could either pull the shared memory addressing logic out into a function, or somehow reparameterize and unify the two code paths, to make it easier to maintain.

This still feels like multiple PRs squashed into one - the shared memory shaping, alt pipelines, f16 conversion, mmvq tuning. It will be much easier to review and reason about each of these if they are in their own PR with their own perf justification.

Regarding the first bullet, I asked codex to verify whether this can apply to all the changes and how it would express it, and this is what it suggested:

// k_pair is the K coordinate measured in FLOAT_TYPEV2 elements. uint a_shmem_index(uint m, uint k_pair) { if (APPLY_SLM_A_RESHAPE) { const uint tile_width = TK / 2; return (k_pair / tile_width) * BM * tile_width + m * tile_width + k_pair % tile_width; } return m * SHMEM_STRIDE + k_pair; } uint a_shmem_stride() { return APPLY_SLM_A_RESHAPE ? TK / 2 : SHMEM_STRIDE; } void store_a(uint m, uint k_pair, FLOAT_TYPEV2 value) { buf_a[a_shmem_index(m, k_pair)] = value; } The layouts represented are exactly: normal: [M][SHMEM_STRIDE] reshaped: [BK / TK][M][TK / 2] The cooperative-matrix load also matches: const uint m = warp_r * WM + cm_row * TM; coopMatLoad( cache_a[cm_row], buf_a, a_shmem_index(m, i / 2), a_shmem_stride(), gl_CooperativeMatrixLayoutRowMajor); For the reshaped case, because i advances by TK: a_shmem_index(m, i / 2) = (i / TK) * BM * (TK / 2) + m * (TK / 2) = i * BM / 2 + m * TK / 2 That is identical to the branch's current load address at ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:324. I checked all the store patterns: - F32, F16, BF16 and most quant formats use: const uint k_pair = row * LOAD_VEC_A / 2; store_a(col, k_pair + offset, value); - Unaligned F32/F16/BF16 uses: const uint k_pair = row; - Q4_0, Q4_1, Q5_1 and IQ4_NL use split halves: const uint k_pair = row * LOAD_VEC_A / 4; store_a(col, k_pair, value0); store_a(col, k_pair + 1, value1); store_a(col, k_pair + 8, value2); store_a(col, k_pair + 9, value3); - Q5_0 and MXFP4 use the same split with k_pair = row. - NVFP4's special mapping remains explicit, but only for its logical K coordinate: const uint k_pair = (row & 3) + (row & ~3) * 2; store_a(col, k_pair, low); store_a(col, k_pair + 4, high); Thus all format-specific code retains responsibility for determining which logical K values it decoded, while one helper owns the physical shared-memory layout.

So, IMO, this seems like a viable direction that would keep the code more maintainable going forward (e.g. when new quants are added). But I'd still like to see this part split into its own PR with its own perf justification (and to make it easier for us to test).

Thanks, very good suggestion and direction. Code change done. And next step will split this part into a new PR.

Regarding the first bullet, I asked codex to verify whether this can apply to all the changes and how it would express it, and this is what it suggested:

// k_pair is the K coordinate measured in FLOAT_TYPEV2 elements. uint a_shmem_index(uint m, uint k_pair) { if (APPLY_SLM_A_RESHAPE) { const uint tile_width = TK / 2; return (k_pair / tile_width) * BM * tile_width + m * tile_width + k_pair % tile_width; } return m * SHMEM_STRIDE + k_pair; } uint a_shmem_stride() { return APPLY_SLM_A_RESHAPE ? TK / 2 : SHMEM_STRIDE; } void store_a(uint m, uint k_pair, FLOAT_TYPEV2 value) { buf_a[a_shmem_index(m, k_pair)] = value; } The layouts represented are exactly: normal: [M][SHMEM_STRIDE] reshaped: [BK / TK][M][TK / 2] The cooperative-matrix load also matches: const uint m = warp_r * WM + cm_row * TM; coopMatLoad( cache_a[cm_row], buf_a, a_shmem_index(m, i / 2), a_shmem_stride(), gl_CooperativeMatrixLayoutRowMajor); For the reshaped case, because i advances by TK: a_shmem_index(m, i / 2) = (i / TK) * BM * (TK / 2) + m * (TK / 2) = i * BM / 2 + m * TK / 2 That is identical to the branch's current load address at ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp:324. I checked all the store patterns: - F32, F16, BF16 and most quant formats use: const uint k_pair = row * LOAD_VEC_A / 2; store_a(col, k_pair + offset, value); - Unaligned F32/F16/BF16 uses: const uint k_pair = row; - Q4_0, Q4_1, Q5_1 and IQ4_NL use split halves: const uint k_pair = row * LOAD_VEC_A / 4; store_a(col, k_pair, value0); store_a(col, k_pair + 1, value1); store_a(col, k_pair + 8, value2); store_a(col, k_pair + 9, value3); - Q5_0 and MXFP4 use the same split with k_pair = row. - NVFP4's special mapping remains explicit, but only for its logical K coordinate: const uint k_pair = (row & 3) + (row & ~3) * 2; store_a(col, k_pair, low); store_a(col, k_pair + 4, high); Thus all format-specific code retains responsibility for determining which logical K values it decoded, while one helper owns the physical shared-memory layout.

So, IMO, this seems like a viable direction that would keep the code more maintainable going forward (e.g. when new quants are added). But I'd still like to see this part split into its own PR with its own perf justification (and to make it easier for us to test).

Hi @jeffbolznv, thanks, created a new PR for this firstly. #25380. Please help review it.

jeffbolznv · 2026-06-16T13:11:02Z

 #ifdef COOPMAT
-    coopmat<FLOAT_TYPE, gl_ScopeSubgroup, TM, TK, gl_MatrixUseA> cache_a;
-    coopmat<FLOAT_TYPE, gl_ScopeSubgroup, TK, TN, gl_MatrixUseB> cache_b;
+    coopmat<FLOAT_TYPE, gl_ScopeSubgroup, TM, TK, gl_MatrixUseA> cache_a[cms_per_row];


It shouldn't be necessary to "batch" the loads from shared memory - the compiler should unroll the loops and is free to reorder the loads and matmuls. In fact I think this change is harmful when combined with the if (warp_c < required_warp_c) { because it will make the compiler think the values all need to be preserved across loop iterations.

It shouldn't be necessary to "batch" the loads from shared memory - the compiler should unroll the loops and is free to reorder the loads and matmuls. In fact I think this change is harmful when combined with the if (warp_c < required_warp_c) { because it will make the compiler think the values all need to be preserved across loop iterations.

Fully agree that ideally compiler should handle the efficient matrix load reorder and variable allocation, the explicit variable declarations and loop order here is to give hint for non-finetuned compiler. Suppose not to impact finetuned compilers that have built-in optimal coopMat flow.

I think for this to be OK, you'd need to move the declarations inside the if (warp_c < required_warp_c) { block, so the compiler wouldn't think it needs to preserve the values across loop iterations.

Moved the declarations inside the "if (warp_c < required_warp_c) {"

So this is another case of putting optimizations into shaders that should be done in the driver? People have done that before and I'd really prefer you to work on the driver instead. Of course the threshold is fuzzy, I have actually tried this kind of change before, but I did not see an improvement on any of the devices I tested.

Below is the perf data for the 'batch' load on Intel PTL iGPU B390. test-backed-ops perf -o
MUL_MAT,MUL_MAT_ID. The GEMV perf tests are noise, since they are not impacted by this 'batch' load change.

Perf: `build_no_batch_load` vs `build_batch_load`

Intel Arc B390 · test-backend-ops perf · ratio = batch_load / no_batch_load (>1 = batch_load faster). 🟢 ≥ +5 % ⚪ ±5 % 🔴 ≤ −5 %

# GGML Operator no_batch_load (GFLOPS) batch_load (GFLOPS) ratio %

1 MUL_MAT(type_a=f16,type_b=f32,m=16416,n=1,k=128,bs=[8,1],nr=[4,1],per=[0,2,1,3],k_v=0,o=1) 389.16 396.53 1.0189 +1.9% ⚪

2 MUL_MAT(type_a=f16,type_b=f32,m=128,n=1,k=16416,bs=[8,1],nr=[4,1],per=[0,1,2,3],k_v=32832,o=1) 362.14 378.5 1.0452 +4.5% ⚪

7 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 56 56.57 1.0102 +1.0% ⚪

8 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 110.93 112.1 1.0105 +1.1% ⚪

9 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 110.82 112.12 1.0117 +1.2% ⚪

10 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 391.01 390.23 0.9980 -0.2% ⚪

11 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 349.63 352.72 1.0088 +0.9% ⚪

12 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 313.51 313.02 0.9984 -0.2% ⚪

13 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 289.87 290.56 1.0024 +0.2% ⚪

14 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 206.95 208.88 1.0093 +0.9% ⚪

15 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1470 1480 1.0068 +0.7% ⚪

16 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 411.32 413.22 1.0046 +0.5% ⚪

17 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 374.18 379.67 1.0147 +1.5% ⚪

18 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 914.5 789.49 0.8633 -13.7% 🔴

19 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 439.47 504.55 1.1481 +14.8% 🟢

20 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 391.4 392.76 1.0035 +0.3% ⚪

21 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 315.39 314.99 0.9987 -0.1% ⚪

22 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 264.07 265.47 1.0053 +0.5% ⚪

23 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 671.17 697.36 1.0390 +3.9% ⚪

24 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 643.82 687.55 1.0679 +6.8% 🟢

25 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 454.51 494.72 1.0885 +8.8% 🟢

26 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 522.07 540.65 1.0356 +3.6% ⚪

27 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1260 1450 1.1508 +15.1% 🟢

28 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 989.27 1130 1.1423 +14.2% 🟢

29 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 392.68 392.83 1.0004 +0.0% ⚪

30 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 455.33 475.41 1.0441 +4.4% ⚪

31 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=1,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 409.84 413.74 1.0095 +1.0% ⚪

32 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 112.99 112.95 0.9996 +0.0% ⚪

33 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 223 223.57 1.0026 +0.3% ⚪

34 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 223.41 218.91 0.9799 -2.0% ⚪

35 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 776.22 767.22 0.9884 -1.2% ⚪

36 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 693.14 692.56 0.9992 -0.1% ⚪

37 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 613.55 610.3 0.9947 -0.5% ⚪

38 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 574.01 579.07 1.0088 +0.9% ⚪

39 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 406.78 412.08 1.0130 +1.3% ⚪

40 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1530 1680 1.0980 +9.8% 🟢

41 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 667.6 708.82 1.0617 +6.2% 🟢

42 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 610.51 671 1.0991 +9.9% 🟢

43 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1490 1530 1.0268 +2.7% ⚪

44 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 798.26 854.14 1.0700 +7.0% 🟢

45 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 784.36 780.16 0.9946 -0.5% ⚪

46 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 621.89 624.54 1.0043 +0.4% ⚪

47 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 517.48 521.13 1.0071 +0.7% ⚪

48 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1100 1180 1.0727 +7.3% 🟢

49 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 994.52 1070 1.0759 +7.6% 🟢

50 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 731.27 775.74 1.0608 +6.1% 🟢

51 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 900.96 916.45 1.0172 +1.7% ⚪

52 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1930 2160 1.1192 +11.9% 🟢

53 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1490 1720 1.1544 +15.4% 🟢

54 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 774.96 779.7 1.0061 +0.6% ⚪

55 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 657.63 692.43 1.0529 +5.3% 🟢

56 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=2,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 782.77 815.48 1.0418 +4.2% ⚪

57 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 166.5 169.5 1.0180 +1.8% ⚪

58 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 331.52 335.3 1.0114 +1.1% ⚪

59 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 332.18 334.97 1.0084 +0.8% ⚪

60 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1120 1140 1.0179 +1.8% ⚪

61 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1010 1030 1.0198 +2.0% ⚪

62 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 884.22 912.44 1.0319 +3.2% ⚪

63 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 846.22 854.98 1.0104 +1.0% ⚪

64 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 608.17 612.61 1.0073 +0.7% ⚪

65 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1710 1810 1.0585 +5.8% 🟢

66 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 918.27 999.69 1.0887 +8.9% 🟢

67 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 848.46 912.89 1.0759 +7.6% 🟢

68 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1920 2040 1.0625 +6.2% 🟢

69 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1100 1180 1.0727 +7.3% 🟢

70 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1140 1150 1.0088 +0.9% ⚪

71 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 927.1 930.27 1.0034 +0.3% ⚪

72 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 763.31 766.81 1.0046 +0.5% ⚪

73 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1200 1260 1.0500 +5.0% 🟢

74 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1180 1290 1.0932 +9.3% 🟢

75 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 891.95 906.56 1.0164 +1.6% ⚪

76 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1060 1100 1.0377 +3.8% ⚪

77 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2590 3170 1.2239 +22.4% 🟢

78 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1860 2080 1.1183 +11.8% 🟢

79 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1150 1150 1.0000 +0.0% ⚪

80 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 747.56 790.81 1.0579 +5.8% 🟢

81 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=3,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1100 1130 1.0273 +2.7% ⚪

82 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 225.73 225.44 0.9987 -0.1% ⚪

83 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 442.36 442.12 0.9995 -0.1% ⚪

84 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 443.33 442.62 0.9984 -0.2% ⚪

85 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1490 1490 1.0000 +0.0% ⚪

86 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1350 1360 1.0074 +0.7% ⚪

87 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1170 1190 1.0171 +1.7% ⚪

88 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1110 1110 1.0000 +0.0% ⚪

89 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 811.14 809.66 0.9982 -0.2% ⚪

90 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1600 1750 1.0938 +9.4% 🟢

91 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1180 1290 1.0932 +9.3% 🟢

92 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1040 1170 1.1250 +12.5% 🟢

93 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2200 2190 0.9955 -0.5% ⚪

94 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1320 1440 1.0909 +9.1% 🟢

95 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1500 1520 1.0133 +1.3% ⚪

96 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1210 1230 1.0165 +1.7% ⚪

97 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 980.56 995.93 1.0157 +1.6% ⚪

98 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1330 1330 1.0000 +0.0% ⚪

99 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1150 1340 1.1652 +16.5% 🟢

100 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 888.09 975.77 1.0987 +9.9% 🟢

101 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1150 1270 1.1043 +10.4% 🟢

102 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2870 3570 1.2439 +24.4% 🟢

103 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2150 2460 1.1442 +14.4% 🟢

104 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1440 1480 1.0278 +2.8% ⚪

105 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 859.61 912.83 1.0619 +6.2% 🟢

106 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=4,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1300 1380 1.0615 +6.2% 🟢

107 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 280.43 281.42 1.0035 +0.4% ⚪

108 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 546.79 554.8 1.0146 +1.5% ⚪

109 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 550.62 553.13 1.0046 +0.5% ⚪

110 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1800 1840 1.0222 +2.2% ⚪

111 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1630 1650 1.0123 +1.2% ⚪

112 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1430 1450 1.0140 +1.4% ⚪

113 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1360 1380 1.0147 +1.5% ⚪

114 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 996 1010 1.0141 +1.4% ⚪

115 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1660 1830 1.1024 +10.2% 🟢

116 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1360 1530 1.1250 +12.5% 🟢

117 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1200 1330 1.1083 +10.8% 🟢

118 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2380 2500 1.0504 +5.0% 🟢

119 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1500 1750 1.1667 +16.7% 🟢

120 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1830 1860 1.0164 +1.6% ⚪

121 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1490 1500 1.0067 +0.7% ⚪

122 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1180 1220 1.0339 +3.4% ⚪

123 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1210 1340 1.1074 +10.7% 🟢

124 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1080 1190 1.1019 +10.2% 🟢

125 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1030 1120 1.0874 +8.7% 🟢

126 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1160 1230 1.0603 +6.0% 🟢

127 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 3260 3930 1.2055 +20.6% 🟢

128 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2120 2400 1.1321 +13.2% 🟢

129 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1620 1730 1.0679 +6.8% 🟢

130 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 811.75 893.64 1.1009 +10.1% 🟢

131 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=5,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1450 1490 1.0276 +2.8% ⚪

132 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 444.11 446.96 1.0064 +0.6% ⚪

133 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 867.04 876.74 1.0112 +1.1% ⚪

134 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 868.89 876.73 1.0090 +0.9% ⚪

135 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2190 2500 1.1416 +14.2% 🟢

136 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2170 2290 1.0553 +5.5% 🟢

137 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1850 2100 1.1351 +13.5% 🟢

138 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1820 2010 1.1044 +10.4% 🟢

139 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1550 1570 1.0129 +1.3% ⚪

140 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1570 2010 1.2803 +28.0% 🟢

141 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1830 2120 1.1585 +15.8% 🟢

142 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1230 1520 1.2358 +23.6% 🟢

143 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2720 2850 1.0478 +4.8% ⚪

144 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1920 2450 1.2760 +27.6% 🟢

145 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2310 2450 1.0606 +6.1% 🟢

146 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2100 2210 1.0524 +5.2% 🟢

147 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1690 1830 1.0828 +8.3% 🟢

148 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1210 1300 1.0744 +7.4% 🟢

149 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1150 1240 1.0783 +7.8% 🟢

150 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1100 1200 1.0909 +9.1% 🟢

151 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1170 1210 1.0342 +3.4% ⚪

152 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 3470 4110 1.1844 +18.4% 🟢

153 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1770 1820 1.0282 +2.8% ⚪

154 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1980 2170 1.0960 +9.6% 🟢

155 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 450.4 441.07 0.9793 -2.1% ⚪

156 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=8,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 1490 1710 1.1477 +14.8% 🟢

157 MUL_MAT(type_a=f32,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 8280 9620 1.1618 +16.2% 🟢

158 MUL_MAT(type_a=f16,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 12120 13890 1.1460 +14.6% 🟢

159 MUL_MAT(type_a=bf16,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 2380 2410 1.0126 +1.3% ⚪

160 MUL_MAT(type_a=q4_0,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16030 19390 1.2096 +21.0% 🟢

161 MUL_MAT(type_a=q4_1,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16450 18840 1.1453 +14.5% 🟢

162 MUL_MAT(type_a=q5_0,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15640 18760 1.1995 +19.9% 🟢

163 MUL_MAT(type_a=q5_1,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16300 19500 1.1963 +19.6% 🟢

164 MUL_MAT(type_a=q8_0,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 14920 18110 1.2138 +21.4% 🟢

165 MUL_MAT(type_a=q1_0,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 17520 21970 1.2540 +25.4% 🟢

166 MUL_MAT(type_a=mxfp4,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 14350 17730 1.2355 +23.6% 🟢

167 MUL_MAT(type_a=nvfp4,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 13880 15960 1.1499 +15.0% 🟢

168 MUL_MAT(type_a=q2_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16130 19410 1.2033 +20.3% 🟢

169 MUL_MAT(type_a=q3_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 12690 14440 1.1379 +13.8% 🟢

170 MUL_MAT(type_a=q4_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15610 18490 1.1845 +18.4% 🟢

171 MUL_MAT(type_a=q5_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 14630 17680 1.2085 +20.8% 🟢

172 MUL_MAT(type_a=q6_K,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 13170 15730 1.1944 +19.4% 🟢

173 MUL_MAT(type_a=iq2_xxs,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 14720 18540 1.2595 +26.0% 🟢

174 MUL_MAT(type_a=iq2_xs,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16060 19550 1.2173 +21.7% 🟢

175 MUL_MAT(type_a=iq2_s,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15580 18630 1.1958 +19.6% 🟢

176 MUL_MAT(type_a=iq3_xxs,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15560 18400 1.1825 +18.3% 🟢

177 MUL_MAT(type_a=iq1_s,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16740 18780 1.1219 +12.2% 🟢

178 MUL_MAT(type_a=iq1_m,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16450 20140 1.2243 +22.4% 🟢

179 MUL_MAT(type_a=iq4_nl,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 16010 19510 1.2186 +21.9% 🟢

180 MUL_MAT(type_a=iq3_s,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15000 17530 1.1687 +16.9% 🟢

181 MUL_MAT(type_a=iq4_xs,type_b=f32,m=4096,n=512,k=14336,bs=[1,1],nr=[1,1],per=[0,1,2,3],k_v=0,o=1) 15210 18410 1.2104 +21.0% 🟢

182 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 1030 1030 1.0000 +0.0% ⚪

183 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 1950 1990 1.0205 +2.1% ⚪

184 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 4870 5180 1.0637 +6.4% 🟢

185 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 3890 3770 0.9692 -3.1% ⚪

186 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 5290 5330 1.0076 +0.8% ⚪

187 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 3050 2940 0.9639 -3.6% ⚪

188 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=1,k=2048) 751.45 821.03 1.0926 +9.3% 🟢

189 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 1400 1540 1.1000 +10.0% 🟢

190 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 2520 2890 1.1468 +14.7% 🟢

191 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 7560 7350 0.9722 -2.8% ⚪

192 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 7070 7070 1.0000 +0.0% ⚪

193 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 4230 4200 0.9929 -0.7% ⚪

194 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 4150 4140 0.9976 -0.2% ⚪

195 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=4,k=2048) 868.34 881.27 1.0149 +1.5% ⚪

196 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 1620 1650 1.0185 +1.9% ⚪

197 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 3100 3150 1.0161 +1.6% ⚪

198 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 8410 7880 0.9370 -6.3% 🔴

199 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 7970 7600 0.9536 -4.6% ⚪

200 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 4450 4390 0.9865 -1.3% ⚪

201 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 4410 4350 0.9864 -1.4% ⚪

202 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=8,k=2048) 889.67 892.23 1.0029 +0.3% ⚪

203 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 108.89 124.9 1.1470 +14.7% 🟢

204 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 236 242.04 1.0256 +2.6% ⚪

205 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 386.87 430.96 1.1140 +11.4% 🟢

206 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 338.17 359.99 1.0645 +6.5% 🟢

207 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 323.4 360.02 1.1132 +11.3% 🟢

208 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 270.07 289.13 1.0706 +7.1% 🟢

209 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=32,k=2048) 249.56 273 1.0939 +9.4% 🟢

210 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 199.14 197.19 0.9902 -1.0% ⚪

211 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 434.17 437.6 1.0079 +0.8% ⚪

212 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 777.75 926.45 1.1912 +19.1% 🟢

213 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 707.46 741.68 1.0484 +4.8% ⚪

214 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 789.22 886.02 1.1227 +12.3% 🟢

215 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 551.44 604.17 1.0956 +9.6% 🟢

216 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=64,k=2048) 809.06 920.16 1.1373 +13.7% 🟢

217 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 351.32 349.6 0.9951 -0.5% ⚪

218 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 836.4 848.14 1.0140 +1.4% ⚪

219 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 1600 1900 1.1875 +18.8% 🟢

220 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 1290 1410 1.0930 +9.3% 🟢

221 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 1600 1880 1.1750 +17.5% 🟢

222 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 1080 1190 1.1019 +10.2% 🟢

223 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=128,k=2048) 1480 1740 1.1757 +17.6% 🟢

224 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 694.23 693.77 0.9993 -0.1% ⚪

225 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 1600 1600 1.0000 +0.0% ⚪

226 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 2960 3470 1.1723 +17.2% 🟢

227 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 2450 2630 1.0735 +7.3% 🟢

228 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 2980 3500 1.1745 +17.4% 🟢

229 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 2010 2300 1.1443 +14.4% 🟢

230 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=256,k=2048) 2740 3040 1.1095 +10.9% 🟢

231 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 1330 1320 0.9925 -0.8% ⚪

232 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 2960 2970 1.0034 +0.3% ⚪

233 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 5230 5900 1.1281 +12.8% 🟢

234 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 4480 4570 1.0201 +2.0% ⚪

235 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 5270 6080 1.1537 +15.4% 🟢

236 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 3740 4110 1.0989 +9.9% 🟢

237 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=128,n_used=8,b=0,m=768,n=512,k=2048) 4920 5440 1.1057 +10.6% 🟢

238 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 1260 1240 0.9841 -1.6% ⚪

239 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 2480 2470 0.9960 -0.4% ⚪

240 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 5990 5960 0.9950 -0.5% ⚪

241 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 4640 4630 0.9978 -0.2% ⚪

242 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 6210 6070 0.9775 -2.3% ⚪

243 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 3590 3530 0.9833 -1.7% ⚪

244 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=1,k=2048) 837.78 830.25 0.9910 -0.9% ⚪

245 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 1630 1640 1.0061 +0.6% ⚪

246 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 3120 3080 0.9872 -1.3% ⚪

247 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 7750 7680 0.9910 -0.9% ⚪

248 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 7410 7160 0.9663 -3.4% ⚪

249 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 4340 4310 0.9931 -0.7% ⚪

250 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 4190 4270 1.0191 +1.9% ⚪

251 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=4,k=2048) 881.61 891.58 1.0113 +1.1% ⚪

252 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 1710 1730 1.0117 +1.2% ⚪

253 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 3250 3220 0.9908 -0.9% ⚪

254 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 8380 8300 0.9905 -1.0% ⚪

255 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 8360 8140 0.9737 -2.6% ⚪

256 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 4570 4460 0.9759 -2.4% ⚪

257 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 4440 4470 1.0068 +0.7% ⚪

258 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=8,k=2048) 867.88 894.78 1.0310 +3.1% ⚪

259 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 195.14 196.76 1.0083 +0.8% ⚪

260 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 414.88 424.36 1.0228 +2.3% ⚪

261 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 629.55 702.17 1.1154 +11.5% 🟢

262 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 523.31 595.73 1.1384 +13.8% 🟢

263 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 528.24 595.63 1.1276 +12.8% 🟢

264 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 415.01 466.79 1.1248 +12.5% 🟢

265 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=32,k=2048) 371.74 430.65 1.1585 +15.8% 🟢

266 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 377.09 375.15 0.9949 -0.5% ⚪

267 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 843.21 840.69 0.9970 -0.3% ⚪

268 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 1400 1590 1.1357 +13.6% 🟢

269 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 1260 1350 1.0714 +7.1% 🟢

270 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 1440 1640 1.1389 +13.9% 🟢

271 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 994.02 1080 1.0865 +8.6% 🟢

272 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=64,k=2048) 1420 1650 1.1620 +16.2% 🟢

273 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 682.92 688.18 1.0077 +0.8% ⚪

274 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 1650 1650 1.0000 +0.0% ⚪

275 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 3010 3550 1.1794 +17.9% 🟢

276 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 2510 2720 1.0837 +8.4% 🟢

277 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 2990 3520 1.1773 +17.7% 🟢

278 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 2120 2360 1.1132 +11.3% 🟢

279 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=128,k=2048) 2830 3170 1.1201 +12.0% 🟢

280 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 1330 1360 1.0226 +2.3% ⚪

281 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 3120 3130 1.0032 +0.3% ⚪

282 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 5380 6260 1.1636 +16.4% 🟢

283 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 4480 4890 1.0915 +9.2% 🟢

284 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 5370 6200 1.1546 +15.5% 🟢

285 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 3900 4290 1.1000 +10.0% 🟢

286 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=256,k=2048) 5000 5640 1.1280 +12.8% 🟢

287 MUL_MAT_ID(type_a=f32,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 2570 2620 1.0195 +1.9% ⚪

288 MUL_MAT_ID(type_a=f16,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 5570 5710 1.0251 +2.5% ⚪

289 MUL_MAT_ID(type_a=q4_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 8720 9920 1.1376 +13.8% 🟢

290 MUL_MAT_ID(type_a=q8_0,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 7700 8120 1.0545 +5.5% 🟢

291 MUL_MAT_ID(type_a=q4_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 8930 9920 1.1109 +11.1% 🟢

292 MUL_MAT_ID(type_a=q6_K,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 6670 7230 1.0840 +8.4% 🟢

293 MUL_MAT_ID(type_a=iq2_xs,type_b=f32,n_mats=32,n_used=4,b=0,m=1792,n=512,k=2048) 8160 8900 1.0907 +9.1% 🟢

294 MUL_MAT_ID(type_a=mxfp4,type_b=f32,n_mats=32,n_used=4,b=0,m=2880,n=1,k=2880) 7890 7740 0.9810 -1.9% ⚪

295 MUL_MAT_ID(type_a=mxfp4,type_b=f32,n_mats=32,n_used=4,b=0,m=2880,n=4,k=2880) 17700 17290 0.9768 -2.3% ⚪

296 MUL_MAT_ID(type_a=mxfp4,type_b=f32,n_mats=32,n_used=4,b=0,m=2880,n=8,k=2880) 19300 19480 1.0093 +0.9% ⚪

297 MUL_MAT_ID(type_a=mxfp4,type_b=f32,n_mats=32,n_used=4,b=0,m=2880,n=512,k=2880) 8970 10420 1.1616 +16.2% 🟢

Hi @fish-jiang if possible can we please put above data into a log / txt file, it's too long on the page.

@jeffbolznv @0cc4m Thanks for the feedback, we have notified someone on our driver side and he should comment soon. But otherwise, would such a change still be ok for merge, or you feel it is a blocking change and should only be addressed in driver?

I would move the declaration inside the for (uint i = 0; i < BK; i += TK) { loop, so there's less chance the compiler should think the variables are live across iterations. But I'm generally OK with making a change like this, since it shouldn't penalize other vendors (but I haven't actually tested it).

Moved the declaration inside the loop for (uint i = 0; i < BK; i += TK) {.

I split out one new small PR from this PR for the MoE shader code changes: #25483. The perf data shows it helps a lot on Intel GPUs, @jeffbolznv, @0cc4m could you please help review it and check whether it would also benefit other vendors?

jeffbolznv · 2026-06-16T13:14:17Z

+
+    if (ctx->device->vendor_id == VK_VENDOR_ID_INTEL &&
+        n < 256 && m < 4096 &&
+        (aligned ? mmp->a_l_alt : mmp->l_alt) != nullptr) {


Some of the changes in this PR, like this "alt" stuff, seems like it may be overfitted to specific models.

The current Vulkan codebase only has a few warptile settings, like one l_warptile. Some shapes are not optimal with it, so I added l_alt here. Do you have any suggestions or guidelines for building more?

Hi @jeffbolznv just following up on this one!

What shapes and why are they not covered well-enough by existing tile sizes?

layout (constant_id = 1) const uint BM = 64;
layout (constant_id = 2) const uint BN = 64;
layout (constant_id = 4) const uint WM = 32;
layout (constant_id = 5) const uint WN = 32;

What I care about here is WM & WN — I want to use different values for different GEMM shapes, in other words, use a different number of subgroups within one workgroup based on M/N shape. However, seems these spec constants are not easy to change at dispatch time; the corresponding warptile must be created ahead of time.

I'm looking into improving how shapes are handled, leave it for the future.

virajwad · 2026-07-01T16:12:04Z

Hi @jeffbolznv @0cc4m thanks for the review so far - could you please check above code review threads? Also, if you have any other feedback please let us know!

…Xe2/Xe3) Co-authored-by: Xia, Jie <jie.xia@intel.com> Co-authored-by: Liu, Russell <russell.liu@intel.com>

…move coopmat cache declarations inside inner loop

fish-jiang requested review from a team, CISC and ggerganov as code owners June 10, 2026 09:34

fish-jiang marked this pull request as draft June 10, 2026 09:34

github-actions Bot added model Model specific Vulkan Issues specific to the Vulkan backend examples ggml changes relating to the ggml tensor library for machine learning labels Jun 10, 2026

This was referenced Jun 10, 2026

vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) #24406

Draft

vulkan: Intel Xe flash attention, GEMM optimizations(Xe-LPG Plus/Xe2/Xe3) [MEGA PR] #24408

Draft

fish-jiang force-pushed the intel/xe-gemm-cw branch from 500eb77 to 8d5877e Compare June 16, 2026 08:41

fish-jiang changed the title ~~vulkan: GEMM/Group GEMM optimizations and optional load-time weight compression for Intel MoE path (3/3, Xe-LPG Plus/Xe2/Xe3)~~ vulkan: GEMM/Group GEMM optimizations on Intel Xe (3/3, Xe-LPG Plus/Xe2/Xe3) Jun 16, 2026

jeffbolznv reviewed Jun 16, 2026

View reviewed changes

fish-jiang force-pushed the intel/xe-gemm-cw branch from eefefe7 to 81e1768 Compare July 2, 2026 01:29

fish-jiang and others added 3 commits July 2, 2026 09:42

vulkan: GEMM/Group GEMM optimizations for Intel Xe (3/3, Xe-LPG Plus/…

1a63d54

…Xe2/Xe3) Co-authored-by: Xia, Jie <jie.xia@intel.com> Co-authored-by: Liu, Russell <russell.liu@intel.com>

Add a new spec constant for SHMEM_STRIDE_PAD

4d54c3a

Move coopmat cache_a/cache_b declarations inside loop body

cb11307

fish-jiang force-pushed the intel/xe-gemm-cw branch 3 times, most recently from 5b0f895 to 375e32f Compare July 2, 2026 02:45

Rename LOAD_A_OPT to APPLY_SLM_A_RESHAPE, and add a new spec constant

4c62a3a

fish-jiang force-pushed the intel/xe-gemm-cw branch from 375e32f to 4c62a3a Compare July 2, 2026 08:05

vulkan: refactor SLM A addressing via a_shmem_index/store_a helpers; …

4cdf2db

…move coopmat cache declarations inside inner loop

SleepinDevil mentioned this pull request Jul 4, 2026

[SYCL] Flash Attention with XMX engine via oneDNN graph API (SDPA) on KV f16 for Xe2 ; Qwen3.6-27b-Q8_0 prefill speed up x1.21 at p=512 and x4.26 at p=80k #25222

Merged

This was referenced Jul 7, 2026

vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat1 on Intel Xe #25380

Merged

vulkan: skip unneeded MoE work in mul_mm coopmat1 path #25483

Draft

0cc4m mentioned this pull request Jul 16, 2026

vulkan: use spec constant for matrix matrix multiplication A-type #25773

Draft

theycallmeloki mentioned this pull request Aug 18, 2026

vulkan: use density gate for MUL_MAT_VEC_ID path #27332

Open

fish-jiang mentioned this pull request Aug 21, 2026

vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 #27471

Draft

Uh oh!

Conversation

fish-jiang commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

GEMM kernel optimizations (Intel Xe)

MoE optimizations (Intel Xe)

Performance (Panther Lake B390 + Windows OS)

Requirements

Uh oh!

ggml-gh-bot Bot commented Jun 10, 2026

Uh oh!

fish-jiang commented Jun 15, 2026

Uh oh!

0cc4m commented Jun 15, 2026

Uh oh!

fish-jiang commented Jun 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sliu39 Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fish-jiang Jun 17, 2026 • edited by 0cc4m Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fish-jiang Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fish-jiang commented Jun 10, 2026 •

edited

Loading

sliu39 Jun 17, 2026 •

edited

Loading

fish-jiang Jun 17, 2026 •

edited by 0cc4m

Loading

fish-jiang Jul 3, 2026 •

edited

Loading

virajwad commented Jul 1, 2026 •

edited

Loading