opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance - #26438
Merged
Merged
Conversation
github-actions
Bot
added
ggml
wanghqc
force-pushed
the
hq/intel-quant-tuning-r0730
branch
from
August 8, 2026 19:04
1573f64 to
a3de642
Compare
wanghqc
force-pushed
the
hq/intel-quant-tuning-r0730
branch
from
August 22, 2026 20:47
a3de642 to
5665bff
Compare
lhez
force-pushed
the
hq/intel-quant-tuning-r0730
branch
from
August 30, 2026 20:44
5665bff to
b0e0fd1
Compare
lhez
approved these changes
Aug 30, 2026
max-krasnyansky
approved these changes
Aug 30, 2026
ilmmatias
pushed a commit
to ilmmatias/llama.cpp
that referenced
this pull request
Sep 1, 2026
…nd PP performance (ggml-org#26438) * opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse * opencl: Q4_K mul_mm 8x8 tile fot Intel * opencl: Q5_K mul_mm 8x8 tile for Intel * opencl: Q4_K mul_mv N_DST 8->16 for Intel
wanghqc
added a commit
to qualcomm/llama.cpp
that referenced
this pull request
Sep 1, 2026
69 commits, four days. Ten touch ggml-opencl or its tests; two of those are our own PRs coming back: 010be96 opencl: tune the quant paths for Intel Xe-LP GPUs (ggml-org#26438) c841aee opencl: use a better matmul path on two Adreno GPU generations (ggml-org#27640) Their content was largely already here, so they reconcile rather than add work -- with one real exception. The Adreno xmem F16xF32 GEMM was still opt-in on this branch while upstream had flipped it default-on; the merge brings that flip, so models with f16 attention weights (gpt-oss-20b-f16) now get the path without an environment variable. Resolutions: * the A7X E031.41 f32 l4_lm bypass: identical code both sides, kept our comment, which carries the measurements the upstream one trims. * mul_mv_q4_k_f32_flat N_DST and its dispatch: ours is a superset (the ndst8 small-output variant and the Q4K_N_DST_OVERRIDE hook). * test-backend-ops FLASH_ATTN kv sweep: took upstream's, which adds 32768 and 65536. * common/speculative.cpp: upstream refactored the DFlash path to a fused decode that encodes and injects in one step. Kept ours, which keeps the separate encode with M-RoPE position rows and the per-stage timing. This is a real divergence in spec-decode semantics on the branch customers test, not a formatting difference, so adopting the fusion is a deliberate follow-up rather than a merge decision.
1 task
thecodacus
pushed a commit
to thecodacus/llama.cpp
that referenced
this pull request
Sep 7, 2026
…nd PP performance (ggml-org#26438) * opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse * opencl: Q4_K mul_mm 8x8 tile fot Intel * opencl: Q5_K mul_mm 8x8 tile for Intel * opencl: Q4_K mul_mv N_DST 8->16 for Intel
zbrad
pushed a commit
to zbrad/llama.cpp
that referenced
this pull request
Sep 10, 2026
…nd PP performance (ggml-org#26438) * opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse * opencl: Q4_K mul_mm 8x8 tile fot Intel * opencl: Q5_K mul_mm 8x8 tile for Intel * opencl: Q4_K mul_mv N_DST 8->16 for Intel
pl752
pushed a commit
to pl752/llama.cpp
that referenced
this pull request
Sep 15, 2026
…nd PP performance (ggml-org#26438) * opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse * opencl: Q4_K mul_mm 8x8 tile fot Intel * opencl: Q5_K mul_mm 8x8 tile for Intel * opencl: Q4_K mul_mv N_DST 8->16 for Intel
zsogitbe
pushed a commit
to zsogitbe/llama.cpp
that referenced
this pull request
Sep 17, 2026
…nd PP performance (ggml-org#26438) * opencl: Q4_K/Q5_K mul_mv N_DST 4->8 on Intel for 2x activation reuse * opencl: Q4_K mul_mm 8x8 tile fot Intel * opencl: Q5_K mul_mm 8x8 tile for Intel * opencl: Q4_K mul_mv N_DST 8->16 for Intel
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR contains various performance tuning optimization for some Intel NEO iGPU
gpu_family == INTEL) .Additional information
Validated (2026-08-01): dell / Intel NEO
MUL_MAT+GET_ROWS933 OK / 0 FAIL, 2/2 backends passed on the raise ref.GGML_OPENCL_USE_ADRENO_KERNELS=OFFRequirements
Only for Intel GPUs.