Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
763c40f
SYCL: add oneMKL GEMM flash attention for XMX-accelerated prompt proc…
johnkarlhill Jun 26, 2026
ab0048e
fattn-mkl: fix interleaved dst layout in normalize kernel
johnkarlhill Jun 28, 2026
5a81c11
Thank you for the review feedback: rename env vars, use GGML_LOG_INFO…
johnkarlhill Jun 29, 2026
bc757b1
Thank you for the review feedback round 2: use ggml_sycl_get_env, rem…
johnkarlhill Jun 29, 2026
b5b58ae
fattn-mkl: enable MKL FA for all KV cache types
johnkarlhill Jun 29, 2026
e8b207e
fattn-mkl: rename mkl_disable -> mkl_enable for clarity
johnkarlhill Jun 30, 2026
7083be7
fattn-mkl: refine MKL FA dispatch gates
johnkarlhill Jun 30, 2026
13f5032
ggml-sycl: fix F16 cache + MKL FA multi-turn corruption; add gate guards
johnkarlhill Jul 2, 2026
b38dd72
fattn-mkl: fix quant+GQA KV strides, tighten MKL gate, add K>=1024 tests
johnkarlhill Jul 7, 2026
9a799a5
Update ggml/src/ggml-sycl/fattn.cpp
johnkarlhill Jul 8, 2026
1418d54
Update ggml/src/ggml-sycl/fattn.cpp
johnkarlhill Jul 8, 2026
0f31930
Update ggml/src/ggml-sycl/fattn.cpp
johnkarlhill Jul 8, 2026
b149877
fattn-mkl: bound attention scratch so it doesn't grow with batch or c…
johnkarlhill Jul 8, 2026
9ef4732
Update ggml/src/ggml-sycl/fattn-mkl.cpp
johnkarlhill Jul 9, 2026
0808d31
Update ggml/src/ggml-sycl/fattn-mkl.cpp
johnkarlhill Jul 9, 2026
75c6312
apply arthw suggestions: enum for dequant modes, macro for wg_size, e…
johnkarlhill Jul 9, 2026
b43b354
Merge branch 'sycl-mkl-flash-attn' of https://github.com/johnkarlhill…
johnkarlhill Jul 9, 2026
94d23ba
Merge remote-tracking branch 'origin/master' into sycl-mkl-flash-attn
johnkarlhill Jul 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/backend/SYCL.md
Original file line number Diff line number Diff line change
Expand Up @@ -795,6 +795,9 @@ use 1 SYCL GPUs: [0] with Max compute units:512
| GGML_SYCL_USE_LEVEL_ZERO_API | 1 (default) or 0 | Use Level Zero API for device memory allocation instead of SYCL. Reduces system RAM usage on Intel dGPUs by avoiding DMA-buf/TTM host memory staging. Requires GGML_SYCL_SUPPORT_LEVEL_ZERO_API=ON at build time. SYCL backend always runs on Level Zero running time even if it's set as OFF (The SYCL api will be usage for memory allocation).|
| GGML_SYCL_ENABLE_DNN | 0 or 1 (default)| Enable running computations through oneDNN and always use oneMKL. |
| GGML_SYCL_ENABLE_VMM | 0 or 1 (default) | Enable the virtual-memory device pool. |
| GGML_SYCL_ENABLE_MKL_FA | 1 (default) or 0 | Enable oneMKL GEMM flash attention for XMX-accelerated prompt processing with quantized KV cache. Automatically activates during prefill (prompt processing) when all conditions are met: (1) flash-attn enabled (`-fa` or `--flash-attn on`), (2) KV cache quantized (`--cache-type-k q8_0 --cache-type-v q8_0` or other `*_0/*_1` types), (3) batch size ≥ 1024 (`--batch-size 1024`), (4) prompt length ≥ 1024 tokens. Set to 0 to force the TILE kernel for A/B testing. Example minimum command: `llama-cli -m model.gguf -fa -ngl 99 --cache-type-k q8_0 --cache-type-v q8_0 --batch-size 1024 -p "your prompt"` |
| GGML_SYCL_MKL_FA_DEBUG | 0 (default) or 1 | Enable per-call diagnostic logging for MKL flash attention: GEMM/softmax timings, interleaved-head detection, and buffer memory usage. |
| GGML_SYCL_MKL_FA_DIAG | 0 (default) or 1 | Enable output fingerprinting for MKL flash attention. Dumps the first 64 float output values for the first 6 FA calls with n_kv ≥ 1024, labeled with kernel type (MKL/TILE/VEC) for cross-kernel comparison. |
| GGML_SYCL_ENABLE_FUSION | 0 or 1 (default) | Enable fused-kernel dispatch in graph compute (currently top-k MoE gating). |
| ZES_ENABLE_SYSMAN | 0 (default) or 1 | Support to get free memory of GPU by sycl::aspect::ext_intel_free_memory.<br>Recommended to use when --split-mode = layer |
| UR_L0_ENABLE_RELAXED_ALLOCATION_LIMITS | 0 (default) or 1 | Allow SYCL/Unified Runtime Level Zero device allocations larger than 4 GiB. llama.cpp's direct Level Zero allocation path requests the relaxed maximum-size limit itself when GGML_SYCL_ENABLE_LEVEL_ZERO=1. |
Expand Down
Loading
Loading