Skip to content

[SYCL] support the missed types in cpy - #26005

Merged
ggerganov merged 3 commits into
ggml-org:masterfrom
arthw:support_cpy
Jul 31, 2026
Merged

[SYCL] support the missed types in cpy#26005
ggerganov merged 3 commits into
ggml-org:masterfrom
arthw:support_cpy

Conversation

@arthw

@arthw arthw commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Overview

The UT will crash after the new cpy UT cases with new types are added.

Additional information

This PR support to the missed data types of the UT cases of CPY OP.
All related cases are passed.

Requirements

@arthw
arthw requested a review from a team as a code owner July 22, 2026 15:26
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Jul 22, 2026
@arthw
arthw marked this pull request as draft July 22, 2026 15:53
@arthw
arthw marked this pull request as ready for review July 28, 2026 13:29
@arthw arthw added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Jul 30, 2026
ggerganov
ggerganov reviewed Jul 31, 2026
View reviewed changes
Comment thread ggml/src/ggml-sycl/cpy.cpp Outdated
#include "cpy.hpp"

#include <float.h>
#include <atomic>

@ggerganov ggerganov Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this header actually used?

@NeoZhangJianyu NeoZhangJianyu Jul 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, they are not used in fact.
Rm them.

Thank you!

@ggerganov
ggerganov merged commit d5d3e05 into ggml-org:master Jul 31, 2026
22 of 29 checks passed
huaxel pushed a commit to huaxel/CachyLLama that referenced this pull request Aug 2, 2026
@arthw @huaxel
[SYCL] support the missed types in cpy (ggml-org#26005)
2f5a861 
* support the missed types in cpy

* use correct funct

* rm unused code
satindergrewal pushed a commit to satindergrewal/llama.cpp that referenced this pull request Aug 12, 2026
@arthw
[SYCL] support the missed types in cpy (ggml-org#26005)
2711c05 
* support the missed types in cpy

* use correct funct

* rm unused code
mndodd added a commit to mndodd/llama.cpp that referenced this pull request Aug 12, 2026
@mndodd
Merge origin/master (876a432, b10216) into sycl-sync-0801
e98815f 
43 upstream commits, 15 of them in our paths. Four conflicts, resolved as follows.
The dangerous change in this range did NOT conflict -- see (2).

1. ggml/src/ggml-sycl/element_wise.cpp -- TOOK UPSTREAM VERBATIM.
   ggml-org#25946 landed upstream as 11b068d. We had been carrying it as a cherry-pick of
   the then-unmerged PR (ca7c42a) plus two commits of our own stacked on top:
     27d821e  fastdiv for the strided unary index reconstruction
     0595878  fastdiv for the fused-GLU index reconstruction
   Upstream's landed form contains BOTH optimisations by the same mechanism
   (init_fastdiv_values host-side + fast_div_modulo in-kernel, on the strided unary
   path and on all five gated_op_fused_* kernels). All three of ours are therefore
   superseded and are dropped; the file is now byte-identical to origin/master.
   Only semantic difference we give up: ours guarded k > u32 with an exact int64
   fallback, upstream asserts ggml_nelements(dst) < 2^31 instead -- stricter by 2x,
   and unreachable for a GLU activation (~8 GB at f32).
   Our ne>0?ne:1 divisor guard is also dropped; init_fastdiv_values asserts d != 0
   and a ggml tensor always has ne[i] >= 1, so it was defensive, not load-bearing.

2. ggml/src/ggml-sycl/fattn.cpp -- PRECEDENCE PRESERVED, both kernels kept.
   Upstream ggml-org#25025 adds a oneMKL GEMM flash-attention path and gives it
   BEST_FATTN_KERNEL_MKL = 300 -- the value we already use for BEST_FATTN_KERNEL_MMA.
   git flagged the enum collision. It did NOT flag the consequential half: upstream
   places the MKL gate ABOVE our MMA check, and that hunk auto-merged clean.
   MKL's gate is default-ON (GGML_SYCL_ENABLE_MKL_FA=1) and its envelope -- gqa_ratio
   >= 2, head_dim % 64 in [64,512], Q->ne[1] >= 32, K->ne[1] >= 1024, no sinks /
   ALiBi / softcap, with a quantized K/V SKIPPING the F16 stride test -- matches our
   deploy prefill exactly. Taken verbatim it would have silently replaced the
   measured MMA kernel with an unmeasured one, staged the whole q8_0 KV cache to F16
   first, and (per upstream's own note) broken SYCL graph capture replay.
   Resolution: MKL renumbered to 400 so both kernels stay reachable, and its gate
   takes an added !ggml_sycl_fattn_mma_supported(dst) conjunct. MMA wins where MMA
   is supported; MKL keeps its FULL envelope for everything MMA declines, which is
   upstream's intent in every case that is not ours. This is a precedence choice,
   not a revert -- and it is A/B-able without a rebuild:
     GGML_SYCL_FATTN_MMA=0     -> MMA declines, MKL takes the path
     GGML_SYCL_ENABLE_MKL_FA=0 -> MKL off entirely
   Also merged both sides' env-gated instruments, hoisting the kernel selection to a
   single call: upstream re-derived it three times (watchdog, switch, fingerprint),
   so an instrument could disagree with what actually ran. All three now read one
   hoisted `k`. Fixed a latent lie in our own FATTN_DEBUG printer while there --
   BEST_FATTN_KERNEL_ONEDNN was printing as "NONE"; ONEDNN and MKL now print.

3. ggml/src/ggml-sycl/cpy.cpp -- kept ours. Upstream's side of the hunk was empty;
   ggml-org#26005 touched adjacent lines. Our GGML_SYCL_CPY_CENSUS instrument is unchanged.

4. tests/test-backend-ops.cpp -- kept both sides, additive and disjoint (same
   resolution as the 07-28 sync). Ours = the finding-85 deployed-shape MUL_MAT
   sweep; upstream's = m==1 either side of MMVF_MAX_BATCH_SIZE.

Gates run before this commit:
  - cmake-option-audit.sh e9fa078 origin/master -> 7/7 watched options unchanged,
    and --selftest fires (rc=3), so the check is proven able to go red.
  - post-configure asserts: GGML_SYCL / _F16 / _DNN / _GRAPH all ON.

NOT done in this commit, and required before any number from this tree is
comparable to a pre-sync one: rebuild + re-baseline. Upstream changed code under
every arm; ratios within one arm survive, absolutes do not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ggerganov ggerganov ggerganov left review comments
+1 more reviewer
@NeoZhangJianyu NeoZhangJianyu NeoZhangJianyu left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ggml changes relating to the ggml tensor library for machine learning merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@arthw @ggerganov @NeoZhangJianyu