webgpu: optimize Gemm and MatMul using subgroup feature

[xhcao]

Description

Motivation and Context

[xhcao]

@guschmue @fs-eire @qjia7 @jchen10 @Jiawei-Shao
Hi, all, I want to discuss with you whether we could optimize Gemm, MatMul and Conv operators on some special vendor and special architectures as this PR.
Reasons:

1. It is difficult to design an algorithm that benifits all vendors and all architectures.
2. Even for the same vendor, but different architectures, it is also difficult.
3. Maintaining and reviewing the code is also difficult if adding some vendor and architecture information in common files.
4. There are not enough devices to test the correctness and performance.
   ...

[jchen10]

@xhcao It seems you didn't include your test cases in this PR. What's your concern? BTW, you'd better provide some performance data for reference!

[Copilot]

Pull Request Overview

This PR adds Intel-specific optimizations for GEMM and MatMul operations in the WebGPU provider, targeting Intel Xe-2 GPU architectures (xe-2lpg and xe-2hpg) using subgroup operations.

• Implements subgroup-based GEMM and MatMul kernels optimized for Intel GPUs
• Adds vendor-specific code paths that are conditionally executed based on adapter info and matrix dimensions
• Integrates Intel optimizations into existing GEMM and MatMul operators

Reviewed Changes

Copilot reviewed 2 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
matmul_intel.h	Defines MatMulSubgroupProgram class and Intel MatMul entry points
matmul_intel.cc	Implements Intel-optimized MatMul with subgroup support and batched optimizations
gemm_utils_intel.h	Declares helper functions for reading/writing matrix data in Intel kernels
gemm_utils_intel.cc	Implements shader generation helpers for Intel-specific GEMM operations
gemm_subgroup.h	Defines constants and functions for subgroup-based matrix operations
gemm_subgroup.cc	Implements vec4 and scalar variants of subgroup matrix multiplication
gemm_intel.h	Defines GemmSubgroupProgram class and Intel GEMM entry points
gemm_intel.cc	Implements Intel-optimized GEMM with subgroup support
matmul.cc	Integrates Intel MatMul optimization check into existing MatMul operator
gemm.cc	Integrates Intel GEMM optimization check into existing GEMM operator

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[xhcao]

@xhcao It seems you didn't include your test cases in this PR. What's your concern? BTW, you'd better provide some performance data for reference!

Firstly, I want being consented to create a vendor directory and make some optimizations on some special platforms, especially from Microsoft reviewers' consent.
If so, I will add the test cases in the other PR and apply performance data here.

[jchen10]

@xhcao It seems you didn't include your test cases in this PR. What's your concern? BTW, you'd better provide some performance data for reference!

Firstly, I want being consented to create a vendor directory and make some optimizations on some special platforms, especially from Microsoft reviewers' consent. If so, I will add the test cases in the other PR and apply performance data here.

You don't need to depend on it. You can just best improve your PR with more test cases, better perf data, and easier to review.

[fs-eire]

could you please help to merge to latest main and push?

The pipeline failures should be unrelated, and may be fixed by rerun.

[guschmue]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[guschmue]

I understand the desire for a vendor directory.
But I'm concerned that we'll have trouble with testing and CI if we have too many vendor specific paths - it will get fragmented and very hard to test all code path variations.
We already head this problem with subgroup.matrixmultiply where I could not test a change for the intel specific code path so I had to let it fall back to different path.

But we can try and see how that works, just need to be careful with the testing

[jchen10]

@guschmue Thank you for your understanding and support. We will fully test our changes and closely monitor the status on LNL/BMG. Once mature enough we could roll out to more devices.

[fs-eire]

The code is generally good to me. But I think there needs a few changes for code maintenance consideration:

1. we can keep the directory structure (onnxruntime/core/providers/webgpu/vendor/intel), but there is no need to add _intel suffix for files since it is already in the file's full path (folder name)
2. Use a sub-namespace for intel instead of including them in function/class name (there are exceptions, see 3), for example:

• use namespace like this:

  namespace onnxruntime {
  namespace webgpu {
  namespace intel {
      ...

• MatMulReadFnSourceIntel(...) --> intel::MatMulReadFnSource(...)

3. For CanApplyXXXIntel or ApplyXXXIntel, it is OK to keep it as-is, because this specific functions are considered as "entry" to vendor specific code.

[qjia7]

Haven't closely look at all code. But several high level comments.

1. Please use workgroup_idx instead workgroup_id.x/y/z to recalculate the right offset.
2. Please use the help function .setByOffset/getByOffset/setByIndices/getByindices to load/store data.
3. Does your shader support any size of subgroup? I don't see sg_size related checking in your shader. I suppose intel's subgroup size range 8-32.
4. Please use template to write the shader if possible.

[xhcao]

@fs-eire Had addressed your comments. @qjia7 Except for the fourth comment (Because I will continue to optimize the PR, and address the comment in future), all the comments have been addressed.
PTAL, thank you.

[xhcao]

Attach shder files

gemm_subgroup_shader.txt
gemm_subgroup_vec4_shader.txt

[xhcao]

@qjia7 @fs-eire PTAL, thanks.

[qjia7]

Will review the shader part tomorrow.

[xhcao]

@qjia7 I had addressed your comments. @fs-eire @guschmue Please take a look, thanks.

[xhcao]

@qjia7 I tested the PR using ~20 models on Rocket-Lake, several models got performance improvement, but was less than Lunar-Lake.
There was no regression on any models, so applied the PR to all intel devices support subgroup feature. Please take a look again.

[qjia7]

The shader changes look good to me. I’ll hand off the remaining structural review to @fs-eire and @guschmue. Thanks!

[xhcao]

@fs-eire @guschmue PTAL

[xhcao]

The performance data of some models is shown as below on Lunar-lake.

<style> </style>

Model	Performance
depth-anything-base-fp16-pow-fp32	10%
depth-anything-base-fp32	15%
florence-2-base-decoder-fp16	15%
florence-2-base-encoder-fp16	35%
florence-2-base-vision-encoder-fp16-0.5-fp32	15%
jina-clip-v1-version	10%
moondream2-vision-encoder-fp16	25%
sam-vit-b-encoder-fp16-demo	20%
sd-turbo-safety-checker-int32-reduceSum	25%
sd-turbo-unet-fp16-demo-layernorm	10%
whisper-base-encoder-lm-fp16-layernorm	20%

[guschmue]

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).