Skip to content

webgpu: use naive reduction - #28174

Merged
guschmue merged 5 commits into
microsoft:mainfrom
xhcao:use-naive-reduction
Jun 16, 2026
Merged

webgpu: use naive reduction#28174
guschmue merged 5 commits into
microsoft:mainfrom
xhcao:use-naive-reduction

Conversation

@xhcao

@xhcao xhcao commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

Use naive reduction when the output size of ReduceMean is far greater than reduce size. Shared reduction method may need to transpose input, which costs much time.

Description

Motivation and Context

Use naive reduction when the output size of ReduceMean is far greater
than reduce size. Shared reduction method may need to transpose input,
which costs much time.
@xhcao

xhcao commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

When I run florence-2-base-vision-encoder-fp16-0.5-fp32 with input [batch_size:1, height:768, width:768], 10 ReduceMean nodes using shared reduction method cost much time,
ReduceMean|Transpose 6.51 "inputs[0] = {1,128,36864} outputs[0] = {1,36864,128} "
ReduceMean 0.30 "inputs[0] = {1,36864,128} outputs[0] = {1,1,36864} "

It is heavy to transpose the input. If we use the naive method,
ReduceMean 0.11 "inputs[0] = {1,128,36864} outputs[0] = {1,1,36864} "

The model's total time is from ~340ms to ~270ms.

@guschmue guschmue added the ep:WebGPU ort-web webgpu provider label Apr 22, 2026
@guschmue
guschmue requested a review from Copilot April 23, 2026 16:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the WebGPU reduction kernel dispatch heuristic to prefer the naive reduction implementation for certain ReduceMean shapes, aiming to avoid transpose overhead that can dominate runtime when the reduced dimension is small but the output is very large.

Changes:

  • Adjusts the use_naive_reduction selection logic in ReduceKernel::ComputeInternal.
  • Adds a ReduceMean-specific threshold to route some cases to ReduceNaiveProgram.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread onnxruntime/core/providers/webgpu/reduction/reduction_ops.cc Outdated
@xhcao

xhcao commented Apr 24, 2026

Copy link
Copy Markdown
Contributor Author

@jchen10 PTAL

@xhcao

xhcao commented Apr 27, 2026

Copy link
Copy Markdown
Contributor Author

@qjia7 PTAL, thanks

Comment thread onnxruntime/core/providers/webgpu/reduction/reduction_ops.cc Outdated
@xhcao

xhcao commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

@guschmue PTAL

@guschmue
guschmue enabled auto-merge (squash) May 14, 2026 15:27
@xhcao

xhcao commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

@guschmue It seems that the CI failure is not related with this PR, please re-trigger the bots again, thanks.

@xhcao

xhcao commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebase the code.

@xhcao

xhcao commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

@guschmue Please take a look it, thanks.

@jchen10

jchen10 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

@guschmue Just a gentle ping.

@guschmue
guschmue merged commit a534347 into microsoft:main Jun 16, 2026
84 of 85 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ep:WebGPU ort-web webgpu provider

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants