[WebGPU] Webgpu im2col fused activation - #32185
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Ananya Anand (4n4ny4)
changed the title
15a2a80 to
7a1125d
Compare
Im2ColMatMulProgram previously refused any fused activation: CanApplyIm2Col- MatMulProgram returned false whenever the Conv carried one, behind a `// TODO: Support fuse`. Fused NHWC fp16 convolutions therefore fell through to conv2d_mm, grouped_conv or matmul even where im2col was otherwise the better kernel. This adds an activation epilogue to im2col_matmul.wgsl.template covering the same six kinds the other WebGPU Conv kernels support -- Relu, Sigmoid, Clip, HardSigmoid, LeakyRelu, Tanh -- and lifts the guard. The epilogue is only tractable because activation parameters are now uniforms. The template dispatches on a single `activation_kind` int, so one branch per kind is enough; had the parameter values still been baked into the shader text, every distinct alpha or clip bound would have needed its own generated variant. This therefore depends on the uniforms refactor and must land after it. Parameters are read as uniforms.activation_param_0/1, which exist because the program picked up WEBGPU_PROGRAM_ACTIVATION_UNIFORM_VARIABLES and calls AppendActivationUniformsData. Slot usage matches GetActivationUsedUniformCount exactly: none for Relu/Sigmoid/Tanh, one for LeakyRelu, two for Clip and HardSigmoid. Two guards keep the template and the C++ enum from drifting apart. static_asserts in im2col_matmul.cc pin each ActivationKind to the numeric value the template tests, and IsActivationSupported enumerates the kinds explicitly so a newly added enumerator falls to `default: return false` and disables the path rather than silently generating no epilogue. CanApplyIm2ColMatMulProgram now takes the Activation rather than a bool, at both call sites in conv.cc. Keeping ComputeInternal and PrePackInternal in agreement matters: they must reach the same conclusion or prepacked weights are produced for a path that will not read them. Reachability is narrow, and no available hardware can execute it. See the pull request description for the five gates and for what is and is not verified. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit d1a4255)
7a1125d to
a3db355
Compare
|
|
|
I tried both. Patched IsDeviceSupported() locally to accept nvidia and all four tests pass on a TITAN V and the profiler shows Im2ColMatMul being picked rather than a fallback. For perf I converted ResNet-50 to fp16 first, since im2col needs fp16 and a kernel bigger than 1x1. That got 68 of 80 Conv2dMM dispatches onto im2col, the 17 non-1x1 convs over 4 iterations, and it came out about 40% slower: 7.75 ms against 5.52 ms over 4 interleaved runs, with all the diffs within 0.08 ms of each other. EfficientNet-B0 had zero qualifying dispatches even at fp16 since it is all depthwise and 1x1, so ResNet-50 was the only model that told me anything. So no benefit on NVIDIA and I'm leaving the Intel-only gate as is. |
|
In your change description, please document the observed perf win
|
Description
Im2ColMatMulProgramrefused any fused activation, so fused NHWC fp16 convs fell through to another kernel even where im2col was the better choice. This adds an activation epilogue toim2col_matmul.wgsl.templatefor the same six kinds the other Conv kernels support (Relu, Sigmoid, Clip, HardSigmoid, LeakyRelu, Tanh) and removes the// TODO: Support fuseguard.Depends on #32116 and should land after it. The template reads parameters from
uniforms.activation_param_0/1, which only exist because of the uniforms refactor there. With values still baked into shader text, every distinct alpha or clip bound would have needed its own generated variant, which is why the TODO was there.This path has not been executed anywhere.
IsDeviceSupported()requires vendorintelplus architecturexe-2lpg,xe-2hpg,xe-3lpgorxe-3lpg-xs(Lunar Lake, Battlemage, Panther Lake). No ORT CI agent has one and neither did any machine I had while writing this, so the parity tests skip everywhere and this code has never run. Please weigh it as unexecuted.Reachability is narrow, all five required: qualifying adapter, fp16 only, channels last, group 1, non 1x1 kernel.
Motivation and Context
Every other WebGPU Conv kernel already fuses activations. im2col was the only one that did not.
Four parity tests cover Relu, LeakyRelu, HardSigmoid and Clip. The Clip one is new. Clip is the only two slot activation whose slots mean
{min, max}instead of{alpha, beta}, so it is the one case where mis indexingactivation_param_0/1would still pass the HardSigmoid test. It skips like the others but belongs in the file so it runs as soon as someone with the right hardware builds this.static_asserts pin eachActivationKindto the value the template checks, so reordering the enum breaks the build, andIsActivationSupportedfalls todefault: return falsefor a new enumerator rather than emitting no epilogue.Goldens regenerated with
UPDATE_WGSL_GOLDEN=1 python wgsl_template/test/run_tests.py.generated/math/subgroup_matrix_*.hare still missing because they belong to #32115, so the smoke test reports a file set difference for those three and no content mismatch.