[CUDA] Deprecate SkipLayerNorm strict mode - #29388
Merged
kunal-vaishnavi merged 2 commits intoJun 30, 2026
Merged
Conversation
tianleiwu
requested review from
Copilot,
kunal-vaishnavi and
yuslepukhin
and removed request for
yuslepukhin
June 29, 2026 07:42
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the deprecated CUDA SkipLayerNorm “strict mode” execution path and the associated provider-option plumbing, since SkipLayerNorm now always uses fp32 accumulation (making strict mode redundant). It also updates tests to stop running separate strict-mode passes and marks the option as deprecated/ignored for backward compatibility.
Changes:
- Remove strict-mode branching in the CUDA SkipLayerNorm kernel and delete the
strict_member/config accessors. - Remove the CUDA plugin adapter shim used to read strict-mode config.
- Update CUDA provider options documentation to indicate the strict-mode option is deprecated, and simplify tests to a single path.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/contrib_ops/cuda/bert/skip_layer_norm.cc | Removes strict-mode routing; always launches SkipLayerNorm kernel. |
| onnxruntime/contrib_ops/cuda/bert/skip_layer_norm.h | Drops the strict_ member from the kernel class. |
| onnxruntime/core/providers/cuda/cuda_execution_provider.h | Removes IsSkipLayerNormInStrictMode() accessor. |
| onnxruntime/core/providers/cuda/plugin/cuda_kernel_adapter.h | Removes the plugin shim accessor for strict-mode. |
| include/onnxruntime/core/providers/cuda/cuda_provider_options.h | Marks enable_skip_layer_norm_strict_mode as deprecated/ignored for back-compat. |
| onnxruntime/test/contrib_ops/skiplayernorm_op_test.cc | Removes strict-mode test runs; keeps only default path execution. |
kunal-vaishnavi
approved these changes
Jun 30, 2026
kunal-vaishnavi
left a comment
Contributor
There was a problem hiding this comment.
We should also remove it from here.
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Removes the deprecated
enable_skip_layer_norm_strict_modeCUDA provider option and its associated "strict mode" code path from the CUDASkipLayerNormkernel.Strict mode previously routed
SkipLayerNormthrough theLayerNormalizationkernel to gain fp32-accumulation accuracy at the cost of performance. After #28682 madeSkipLayerNorm/EmbedLayerNormCUDA kernels always accumulate in fp32, the strict-mode path is redundant: the default kernel already provides the same accuracy with better performance. This change deletes the now-dead branch and the plumbing that fed it.Key Changes
skip_layer_norm.cc: Remove thestrict_branch that calledHostApplyLayerNorm; always launchLaunchSkipLayerNormKernel. Drop the now-unusedlayer_norm_impl.hinclude and the input/skip same-shape strict-mode shape check.skip_layer_norm.h: Remove thestrict_member.cuda_execution_provider.h: RemoveIsSkipLayerNormInStrictMode().cuda_kernel_adapter.h: RemoveGetCudaKernelAdapterSkipLayerNormStrictMode()shim.cuda_provider_options.h: Keepenable_skip_layer_norm_strict_modefield for ABI/back-compat but mark it deprecated and ignored.skiplayernorm_op_test.cc: Drop the redundant strict-mode test passes; tests now run a single default path.The provider option is retained (ignored) to preserve backward compatibility — existing configs that set it continue to work without error.
Motivation
Follow-up cleanup to #28682, which switched the CUDA kernels to fp32 accumulation, making strict mode obsolete.
Testing
skiplayernorm_op_test.cccovers fp16/fp32/bf16 default path; strict-mode passes removed.