Skip to content

[CUDA] Deprecate SkipLayerNorm strict mode - #29388

Merged
kunal-vaishnavi merged 2 commits into
microsoft:mainfrom
tianleiwu:tlwu/remove_sln_strict_mode
Jun 30, 2026
Merged

[CUDA] Deprecate SkipLayerNorm strict mode#29388
kunal-vaishnavi merged 2 commits into
microsoft:mainfrom
tianleiwu:tlwu/remove_sln_strict_mode

Conversation

@tianleiwu

Copy link
Copy Markdown
Contributor

Description

Removes the deprecated enable_skip_layer_norm_strict_mode CUDA provider option and its associated "strict mode" code path from the CUDA SkipLayerNorm kernel.

Strict mode previously routed SkipLayerNorm through the LayerNormalization kernel to gain fp32-accumulation accuracy at the cost of performance. After #28682 made SkipLayerNorm/EmbedLayerNorm CUDA kernels always accumulate in fp32, the strict-mode path is redundant: the default kernel already provides the same accuracy with better performance. This change deletes the now-dead branch and the plumbing that fed it.

Key Changes

  • skip_layer_norm.cc: Remove the strict_ branch that called HostApplyLayerNorm; always launch LaunchSkipLayerNormKernel. Drop the now-unused layer_norm_impl.h include and the input/skip same-shape strict-mode shape check.
  • skip_layer_norm.h: Remove the strict_ member.
  • cuda_execution_provider.h: Remove IsSkipLayerNormInStrictMode().
  • cuda_kernel_adapter.h: Remove GetCudaKernelAdapterSkipLayerNormStrictMode() shim.
  • cuda_provider_options.h: Keep enable_skip_layer_norm_strict_mode field for ABI/back-compat but mark it deprecated and ignored.
  • skiplayernorm_op_test.cc: Drop the redundant strict-mode test passes; tests now run a single default path.

The provider option is retained (ignored) to preserve backward compatibility — existing configs that set it continue to work without error.

Motivation

Follow-up cleanup to #28682, which switched the CUDA kernels to fp32 accumulation, making strict mode obsolete.

Testing

  • skiplayernorm_op_test.cc covers fp16/fp32/bf16 default path; strict-mode passes removed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the deprecated CUDA SkipLayerNorm “strict mode” execution path and the associated provider-option plumbing, since SkipLayerNorm now always uses fp32 accumulation (making strict mode redundant). It also updates tests to stop running separate strict-mode passes and marks the option as deprecated/ignored for backward compatibility.

Changes:

  • Remove strict-mode branching in the CUDA SkipLayerNorm kernel and delete the strict_ member/config accessors.
  • Remove the CUDA plugin adapter shim used to read strict-mode config.
  • Update CUDA provider options documentation to indicate the strict-mode option is deprecated, and simplify tests to a single path.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
onnxruntime/contrib_ops/cuda/bert/skip_layer_norm.cc Removes strict-mode routing; always launches SkipLayerNorm kernel.
onnxruntime/contrib_ops/cuda/bert/skip_layer_norm.h Drops the strict_ member from the kernel class.
onnxruntime/core/providers/cuda/cuda_execution_provider.h Removes IsSkipLayerNormInStrictMode() accessor.
onnxruntime/core/providers/cuda/plugin/cuda_kernel_adapter.h Removes the plugin shim accessor for strict-mode.
include/onnxruntime/core/providers/cuda/cuda_provider_options.h Marks enable_skip_layer_norm_strict_mode as deprecated/ignored for back-compat.
onnxruntime/test/contrib_ops/skiplayernorm_op_test.cc Removes strict-mode test runs; keeps only default path execution.

Comment thread include/onnxruntime/core/providers/cuda/cuda_provider_options.h Outdated
Comment thread onnxruntime/test/contrib_ops/skiplayernorm_op_test.cc
Comment thread onnxruntime/core/providers/cuda/plugin/cuda_kernel_adapter.h

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

@tianleiwu tianleiwu changed the title Remove deprecated SkipLayerNorm CUDA strict mode [CUDA] Deprecate SkipLayerNorm strict mode Jun 29, 2026

@kunal-vaishnavi kunal-vaishnavi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also remove it from here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants