Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rewriter] Remove redundant op.Slice and op.ScatterND #1925

Merged

Conversation

titaiwangms
Copy link
Contributor

@titaiwangms titaiwangms commented Oct 30, 2024

Fixes https://github.com/microsoft/onnx-converters-private/issues/270

It is observed that ExportedProgram could generates aten::slice.Tensor and aten::slice_scatter.default that slice nothing:

Screenshot 2024-10-29 112157

The slices would result in redundant op.Slice ops in ONNX graph that does nothing, and op.ScatterND that basically replaces the whole input to updates, which takes a lot of time in inference.

This rule set recognizes the redundant slices by checking if the following requirements are met:
(1) starts = 0
(2) ends >= inputs[dim].shape or ends == _INT64_MAX
(3) steps == 1

This rule set recognizes the redundant scatterND by checking if the following requirements are met:
(1) indices has the same length as the first dim of input
(2) indices is from 0 to input.shape[0]
(3)input has the same shape as updates

Benchmark on ghostnet_100 (the original speed up was 0.0256):

Stat Speedup Increase Med
Suite Model Name onnx_dynamo onnx_dynamo onnx_dynamo
Timm ghostnet_100 1.1599 15.986% 1.1580

Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 74.10072% with 36 lines in your changes missing coverage. Please review.

Project coverage is 75.23%. Comparing base (c13e4fd) to head (7eca5a6).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
onnxscript/rewriter/collapse_slices.py 53.52% 22 Missing and 11 partials ⚠️
onnxscript/rewriter/testing.py 88.88% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main    #1925    +/-   ##
========================================
  Coverage   75.23%   75.23%            
========================================
  Files         254      257     +3     
  Lines       27644    27782   +138     
  Branches     3225     3242    +17     
========================================
+ Hits        20797    20901   +104     
- Misses       5877     5900    +23     
- Partials      970      981    +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@titaiwangms titaiwangms linked an issue Oct 30, 2024 that may be closed by this pull request
onnxscript/rewriter/collapse_slices.py Fixed Show fixed Hide fixed
onnxscript/rewriter/collapse_slices.py Fixed Show fixed Hide fixed
onnxscript/rewriter/collapse_slices.py Fixed Show fixed Hide fixed
onnxscript/rewriter/collapse_slices.py Fixed Show fixed Hide fixed
onnxscript/rewriter/testing.py Fixed Show fixed Hide fixed
onnxscript/rewriter/testing.py Fixed Show fixed Hide fixed
onnxscript/rewriter/testing.py Fixed Show fixed Hide fixed
onnxscript/rewriter/testing.py Fixed Show fixed Hide fixed
onnxscript/rewriter/collapse_slices.py Outdated Show resolved Hide resolved
onnxscript/rewriter/testing.py Outdated Show resolved Hide resolved
onnxscript/rewriter/testing.py Outdated Show resolved Hide resolved
onnxscript/rewriter/collapse_slices_test.py Outdated Show resolved Hide resolved
@titaiwangms titaiwangms enabled auto-merge (squash) November 1, 2024 16:28
@titaiwangms titaiwangms merged commit 1ceb85b into microsoft:main Nov 1, 2024
20 of 41 checks passed
@titaiwangms titaiwangms deleted the titaiwang/add_collapsed_slice_rule branch November 1, 2024 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

[optimizer] Collapse Slices
2 participants