Skip to content

implement gridsample 20#17744

Merged
liqunfu merged 24 commits intomainfrom
liqun/gridsample20
Nov 7, 2023
Merged

implement gridsample 20#17744
liqunfu merged 24 commits intomainfrom
liqun/gridsample20

Conversation

@liqunfu
Copy link
Copy Markdown
Contributor

@liqunfu liqunfu commented Sep 29, 2023

Description

gridsample has been update in ONNX 20. update cpu provider to it

Motivation and Context

prepare for ORT 1.17.0

Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Copy link
Copy Markdown
Contributor

@github-advanced-security github-advanced-security AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lintrunner found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
@liqunfu liqunfu added this to the 1.17 milestone Oct 5, 2023
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Comment thread onnxruntime/core/providers/cpu/tensor/grid_sample.cc Outdated
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Comment thread onnxruntime/test/providers/cpu/tensor/grid_sample_test_gen.py
Comment thread onnxruntime/core/providers/cpu/tensor/grid_sample.h Outdated
Comment thread onnxruntime/core/providers/cpu/tensor/grid_sample.h
Comment thread onnxruntime/core/providers/cpu/tensor/grid_sample.cc
Signed-off-by: Liqun Fu <liqfu@microsoft.com>
Comment thread onnxruntime/test/providers/cpu/tensor/grid_sample_test.cc
@liqunfu liqunfu merged commit 6127dd1 into main Nov 7, 2023
@liqunfu liqunfu deleted the liqun/gridsample20 branch November 7, 2023 18:42
@snnn
Copy link
Copy Markdown
Contributor

snnn commented Mar 11, 2024

#17600

@ZelboK
Copy link
Copy Markdown

ZelboK commented Mar 12, 2024

Hi,

Sorry to barge in. I was looking into #18313 and was confused as the CPU implementation seemed complete, and this is how I ended up here.

IIUC only the CPU implementation is complete? Could I try to contribute towards the CUDA implementation? Just looking to contribute and learn. contrib_defs.cc currently only allows for 4D atm, should that not be updated yet even if the CPU impl is ready?

kleiti pushed a commit to kleiti/onnxruntime that referenced this pull request Mar 22, 2024
@tianlinzx
Copy link
Copy Markdown

@liqunfu Hi ,Do you have any plan to implement a CUDA version ?

@juntaosun
Copy link
Copy Markdown

After torch.nn.functional.grid_sample is exported to onnx, it is very slow compared to torch. Are there plans to improve the performance of grid_sample in onnxruntime-gpu?

@fedral
Copy link
Copy Markdown

fedral commented Sep 20, 2024

After torch.nn.functional.grid_sample is exported to onnx, it is very slow compared to torch. Are there plans to improve the performance of grid_sample in onnxruntime-gpu?

Agreed.
On onnxruntime 1.17.0 +cuda11.8+opset 20 , grid_sample 1080p output takes 70 ms with CPU, while GPU is much slow than CPU mode, around 140ms doubled. Compared with torch implementation, inference only takes 0.01ms.

tianleiwu pushed a commit that referenced this pull request Mar 13, 2026
…operator (#27201)

### Description
1. Supports volumetric input grid sampling in the CUDA EP `GridSample`
operator (i.e.) 5-D input tensor a.k.a 3-D spatial data
2. Registers the CUDA `GridSample` operator for opsets 20 and 22
3. Supports both NCHW and NHWC layouts for volumetric inputs
4. Does not support `cubic` mode for volumetric inputs for now and this
is consistent with the CPU version of the implementation and hence will
not cause "functional regression" (i.e.) `cubic` mode for 3-D spatial
data is not supported on CPU and CUDA before and after this change. This
is a TODO for the future.
5. There are enough unit tests in `grid_sample_test.cc` to cover the
volumetric input case and this is run in both NCHW (NCDHW for volumetric
case) and NHWC (NDHWC for volumetric case) layouts for the CUDA EP

### Motivation and Context
Resolve #21382
Resolve #18942
Resolve #16581
Resolve #18313

Related CPU PRs (for opset 20 and opset 22):
#17744 &&
#23344
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants