fix: out of bounds access for resize operation

[lukas-folle-snkeos]

Description

This PR fixes:

• An out-of-bounds write in CUDA Resize for LINEAR mode when running trilinear paths (3D/5D)
• A race condition for the reduction kernel

Root cause

1. The temporary dims-mapping buffer for LINEAR mode was sized using only H+W, while the trilinear coordinate mapping kernel writes D+H+W entries.
2. shared-memory race in the block-level reduction loop inside reduction_functions.cu. The condition allowed threads outside the active lower half to update shared memory in the same stride phase, creating overlapping read/write hazards

My colleague @korbinian-mechlem-snkeos noticed this warning from compute-sanitzer

========= Invalid global write of size 4 bytes
========= at void onnxruntime::cuda::_ResizeTrilinearCoordinateMapping<float, onnxruntime::cuda::TransformCoordinate_HALF_PIXEL>(long long, long long, long long, long long, long long, long long, float, float, float, float, float, float, float, float, float, unsigned long long, bool, const T2 &, onnxruntime::cuda::LinearMappingInfo *)+0x400
========= by thread (17,0,0) in block (2,0,0)
========= Address 0xb28fff7cc is out of bounds
========= and is 205 bytes after the nearest allocation at 0xb28fff400 of size 768 bytes
========= Saved host backtrace up to driver entry point at kernel launch time

AND

========= Warning: Race reported between Read access at void onnxruntime::cuda::detail::reduce_matrix_columns_kernel<float, float, float, onnxruntime::cuda::Identity, onnxruntime::cuda::Identity, (bool)0>(int, int, const T1 *, T2 *, T3 *, int *)+0xe80
========= and Write access at void onnxruntime::cuda::detail::reduce_matrix_columns_kernel<float, float, float, onnxruntime::cuda::Identity, onnxruntime::cuda::Identity, (bool)0>(int, int, const T1 *, T2 *, T3 *, int *)+0xea0 [337920 hazards]

Motivation and Context

Update LINEAR buffer size calculation to:

• use H+W for bilinear (2D/4D)
• use D+H+W for trilinear (3D/5D)

Prevents invalid global writes and intermittent CUDA memory errors in trilinear resize workloads.

@johannes-rehm-snkeos

[lukas-folle-snkeos]

@microsoft-github-policy-service agree

[xadupre]

A unit test would be good.

[lukas-folle-snkeos]

@xadupre our internal testing showed that the onnxruntime memory context corruption did indeed disappear with those two fixes in place.

[Copilot]

Pull request overview

Fixes two CUDA correctness issues identified via compute-sanitizer: an out-of-bounds write in Resize LINEAR trilinear coordinate mapping (3D/5D paths) and a shared-memory race in the reduction block-level reduction loop.

Changes:

• Adjust CUDA Resize LINEAR temp buffer sizing to account for trilinear mapping requiring D+H+W entries.
• Fix shared-memory reduction loop condition to avoid overlapping reads/writes within a reduction stride.
• Add regression tests covering repeated column-reduction execution and a CUDA-only 5D trilinear Resize case.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
onnxruntime/test/providers/cuda/test_cases/reduction_functions_test.cc	Adds a repeated-run test to help catch intermittent reduction race issues.
onnxruntime/test/providers/cpu/tensor/resize_op_test.cc	Adds a CUDA-only trilinear Resize regression test exercising the 5D linear path.
onnxruntime/core/providers/cuda/tensor/resize_impl.cu	Fixes LINEAR mapping buffer size calculation for trilinear (3D/5D) to prevent OOB writes.
onnxruntime/core/providers/cuda/reduction/reduction_functions.cu	Fixes block-level reduction condition to prevent shared-memory race hazards.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tianleiwu]

/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).