You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This pull request addresses edge cases in the TensorScatter operator, particularly around handling large or potentially overflowing write_indices values. It also improves test coverage for these scenarios, ensuring robust and correct behavior in both linear and circular modes.
Bug fixes and safety improvements
Fixed a potential overflow bug in linear mode by updating the bounds check to prevent write_indices values that could cause overflow when added to sequence_length. (onnxruntime/core/providers/cpu/llm/tensorscatter.cc)
Refined the circular mode logic to correctly handle very large write_indices values, ensuring correct wraparound behavior without overflow. (onnxruntime/core/providers/cpu/llm/tensorscatter.cc)
Test coverage enhancements
Added a test for linear mode that verifies the operator fails gracefully when write_indices addition would overflow, ensuring the new bounds check is enforced. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)
Added a test for circular mode to confirm that very large write_indices values wrap correctly without overflow, verifying correct wraparound logic. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)
Included <limits> header to support the use of std::numeric_limits<int64_t>::max() in tests. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)
Circular mode now evaluates wi % max_sequence_length before entering the copy loop. A shape with a zero-length cache axis and zero-length update is accepted by the existing sequence_length <= max_sequence_length check. With, for example, cache/update shapes [1, 0, 1], prefix_count is still 1, so this executes 0 % 0, which is undefined behavior.
Previously the zero-length loop performed no modulo operation and returned successfully.
Please return as a no-op when sequence_length == 0, or explicitly reject a zero cache axis before computing wi_mod.
The zero-sequence-length fix now loops over every batch entry even when write_indices is omitted and there is nothing to validate. Zero-volume tensors can have arbitrarily large nonzero dimensions without allocating corresponding data. For example, past_cache and update shaped [INT64_MAX, 0, 1] have zero elements and pass shape validation, but the new early-return path performs INT64_MAX iterations with wi hardcoded to zero.
This turns a zero-byte inference into an effectively unbounded CPU loop. When write_indices == nullptr, please return immediately for sequence_length == 0; retain the validation loop only when caller-provided indices actually need checking.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request addresses edge cases in the
TensorScatteroperator, particularly around handling large or potentially overflowingwrite_indicesvalues. It also improves test coverage for these scenarios, ensuring robust and correct behavior in both linear and circular modes.Bug fixes and safety improvements
write_indicesvalues that could cause overflow when added tosequence_length. (onnxruntime/core/providers/cpu/llm/tensorscatter.cc)write_indicesvalues, ensuring correct wraparound behavior without overflow. (onnxruntime/core/providers/cpu/llm/tensorscatter.cc)Test coverage enhancements
write_indicesaddition would overflow, ensuring the new bounds check is enforced. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)write_indicesvalues wrap correctly without overflow, verifying correct wraparound logic. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)<limits>header to support the use ofstd::numeric_limits<int64_t>::max()in tests. (onnxruntime/test/providers/cpu/llm/tensorscatter_op_test.cc)