Enforce static dimensions in generation of flow.tensor.transfer #205
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Boian Petkantchin <[email protected]>
sogartar
force-pushed
the
flow-tensor-transfer-enforce-static-dims
branch
from
d23a84e
to
657ec91
Compare
sogartar
changed the title
|
The CI seems to be failing with unrelated errors. |
rsuderman
approved these changes
Oct 8, 2024
sogartar
added a commit
to sogartar/sharktank
that referenced
this pull request
Oct 9, 2024
The fix iree-org/iree-turbine#205 solves the issue with this test. Xfail the Unet Resnet block test with maybe low accuracy.
sogartar
added a commit
to nod-ai/shark-ai
that referenced
this pull request
Oct 9, 2024
The fix iree-org/iree-turbine#205 solves the issue with this test. Xfail the Unet Resnet block test with maybe low accuracy.
stellaraccident
pushed a commit
that referenced
this pull request
Oct 13, 2024
This solves the problem in iree-org/iree#18283 The issue is that we generate cast to/from dynamic tensors that later lowering in IREE chokes on it. My assumption is that it should be able to digest this IR since it is of the form. ```mlir %2 = torch_c.to_builtin_tensor %arg0 : !torch.vtensor<[2,3,11,13],f32> -> tensor<2x3x11x13xf32> %cast = tensor.cast %2 : tensor<2x3x11x13xf32> to tensor<?x?x?x?xf32> %c0 = arith.constant 0 : index %dim = tensor.dim %cast, %c0 : tensor<?x?x?x?xf32> %c1 = arith.constant 1 : index %dim_0 = tensor.dim %cast, %c1 : tensor<?x?x?x?xf32> %c2 = arith.constant 2 : index %dim_1 = tensor.dim %cast, %c2 : tensor<?x?x?x?xf32> %c3 = arith.constant 3 : index %dim_2 = tensor.dim %cast, %c3 : tensor<?x?x?x?xf32> %3 = flow.tensor.transfer %cast : tensor<?x?x?x?xf32>{%dim, %dim_0, %dim_1, %dim_2} to #hal.device.promise<@__device_0> %cast_3 = tensor.cast %3 : tensor<?x?x?x?xf32> to tensor<2x3x11x13xf32> %4 = torch_c.from_builtin_tensor %cast_3 : tensor<2x3x11x13xf32> -> !torch.vtensor<[2,3,11,13],f32> ``` It essentially casts to a dynamic `tensor<...>` for the purpose of performing `flow.tensor.transfer` and then casts back to a static `torch.vtensor`. So it should be fine. With this change we get ```mlir %2 = torch_c.to_builtin_tensor %arg0 : !torch.vtensor<[2,3,11,13],f32> -> tensor<2x3x11x13xf32> %3 = flow.tensor.transfer %2 : tensor<2x3x11x13xf32> to #hal.device.promise<@__device_0> %4 = torch_c.from_builtin_tensor %3 : tensor<2x3x11x13xf32> -> !torch.vtensor<[2,3,11,13],f32> ``` Signed-off-by: Boian Petkantchin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This solves the problem in iree-org/iree#18283
The issue is that we generate cast to/from dynamic tensors that later lowering in IREE chocks on it. My assumption is that it should be able to digest this IR since it is of the form.
It essentially cast to a dynamic
tensor<...>for the purpose of performingflow.tensor.transferand then cast back to a statictorch.vtensor. So it should be fine.With this change we get
I am not convinced that specializing all dimensions is correct. What should we do if we want some dynamic dimensions? How should this be represented?