Skip to content

[Integrate] Drop llvm/llvm-project@b4c31dc revert.#21851

Merged
hanhanW merged 12 commits intomainfrom
integrates/llvm-20250904-drop-from-element-revert
Sep 5, 2025
Merged

[Integrate] Drop llvm/llvm-project@b4c31dc revert.#21851
hanhanW merged 12 commits intomainfrom
integrates/llvm-20250904-drop-from-element-revert

Conversation

@hanhanW
Copy link
Contributor

@hanhanW hanhanW commented Sep 4, 2025

It carries a cherry-pick fix that gets the operands from the adaptor:

Changes:

  • Update most lit tests to check vector.from_elements.
  • Add unrolling patterns to the final conversion.
  • Implement n-D vector::ToElementsOp lowering, which will be dropped after [mlir][vector] Add support for lowering n-D vector.to_elements op. llvm/llvm-project#156992 is landed. It should be added to all the backends, but somehow only AMDGPU backend needs the pattern. The other backends may address the issue via specialized tiling config + dropping vector unit dim patterns.

Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
@hanhanW
Copy link
Contributor Author

hanhanW commented Sep 4, 2025

To repro: iree-compile --iree-hal-target-device=hip --iree-hip-target=gfx942 ~/repro.mlir

func.func @attention_2_1024_128_128_64_dtype_f16_f16_f16_f16(%query: tensor<2x1024x128xf16>, %key: tensor<2x128x128xf16>, %value: tensor<2x128x64xf16>, %scale: f32) -> tensor<2x1024x64xf16> {
  %result0 = tensor.empty(): tensor<2x1024x64xf16>
  %scale_f16 = arith.truncf %scale : f32 to f16
  %result1 = iree_linalg_ext.attention {
      indexing_maps = [affine_map<(batch, m, n, k1, k2) -> (batch, m, k1)>,
                       affine_map<(batch, m, n, k1, k2) -> (batch, k2, k1)>,
                       affine_map<(batch, m, n, k1, k2) -> (batch, k2, n)>,
                       affine_map<(batch, m, n, k1, k2) -> ()>,
                       affine_map<(batch, m, n, k1, k2) -> (batch, m, n)>]
}      ins(%query, %key, %value, %scale_f16: tensor<2x1024x128xf16>, tensor<2x128x128xf16>, tensor<2x128x64xf16>, f16)
      outs(%result0: tensor<2x1024x64xf16>) {
   ^bb0(%score: f32):
   iree_linalg_ext.yield %score : f32
 } -> tensor<2x1024x64xf16>
 return %result1: tensor<2x1024x64xf16>
}

Signed-off-by: hanhanW <hanhan0912@gmail.com>
@hanhanW hanhanW changed the title [Integrate] Drop llvm/llvm-project@4eb1a07 revert. [Integrate] Drop llvm/llvm-project@b4c31dc revert. Sep 5, 2025
@hanhanW
Copy link
Contributor Author

hanhanW commented Sep 5, 2025

@kuhar @MaheshRavishankar the CI is green. The cherry-picked commit is landed in upstream. I think we can land the PR, if you can help review the changes.

@hanhanW hanhanW requested a review from kuhar September 5, 2025 01:02
Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: hanhanW <hanhan0912@gmail.com>
@hanhanW hanhanW merged commit f807607 into main Sep 5, 2025
47 checks passed
@hanhanW hanhanW deleted the integrates/llvm-20250904-drop-from-element-revert branch September 5, 2025 03:01
Comment on lines +187 to +188
Value shapeCast = rewriter.create<vector::ShapeCastOp>(
op.getLoc(), vec1DType, op.getSource());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if flattening was the right call, this shouldve been an unrolling pattern. Everything else does unrolling.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's made explicit this is mostly a stopgap patch

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough

hhkit pushed a commit to opencompl/iree that referenced this pull request Sep 11, 2025
It carries a cherry-pick fix that gets the operands from the adaptor:
-
iree-org/llvm-project@8b88014

Changes:
- Update most lit tests to check `vector.from_elements`.
- Add unrolling patterns to the final conversion.
- Implement n-D `vector::ToElementsOp` lowering, which will be dropped
after llvm/llvm-project#156992 is landed. It
should be added to all the backends, but somehow only AMDGPU backend
needs the pattern. The other backends may address the issue via
specialized tiling config + dropping vector unit dim patterns.

---------

Signed-off-by: hanhanW <hanhan0912@gmail.com>
Signed-off-by: Ivan Ho <ivan.ho@cl.cam.ac.uk>
hanhanW added a commit to hanhanW/iree that referenced this pull request Sep 16, 2025
It is a follow-up to iree-org#21851. The
CPU backend also needs the patterns, which is not surprising. It was not
detected in the first place because a test did not exist. It only
happens in dynamic gather+attention cases.

The test is failing on VMVX and SPIR-V backend. AMDGPU does not have a
test suite, but it can compile the program. Thus, the test is only added
to CPU for now.

Signed-off-by: hanhanW <hanhan0912@gmail.com>
hanhanW added a commit that referenced this pull request Sep 17, 2025
…2010)

It is a follow-up to #21851. The
CPU backend also needs the patterns, which is not surprising. It was not
detected in the first place because a test did not exist. It only
happens in dynamic gather+attention cases.

The test is failing on VMVX and SPIR-V backend. Thus, the test is only
added to CPU and hip for now.

Fixes #22007

---------

Signed-off-by: hanhanW <hanhan0912@gmail.com>
bangtianliu pushed a commit to bangtianliu/iree that referenced this pull request Sep 22, 2025
…ee-org#22010)

It is a follow-up to iree-org#21851. The
CPU backend also needs the patterns, which is not surprising. It was not
detected in the first place because a test did not exist. It only
happens in dynamic gather+attention cases.

The test is failing on VMVX and SPIR-V backend. Thus, the test is only
added to CPU and hip for now.

Fixes iree-org#22007

---------

Signed-off-by: hanhanW <hanhan0912@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants