Reapply "[GPU] Allow multi result and indexing compute generic ops in TilleAndFuse pipeline" (#22205)"

[nirvedhmeshram]

Supporting this cases exposed an unrelated issue with broadcasting dims from a producer of the consumer op which is now fixed in this version of the PR. We wont distibute the broadcasted dims as the producer cant be fused in that case.

Fixes : #22204
Fixes : #22175

[qedawkins]

This needs a stronger check that the broadcasted producer is returned from the executable. Then otherwise this looks good.

[pdhirajkumarprasad]

We have regression in compile for llama model due to this change.

command:

iree-compile output.mlir --iree-hip-target=gfx942 -o tt.vmfb --iree-hal-target-device=hip --iree-opt-level=O3 --iree-hal-indirect-command-buffers=true --iree-stream-resource-memory-model=discrete --iree-hip-enable-tensor-ukernels --iree-stream-affinity-solver-max-iterations=1024 --iree-hal-memoization=true --iree-codegen-enable-default-tuning-specs=true --iree-llvmgpu-test-combine-layout-transformation=false

output.mlir.txt

[IanWood1]

@pdhirajkumarprasad which model variant is regressing?

[Groverkss]

We have CI to prevent this.

If this is a llama 8b fp8/fp16 variant, can we first update the IR in the cI here: https://github.com/iree-org/iree/blob/main/tests/external/iree-test-suites/torch_models/llama_8b_fp16/modules/llama_gfx942.json

and

https://github.com/iree-org/iree/blob/main/tests/external/iree-test-suites/torch_models/llama_8b_fp8/modules/llama_gfx942.json

We can then catch these regressions in CI

[MaheshRavishankar]

We have regression in compile for llama model due to this change.

command:

iree-compile output.mlir --iree-hip-target=gfx942 -o tt.vmfb --iree-hal-target-device=hip --iree-opt-level=O3 --iree-hal-indirect-command-buffers=true --iree-stream-resource-memory-model=discrete --iree-hip-enable-tensor-ukernels --iree-stream-affinity-solver-max-iterations=1024 --iree-hal-memoization=true --iree-codegen-enable-default-tuning-specs=true --iree-llvmgpu-test-combine-layout-transformation=false

output.mlir.txt

@pdhirajkumarprasad We have to revert a few changes if we need to revert. A little more information about the failure would help..

@nirvedhmeshram / @IanWood1 can you see if ToM is failing with this?

[IanWood1]

Here are the failing executable sources from ToM(4efac5b): https://gist.github.com/IanWood1/9a6ee98c76a8ef186071a69c539fcefa

To reproduce use iree-compile source.mlir -o /dev/null --compile-from=executable-sources --iree-llvmgpu-test-combine-layout-transformation=false on any of the sources.

--iree-llvmgpu-test-combine-layout-transformation=false seems to be related. Removing it fixes the compilation error. Why are we using a "test" flag?

[nirvedhmeshram]

Here are the failing executable sources from ToM(4efac5b): https://gist.github.com/IanWood1/9a6ee98c76a8ef186071a69c539fcefa

To reproduce use iree-compile source.mlir -o /dev/null --compile-from=executable-sources --iree-llvmgpu-test-combine-layout-transformation=false on any of the sources.

--iree-llvmgpu-test-combine-layout-transformation=false seems to be related. Removing it fixes the compilation error. Why are we using a "test" flag?

ok so the problem is without that flag we end up with iree_tensor_ext.dispatch.tensor.store but now our codegen is set up for iree_codegen.store_to_buffer @Max191 is it ok to not use this flag? would like to understand the context why model people are using it..

[nirvedhmeshram]

ok I have a fix over here #22291