[HAL][Metal] Runtime returns incorrect tensor values

[chrsmcgrr]

What happened?

Compiling a simple MatMul example with IREE using the Metal backend and executing with the runtime yields invalid results.

iree-run-module --module=model.vmfb --input=10x10xf32=0.42 --function="main" --device=metal
EXEC @main
result[0]: hal.buffer_view
10x10xf32=[0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0][0 0 0 0 0 0 0 0 0 0]

Steps to reproduce your issue

1. Given the following linalg

module @module {
  util.func public @main$async(%arg0: !hal.buffer_view, %arg1: !hal.fence, %arg2: !hal.fence) -> !hal.buffer_view attributes {inlining_policy = #util.inline.never, iree.abi.model = "coarse-fences", iree.abi.stub} {
    %cst = arith.constant 0.000000e+00 : f32
    %0 = hal.tensor.import wait(%arg1) => %arg0 : !hal.buffer_view -> tensor<10x10xf32>
    %1 = tensor.empty() : tensor<10x10xf32>
    %2 = linalg.fill ins(%cst : f32) outs(%1 : tensor<10x10xf32>) -> tensor<10x10xf32>
    %3 = linalg.matmul ins(%0, %0 : tensor<10x10xf32>, tensor<10x10xf32>) outs(%2 : tensor<10x10xf32>) -> tensor<10x10xf32>
    %4 = hal.tensor.barrier join(%3 : tensor<10x10xf32>) => %arg2 : !hal.fence
    %5 = hal.tensor.export %4 : tensor<10x10xf32> -> !hal.buffer_view
    util.return %5 : !hal.buffer_view
  }
  util.func public @main(%arg0: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub} {
    %0 = util.null : !hal.fence
    %c-1_i32 = arith.constant -1 : i32
    %c0 = arith.constant 0 : index
    %device_0 = hal.devices.get %c0 : !hal.device
    %fence = hal.fence.create device(%device_0 : !hal.device) flags("None") : !hal.fence
    %1 = util.call @main$async(%arg0, %0, %fence) : (!hal.buffer_view, !hal.fence, !hal.fence) -> !hal.buffer_view
    %status = hal.fence.await until([%fence]) timeout_millis(%c-1_i32) : i32
    util.return %1 : !hal.buffer_view
  }
}

2. Compiling the linalg with:

iree-compile --iree-input-type=auto --iree-hal-target-backends=metal-spirv iree.mlir -o model.vmfb

3. Executing with run-module

iree-run-module --module=model.vmfb --input=10x10xf32=0.42 --function="main" --device=metal

4. Check that tensor returns elements that are not zero.

What component(s) does this issue relate to?

Runtime

Version information

iree-base-compiler 3.0.0
iree-base-runtime 3.0.0

Additional context

If I run on latest main: ed9a028d3f3bfb0ab32004881c87539577048aa8

I get the following errorin the runtime:

Assertion failed: (placement.device), function iree_hal_metal_buffer_wrap, file metal_buffer.m, line 52.

[ScottTodd]

Thanks for the report! Two clarifying questions:

1. Do your repro steps work as expected on other backends, like --iree-hal-target-backends=llvm-cpu and --device=local-sync?

2. When you tried with

   If I run on latest main: ed9a028d3f3bfb0ab32004881c87539577048aa8

   I get the following errorin the runtime:

   Assertion failed: (placement.device), function iree_hal_metal_buffer_wrap, file metal_buffer.m, line 52.
   

   Did you also recompile the .vmfb file? Compatibility with the prior release may have changed, and that looks like a different error than the output being incorrect. Could either be a bug in the runtime code or a mismatch between the compiler and the runtime, which should be fixed by compiling with a recent version of iree-compile (e.g. from the nightly releases: https://iree.dev/reference/bindings/python/#__tabbed_2_2)

[chrsmcgrr]

Thanks for the quick response!

Do your repro steps work as expected on other backends, like --iree-hal-target-backends=llvm-cpu and --device=local-sync?

Yes compiling for CPU works with llvm-cpu and local-sync

Did you also recompile the .vmfb file?

I did with locally built binaries of iree-compile and iree-run-module. I will debug a little further.