feat(hiptensor): add bf16 support for elementwise permute

projects/hiptensor/CHANGELOG.md

@@ -8,6 +8,13 @@ Full documentation for hipTensor is available at [rocm.docs.amd.com/projects/hip
 * Added support for the following new GPU targets:
   * gfx11-7: gfx1170, gfx1171.
 * Added fp64 support to hiptensorPermute for ranks 2-6.
+* Added bf16 support to hiptensorPermute for ranks 2-6, with a bf16 or fp32 scalar type.
+
+### Changed
+* Changed `hiptensorPermute` to return `HIPTENSOR_STATUS_NOT_SUPPORTED` when the output tensor's data type differs from the input tensor's. Previously the mismatch passed validation and the call failed with `HIPTENSOR_STATUS_INTERNAL_ERROR` because no kernel matched.
+
+### Resolved issues
+* Fixed `hiptensorElementwiseBinaryExecute` and `hiptensorReduce` treating the alpha, beta, and gamma scalars as 0 when the compute descriptor is `HIPTENSOR_COMPUTE_DESC_16BF`.
 
 ## hipTensor 2.5.0 for ROCm 10.2
 

projects/hiptensor/TESTING.md

@@ -49,16 +49,19 @@ cmake --install <build_dir> --component tests
 
 **Purpose:** Validate hardware-independent logic without dispatching a GPU kernel.
 
-Unit tests live in `test/00_unit/` and are built with the `add_hiptensor_unit_test()` CMake function, which links against `hiptensor::hiptensor` and GoogleTest. Most do **not** require a configured GPU to run (the exception is `plan_lifetime_test`, noted below). The test binaries are:
+Unit tests live in `test/00_unit/` and are built with the `add_hiptensor_unit_test()` CMake function, which links against `hiptensor::hiptensor` and GoogleTest. Most do **not** launch kernels (the exceptions are `plan_lifetime_test` and `elementwise_broadcast_test`, noted below). The test binaries are:
 
 | Binary | What it tests |
 |---|---|
 | `logger_test` | Logger singleton identity, callback registration, log-level filtering, file-sink behavior |
 | `yaml_test` | YAML config parsing and round-trip correctness |
 | `elementwise_op_test` | CPU-side elementwise operator type traits and identity/scale op correctness |
-| `util_test` | Utility helpers (type conversion, index math, stride calculation) |
+| `util_test` | Utility helpers (type conversion, scalar reads per compute type, index math, stride calculation) |
 | `hiptensor_options_test` | CLI/env option parsing (`--hot_runs`, `--cold_runs`, `-y`, `HIPTENSOR_*` env vars) |
 | `plan_lifetime_test` | Plan object lifetime: using a permutation plan after its descriptor or preference object is destroyed. Despite living in `00_unit/`, it allocates device memory and launches a real permute kernel, so it **requires a GPU** and is registered as a device tier test |
+| `batched_contraction_test` | `hiptensorCreateContraction` / `hiptensorCreateContractionTrinary` return `HIPTENSOR_STATUS_NOT_SUPPORTED` for batched modes (a mode shared by both inputs and the output) and accept non-batched ones. No kernel launch |
+| `elementwise_mode_test` | Elementwise mode validation: subset and reordered input modes are accepted; a mode missing from the output or repeated within a tensor returns `NOT_SUPPORTED`; a shared mode with mismatched extents returns `INVALID_VALUE`; a permute whose output data type differs from the input's returns `NOT_SUPPORTED`. Allocates small device buffers but launches no kernel |
+| `elementwise_broadcast_test` | Elementwise broadcast results for permute, binary, and trinary (inputs carrying a subset of the output modes). Launches kernels, so it **requires a GPU** |
 
 > `interface_test` exists in the source tree but is commented out in `test/00_unit/CMakeLists.txt` — it is not currently built.
 
@@ -135,18 +138,19 @@ Integration tests are device tests. They require a physical AMD GPU matching the
 | Plan cache | `plan_cache_test` | Cache write/read round-trip on device |
 | Plan lifetime | `plan_lifetime_test` | Permute plan reuse after descriptor/preference destruction (launches a kernel) |
 | Contraction mode | `contraction_mode_test` | CPU-side mode validation (host test) |
-| Elementwise — permute | `rank{2-6}_elementwise_permute_test` | All rank/type combinations |
-| Elementwise — binary op | `rank{2-6}_elementwise_binary_op_test` | Binary elementwise ops |
-| Elementwise — trinary op | `rank{2-6}_elementwise_trinary_op_test` | Trinary elementwise ops |
-| Elementwise — CPU impl | `elementwise_cpu_test` | CPU reference validation (host test; compiles the `elementwise_cpu_impl`, `elementwise_binary_cpu_impl`, and `elementwise_trinary_cpu_impl` sources into one binary) |
+| Elementwise — permute | `rank{2-6}_elementwise_permute_test` | FP16/BF16/FP32/FP64; FP16 and BF16 also with an FP32 scalar; ranks 2–6 |
+| Elementwise — binary op | `rank{2-6}_elementwise_binary_op_test` | FP16/FP32/FP64; ranks 2–6 (BF16 is accepted by the API but not tested — see Known Gaps) |
+| Elementwise — trinary op | `rank{2-6}_elementwise_trinary_op_test` | FP16/FP32/FP64; ranks 2–6 |
+| Elementwise — output strides | `elementwise_output_strides_test` | User-supplied (gapped and packed) output strides for permute, binary, and trinary (FP32); launches kernels |
+| Elementwise — CPU impl | `elementwise_cpu_test` | CPU reference validation against fixed expected values (host test; compiles the `elementwise_cpu_impl`, `elementwise_binary_cpu_impl`, and `elementwise_trinary_cpu_impl` sources into one binary). Permute: FP32/FP16/BF16 (with a BF16 or FP32 scalar)/FP64 (checked at FP64 precision); binary and trinary: FP32/FP16 |
 | Reduction | `rank{1-6}_reduction_test` | All rank/type combinations |
 | Reduction — CPU impl | `reduction_cpu_impl_test` | CPU reference validation (host test) |
 
 All device tests compare GPU results against a CPU reference implementation using a per-type tolerance.
 
-**GPU-required tests:** All `bilinear_contraction_test_*`, `scale_contraction_test_*`, `complex_*`, `trinary_*`, `*_unary_ops_test`, `rank*_elementwise_*`, `rank*_reduction_test`, `plan_cache_test`, `plan_lifetime_test`.
+**GPU-required tests:** All `bilinear_contraction_test_*`, `scale_contraction_test_*`, `complex_*`, `trinary_*`, `*_unary_ops_test`, `rank*_elementwise_*`, `rank*_reduction_test`, `plan_cache_test`, `plan_lifetime_test`, `elementwise_broadcast_test`, `elementwise_output_strides_test`.
 
-**CPU-only (host) tests:** `contraction_mode_test`, `elementwise_cpu_test`, `reduction_cpu_impl_test`, and the `00_unit/` binaries (`logger_test`, `yaml_test`, `elementwise_op_test`, `util_test`, `hiptensor_options_test`).
+**CPU-only (host) tests:** `contraction_mode_test`, `elementwise_cpu_test`, `reduction_cpu_impl_test`, and the `00_unit/` binaries (`logger_test`, `yaml_test`, `elementwise_op_test`, `util_test`, `hiptensor_options_test`). `batched_contraction_test` and `elementwise_mode_test` launch no kernels but create a hipTensor handle and descriptors (`elementwise_mode_test` also allocates small device buffers).
 
 **Test tiers (CTest labels):**
 
@@ -268,6 +272,8 @@ llvm-profdata merge -sparse default.profraw -o coverage.profdata
 llvm-cov report ./bin/<test_binary> -instr-profile=coverage.profdata
 ```
 
+**Coverage test configs:** the `codecoverage` set in `rtest.xml` reads its YAML configs from `<build_dir>/code_coverage_config/`, which the build populates (`copy_code_coverage_config_files` in `cmake/Functions/CopyYamlFolder.cmake`). Contraction and reduction copy from their `configs/code_coverage/` directories; elementwise copies from `configs/emulation/quick/` (permutation, binary_op, trinary_op). `test/02_elementwise/configs/code_coverage/` is not used — edit `emulation/quick` to change elementwise coverage.
+
 **Coverage targets:**
 
 | Scope | Target | Notes |
@@ -355,7 +361,7 @@ There is no client-side known-bug list (analogous to `known_bugs.yaml` in other
 | New CPU-side logic (logging, YAML, option parsing, utility math) | Unit test in `test/00_unit/` using GoogleTest |
 | New or changed GPU operation behavior | Integration test: add a YAML config entry in the appropriate `validation/` tier |
 | Bug fix | A regression test (unit if CPU-side, integration config entry if GPU-side) that fails before the fix |
-| New data type or tensor rank | Config entries across all relevant validation tiers; empty config file for tiers where the test is already covered |
+| New data type or tensor rank | Config entries across all relevant validation tiers; empty config file for tiers where the test is already covered. Also add entries to `emulation/quick` **and** `emulation/full` (the FFM tiers do not nest, so `ffm-full` does not inherit `ffm-quick`), to `bench`, and a case in the op's host CPU-reference test (e.g. `elementwise_cpu_test`) where one exists |
 | Performance-sensitive change | Run `ctest -V -O bench.log` from `<prefix>/bin/hiptensor/bench` before and after; include the comparison in the PR description. The `scripts/performance/Benchmark*.sh` scripts are an alternative. |
 | New GPU architecture | Validate on that target; update the Supported Configurations table |
 
@@ -374,6 +380,9 @@ There is no client-side known-bug list (analogous to `known_bugs.yaml` in other
 | Trinary/unary-ops variants have no bench configs | Low | Low | Not performance-gated |
 | `clang-format` and static analysis not enforced in CI | Low | Low | Manual enforcement in code review |
 | No sanitizer (ASAN/TSAN) gate in CI | Low | High if hit | None; ASAN can be built manually |
+| Elementwise binary op accepts BF16 but the binary test harness has no BF16 support | Medium | Medium | BF16 scalar reads covered by `util_test`; no end-to-end binary BF16 test |
+| Permute `bench` configs exceed device memory on smaller GPUs: the binary aborts with a HIP out-of-memory error partway through (gfx908 and gfx1100 already in the FP32 entries; gfx90a in the FP64 entries at ranks 2 and 4, which are listed last so the other types complete) | Low | Medium | Bench results on those targets are incomplete; run bench on MI300-class GPUs |
+| `test/02_elementwise/configs/code_coverage/` is unused (elementwise coverage reads `emulation/quick`) | Low | Low | Documented in the Coverage section |
 
 ---
 

projects/hiptensor/docs/api-reference/api-reference.rst

@@ -100,6 +100,18 @@ Data Types **<Ti / To / Tc>** = <Input type / Output Type / Compute Type>, where
 |                     +------------------------------+                     |                     |
 |                     |     cf64 / cf64 / cf64       |                     |                     |
 +---------------------+------------------------------+---------------------+---------------------+
+|                     |     f16 / f16 / f16          |  gfx9 gfx11 gfx12   | Rank2 - Rank6       |
+| Element-wise        +------------------------------+                     |                     |
+| Permutation         |     f16 / f16 / f32          |                     |                     |
+|                     +------------------------------+                     |                     |
+|                     |     bf16 / bf16 / bf16       |                     |                     |
+|                     +------------------------------+                     |                     |
+|                     |     bf16 / bf16 / f32        |                     |                     |
+|                     +------------------------------+                     |                     |
+|                     |     f32 / f32 / f32          |                     |                     |
+|                     +------------------------------+---------------------+                     |
+|                     |     f64 / f64 / f64          |  gfx90a+ gfx1250    |                     |
++---------------------+------------------------------+---------------------+---------------------+
 |                     |     f16 / f16 / \-           |  gfx9 gfx11 gfx12   | Rank2 - Rank6       |
 | Element-wise        +------------------------------+                     |                     |
 | Operations          |     f16 / f32 / \-           |                     |                     |

projects/hiptensor/docs/install/installation.rst

@@ -341,11 +341,11 @@ Executable name                                  Description
 ``complex_scale_contraction_test_m4n4k4``        Scale contraction test [D = alpha * (A x B)] with complex single and double precision datatypes of rank 8
 ``complex_scale_contraction_test_m5n5k5``        Scale contraction test [D = alpha * (A x B)] with complex single and double precision datatypes of rank 10
 ``complex_scale_contraction_test_m6n6k6``        Scale contraction test [D = alpha * (A x B)] with complex single and double precision datatypes of rank 12
-``rank2_elementwise_permute_test``               Permutation test with half, single and double precision datatypes of rank 2
-``rank3_elementwise_permute_test``               Permutation test with half, single and double precision datatypes of rank 3
-``rank4_elementwise_permute_test``               Permutation test with half, single and double precision datatypes of rank 4
-``rank5_elementwise_permute_test``               Permutation test with half, single and double precision datatypes of rank 5
-``rank6_elementwise_permute_test``               Permutation test with half, single and double precision datatypes of rank 6
+``rank2_elementwise_permute_test``               Permutation test with half, bfloat16, single and double precision datatypes of rank 2
+``rank3_elementwise_permute_test``               Permutation test with half, bfloat16, single and double precision datatypes of rank 3
+``rank4_elementwise_permute_test``               Permutation test with half, bfloat16, single and double precision datatypes of rank 4
+``rank5_elementwise_permute_test``               Permutation test with half, bfloat16, single and double precision datatypes of rank 5
+``rank6_elementwise_permute_test``               Permutation test with half, bfloat16, single and double precision datatypes of rank 6
 ``rank2_elementwise_binary_op_test``             Element-wise binary operation test with half, single, and double precision datatypes of rank 2
 ``rank3_elementwise_binary_op_test``             Element-wise binary operation test with half, single, and double precision datatypes of rank 3
 ``rank4_elementwise_binary_op_test``             Element-wise binary operation test with half, single, and double precision datatypes of rank 4

projects/hiptensor/library/src/elementwise/CMakeLists.txt

@@ -67,30 +67,40 @@ set(HIPTENSOR_ELEMENTWISE_SOURCES
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_cpu_reference_rank6_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_double_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_double_noop_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_bhalf_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_bhalf_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_float_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_float_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_half_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank2_half_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_double_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_double_noop_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_bhalf_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_bhalf_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_float_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_float_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_half_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank3_half_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_double_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_double_noop_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_bhalf_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_bhalf_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_float_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_float_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_half_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank4_half_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_double_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_double_noop_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_bhalf_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_bhalf_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_float_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_float_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_half_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank5_half_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_double_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_double_noop_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_bhalf_instances.cpp
+   ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_bhalf_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_float_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_float_noop_instances.cpp
    ${CMAKE_CURRENT_SOURCE_DIR}/instances/elementwise_permute_solution_rank6_half_instances.cpp