[cliffxzx]

Motivation

After some experimentation, we found that the pack code in the F8 NT and NN cases was the performance bottleneck because the WMMA and pack code share the same pipeline. Starting from GFX12, a new instruction called global_load_tr allows us to eliminate the cost of the pack code by remapping the data. This makes it possible to support DirectToVgpr in the TLU case.

Technical Details

To use it as 16x16x16x1 WMMA input, the offset for global_load_tr should follow this order:

global_load_tr_b128

global_load_tr_b64

Test Plan

Test different MacroTile and parameters combinations on F8F8S and HHS across NN, NT, and TT cases.

      ForkParameters:
        - MatrixInstruction:
          - [16, 16,16, 1, 1, 1, 1, 1, 1]

          - [16, 16,16, 1, 1, 1, 1, 4, 1]
          - [16, 16,16, 1, 1, 2, 1, 4, 1]
          - [16, 16,16, 1, 1, 2, 1, 2, 2]
          - [16, 16,16, 1, 1, 3, 1, 2, 2]
          - [16, 16,16, 1, 1, 7, 1, 1, 4]
          - [16, 16,16, 1, 1, 8, 1, 1, 4]
          - [16, 16,16, 1, 1, 9, 1, 1, 4]
          - [16, 16,16, 1, 1, 10, 1, 1, 4]

          - [16, 16,16, 1, 1, 1, 1, 1, 4]
          - [16, 16,16, 1, 1, 1, 2, 1, 4]
          - [16, 16,16, 1, 1, 1, 2, 2, 2]
          - [16, 16,16, 1, 1, 1, 3, 2, 2]
          - [16, 16,16, 1, 1, 1, 7, 4, 1]
          - [16, 16,16, 1, 1, 1, 8, 4, 1]
          - [16, 16,16, 1, 1, 1, 9, 4, 1]
          - [16, 16,16, 1, 1, 1, 10, 4, 1]

          - [16, 16, 16, 1, 1, 2, 2, 2, 4]
          - [16, 16, 16, 1, 1, 2, 2, 4, 1]
        - ScheduleIterAlg: [3]
        - PrefetchLocalRead: [1]
        - PrefetchGlobalRead: [2]
        - DepthU: [16,64]
        - WavefrontSize: [32]
        - LocalReadVectorWidth: [8]
        - GlobalReadVectorWidthA: [8]
        - GlobalReadVectorWidthB: [8]
        - InnerUnroll: [1]
        - VectorWidthA: [1]
        - VectorWidthB: [1]
        - StoreVectorWidth: [-1]
        - StoreRemapVectorWidth: [0]
        - TransposeLDS: [-1,2]
        - 1LDSBuffer: [0,1]
        - ScheduleLocalWrite: [1]
        - GlobalSplitU: [1]
        - GlobalSplitUAlgorithm: ["MultipleBuffer"]
        - SourceSwap: [0,1]
        - ExpandPointerSwap: [0,1]
        - PreloadKernArgs: [1]
        - DirectToVgprA: [0,1]
        - DirectToVgprB: [0,1]
      BenchmarkFinalParameters:
        - ProblemSizes:
          - Range: [[1], [1], [1], [1]]
          - Range: [[64], [64], [1], [64]]
          - Range: [[128], [128], [1], [128]]
          - Range: [[256], [256], [1], [256]]
          - Range: [[512], [512], [1], [512]]
          - Range: [[1024], [1024], [1], [1024]]

          - Range: [[32], [32], [1], [32]]
          - Range: [[64], [64], [1], [32]]
          - Range: [[128], [128], [1], [64]]
          - Range: [[128], [128], [1], [32]]
          - Range: [[64], [64], [1], [512]]
          - Range: [[64], [64], [1], [1024]]
          - Range: [[64], [64], [1], [2048]]
          - Range: [[512], [512], [1], [32]]
          - Range: [[512], [128], [1], [64]]
          - Range: [[128], [128], [1], [512]]
          - Range: [[256], [128], [1], [32]]
          - Range: [[256], [128], [1], [64]]
          - Range: [[128], [512], [1], [64]]
          - Range: [[128], [512], [1], [512]]
          - Range: [[512], [512], [1], [512]]
          - Range: [[512], [512], [1], [128]]
          - Range: [[64], [64], [1], [256]]
          - Range: [[64], [64], [1], [512]]
          - Range: [[256], [512], [1], [256]]
          - Range: [[512], [384], [1], [384]]
          - Range: [[640], [512], [1], [512]]
          - Range: [[512], [256], [1], [640]]
          - Range: [[512], [512], [1], [896]]
          - Range: [[512], [512], [1], [1152]]
          - Range: [[512], [256], [1], [1024]]

          - Range: [[128], [128], [1], [63]]
          - Range: [[512], [512], [1], [65]]
          - Range: [[512], [512], [1], [66]]
          - Range: [[512], [512], [1], [127]]
          - Range: [[512], [512], [1], [129]]
          - Range: [[513], [513], [1], [129]]

          - Range: [[127], [127], [1], [127]]
          - Range: [[129], [129], [1], [129]]
          - Range: [[128], [128], [1], [96]]
          - Range: [[85], [128], [1], [64]]
          - Range: [[640], [513], [1], [64]]
          - Range: [[640], [512], [1], [64]]
          - Range: [[528], [512], [1], [64]]
          - Range: [[520], [512], [1], [64]]
          - Range: [[128], [128], [1], [64]]
          - Range: [[512], [513], [1], [64]]
          - Range: [[524], [512], [1], [64]]
          - Range: [[513], [513], [1], [64]]
          - Range: [[513], [512], [1], [64]]
          - Range: [[513], [513], [1], [64]]

          - Range: [ [8, 35, 256], [8, 35, 256], [1], [8, 36, 256] ]
          - Range: [ [8, 43, 512], [8, 65, 512], [1], [8, 88, 512] ]
          - Range: [ [32, 32, 256], [32, 32, 256], [1], [32, 32, 256] ]
        - BiasTypeArgs: s
        - ActivationArgs:
          - [Enum: relu]

Test Result

All Passed

Benchmark Result

7k eff	DTVA	DTVB	DTVA+DTVB	Ori	New
NT F16	83.34%	80.98%	88.43%	80%	88.43%
NT F8	75%	74.06%	87.80%	60%	87.80%
NN F16	91.03%	85%	75.04%	85%	91.03%
NN F8	82.21%	65.01%	74.21%	65%	82.21%

Benchmark results on LLaMA model sizes show about a 7.3% average performance improvement.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[Copilot]

Pull Request Overview

This PR adds TLU (transpose/layout) support for DirectToVgpr via global_load_tr instruction, targeting improved performance on GFX12 architecture. The implementation introduces new global_load_tr instructions (TR8_B64 and TR16_B128) that allow data remapping during load operations, eliminating the performance bottleneck from pack code that previously shared the same pipeline as WMMA instructions.

Key changes include:

• Added new global_load_tr instructions for 8-byte (TR8_B64) and 16-byte (TR16_B128) transpose operations
• Renamed class hierarchy from "GlobalRead/Write" to "TextureRead/Write" with corresponding function updates
• Updated kernel generation logic to support GL (Global Load) transpose operations
• Added hardware capability detection for the new transpose instructions

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
rocisa/test/test_instruction.py	Updated test function name from countGlobalRead to countTextureRead
rocisa/src/instruction/mem.cpp	Renamed GlobalInstruction classes to TextureInstruction and added GLOBALReadInstruction with new TR instruction bindings
rocisa/src/enum.cpp	Added new instruction types INST_TR8_B64 and INST_TR16_B128
rocisa/src/count.cpp	Renamed countGlobalRead function to countTextureRead
rocisa/src/container.cpp	Added GLOBALModifiers container class for global load instruction modifiers
rocisa/include/instruction/mem.hpp	Refactored class hierarchy and added GlobalLoadTR8B64 and GlobalLoadTR16B128 instruction classes
rocisa/include/hardware_caps.hpp	Added hardware capability detection for GL transpose instructions
rocisa/include/enum.hpp	Added TR instruction enum values
rocisa/include/container.hpp	Added GLOBALModifiers struct definition
Tensile/Tests/common/gemm/gfx12/*.yaml	Added test configurations for FP8 and FP16 with DirectToVgpr transpose support
Tensile/SolutionStructs/Solution.py	Added enableGLTr logic and DirectToVgpr compatibility checks for transpose operations
Tensile/KernelWriterAssembly.py	Updated kernel generation to support GL transpose instructions and memory access patterns
Tensile/KernelWriter.py	Updated function references from countGlobalRead to countTextureRead
Tensile/Components/*.py	Updated function references and added transpose-aware code generation

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[cmingch]

good job