Repository navigation
Enable opt-in wider gfx1250 FP8 WMMA - #14391
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The narrowly scoped change preserves default behavior and opts into existing K128 backend support, with no blocking issues identified.
No open issues 🎉
What changed in this PR
Adds opt-in wider FP8/BF8 WMMA selection for CK Tile on gfx1250 while preserving existing defaults.
Changes:
- Adds a
WiderInstructiontemplate parameter, defaulting tofalse. - Selects K128 for eligible FP8/BF8 tiles when enabled, leaving other selection paths unchanged.
| File | Description |
|---|---|
projects/composablekernel/include/ck_tile/ops/gemm/pipeline/tile_gemm_shape.hpp |
Adds default-disabled K128 selection to get_k_warp_tile(). |
Review effort: Balanced
⏳ Waiting on Required Checks⏳ Required checks have not all concluded yet. This bot run is exiting without failing so those checks can report independently.
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
⏳ Waiting for required checks to start or finish:
|
illsilin
approved these changes
Oct 26, 2026
assistant-librarian Bot
pushed a commit
to ROCm/composable_kernel
that referenced
this pull request
Oct 27, 2026
Enable opt-in wider gfx1250 FP8 WMMA ## Motivation Enable CK Tile configurations on gfx1250 to select the native FP8/BF8 `16x16x128` WMMA instruction instead of the default `16x16x64` instruction. This must remain opt-in because K128 requires a compatible block K tile and may increase register pressure or reduce occupancy. Smaller tiles can perform better in some cases: | FP8 problem | Pipeline | K64 block tile | K128 block tile | K64 advantage | |---|---|---:|---:|---:| | `512x512x512` | Basic V1 | 11.803 TFLOP/s | 9.401 TFLOP/s | +25.6% | | `512x512x512` | TDM | 12.474 TFLOP/s | 11.822 TFLOP/s | +5.5% | | `4096x4096x4096` | TDM | 1336.35 TFLOP/s | 1210.67 TFLOP/s | +10.4% | These results compare block tile sizes rather than only instruction width, demonstrating why configurations should retain explicit control over K128 selection. ## Technical Details Adds a `WiderInstruction` template parameter to `get_k_warp_tile()`. - Defaults to `false`, preserving existing behavior. - When enabled for gfx1250 FP8/BF8 with a `16x16` warp tile, selects K128 instead of K64. - Does not change FP16, FP32, integer, or non-gfx1250 selection. - Configurations must opt in only when their block K supports the K128 atom. Example: ```cpp get_k_warp_tile<fp8_t, 16, false, true>() ``` ## Test Plan - Compile-time verification that default gfx1250 FP8 still selects K64. - Compile-time verification that opt-in FP8 and BF8 select K128. - Compile-time verification that opt-in FP16 remains K32. - Compile the selector assertions with the gfx1250 toolchain. - Run `clang-format-18`. Benchmark FP8 RCR GEMM with physical `16x16x128` WMMA instructions. Each result is the median of three trials with 20 warmups and 100 measured iterations. ## Test Result All compile-time checks and formatting checks passed. | Shape | Block / warp tile | Reference | K128 | Gain | Latency | |---|---|---:|---:|---:|---:| | `2304x512x16384` | `64x64x256` / `16x16x128` | 464.9 | **550.3** | +18.4% | 70.2 us | | `512x2304x16384` | `64x64x256` / `16x16x128` | 452.2 | **547.0** | +21.0% | 70.7 us | | `1024x5120x8192` | `64x128x128` / `32x32x128` | 1389.5 | **1805.6** | +29.9% | 47.6 us | | `5120x1024x8192` | `64x128x128` / `32x32x128` | 1516.9 | **1920.9** | +26.6% | 44.7 us | | `4096x4096x16384` | `256x128x128` / `32x32x128` | 1942.4 | **2423.9** | +24.8% | 226.8 us | | `14336x2048x1024` | `128x64x128` / `32x32x128` | 1334.4 | **1453.7** | +8.9% | 41.4 us | | `2048x14336x1024` | `128x64x128` / `32x32x128` | 1275.0 | **1434.4** | +12.5% | 41.9 us | The `32x32x128` logical tiles decompose into four physical `16x16x128` WMMA operations. The reference column represents the previous benchmark champion and is not a strict same-kernel K64/K128 comparison. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests. Co-authored-by: Marton Bidlek <marton.bidlek@streamhpc.com>
Contributor
TheRock Submodule Bump ActivityNewest first
|
shumway
pushed a commit
to ROCm/composable_kernel
that referenced
this pull request
Nov 4, 2026
Enable opt-in wider gfx1250 FP8 WMMA ## Motivation Enable CK Tile configurations on gfx1250 to select the native FP8/BF8 `16x16x128` WMMA instruction instead of the default `16x16x64` instruction. This must remain opt-in because K128 requires a compatible block K tile and may increase register pressure or reduce occupancy. Smaller tiles can perform better in some cases: | FP8 problem | Pipeline | K64 block tile | K128 block tile | K64 advantage | |---|---|---:|---:|---:| | `512x512x512` | Basic V1 | 11.803 TFLOP/s | 9.401 TFLOP/s | +25.6% | | `512x512x512` | TDM | 12.474 TFLOP/s | 11.822 TFLOP/s | +5.5% | | `4096x4096x4096` | TDM | 1336.35 TFLOP/s | 1210.67 TFLOP/s | +10.4% | These results compare block tile sizes rather than only instruction width, demonstrating why configurations should retain explicit control over K128 selection. ## Technical Details Adds a `WiderInstruction` template parameter to `get_k_warp_tile()`. - Defaults to `false`, preserving existing behavior. - When enabled for gfx1250 FP8/BF8 with a `16x16` warp tile, selects K128 instead of K64. - Does not change FP16, FP32, integer, or non-gfx1250 selection. - Configurations must opt in only when their block K supports the K128 atom. Example: ```cpp get_k_warp_tile<fp8_t, 16, false, true>() ``` ## Test Plan - Compile-time verification that default gfx1250 FP8 still selects K64. - Compile-time verification that opt-in FP8 and BF8 select K128. - Compile-time verification that opt-in FP16 remains K32. - Compile the selector assertions with the gfx1250 toolchain. - Run `clang-format-18`. Benchmark FP8 RCR GEMM with physical `16x16x128` WMMA instructions. Each result is the median of three trials with 20 warmups and 100 measured iterations. ## Test Result All compile-time checks and formatting checks passed. | Shape | Block / warp tile | Reference | K128 | Gain | Latency | |---|---|---:|---:|---:|---:| | `2304x512x16384` | `64x64x256` / `16x16x128` | 464.9 | **550.3** | +18.4% | 70.2 us | | `512x2304x16384` | `64x64x256` / `16x16x128` | 452.2 | **547.0** | +21.0% | 70.7 us | | `1024x5120x8192` | `64x128x128` / `32x32x128` | 1389.5 | **1805.6** | +29.9% | 47.6 us | | `5120x1024x8192` | `64x128x128` / `32x32x128` | 1516.9 | **1920.9** | +26.6% | 44.7 us | | `4096x4096x16384` | `256x128x128` / `32x32x128` | 1942.4 | **2423.9** | +24.8% | 226.8 us | | `14336x2048x1024` | `128x64x128` / `32x32x128` | 1334.4 | **1453.7** | +8.9% | 41.4 us | | `2048x14336x1024` | `128x64x128` / `32x32x128` | 1275.0 | **1434.4** | +12.5% | 41.9 us | The `32x32x128` logical tiles decompose into four physical `16x16x128` WMMA operations. The reference column represents the previous benchmark champion and is not a strict same-kernel K64/K128 comparison. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests. Co-authored-by: Marton Bidlek <marton.bidlek@streamhpc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Enable CK Tile configurations on gfx1250 to select the native FP8/BF8
16x16x128WMMA instruction instead of the default16x16x64instruction.This must remain opt-in because K128 requires a compatible block K tile and may increase register pressure or reduce occupancy. Smaller tiles can perform better in some cases:
512x512x512512x512x5124096x4096x4096These results compare block tile sizes rather than only instruction width, demonstrating why configurations should retain explicit control over K128 selection.
Closes #14391
Technical Details
Adds a
WiderInstructiontemplate parameter toget_k_warp_tile().false, preserving existing behavior.16x16warp tile, selects K128 instead of K64.Example:
Test Plan
clang-format-18.Benchmark FP8 RCR GEMM with physical
16x16x128WMMA instructions. Each result is the median of three trials with 20 warmups and 100 measured iterations.Test Result
All compile-time checks and formatting checks passed.
2304x512x1638464x64x256/16x16x128512x2304x1638464x64x256/16x16x1281024x5120x819264x128x128/32x32x1285120x1024x819264x128x128/32x32x1284096x4096x16384256x128x128/32x32x12814336x2048x1024128x64x128/32x32x1282048x14336x1024128x64x128/32x32x128Submission Checklist