Repository navigation
feat(rocke): IR dtypes in TileMma, a storage type under every Fragment, and MMA issue from the measured backend coverage - #14537
Merged
cgmillette merged 7 commits intoOct 31, 2026
Conversation
github-actions
Bot
added
documentation
project: hip-kernel-provider
rocke: platform
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved fragment-transform and unsigned multi-tile failures remain, alongside repeated backend-table loading and missing numerical validation of MMA K pairing.
4 open issues
What changed in this PR
This PR builds on the MMA catalog work in #14329 to carry IR dtypes through rocKE’s tiling layer, separate fragment storage from logical types, and issue the exact backend instruction recorded for each target.
Changes:
- Adds storage-backed fragments and IR-typed MMA operands, accumulators, and results.
- Uses measured backend coverage for instruction selection and canonical layouts for mixed-width K pairing.
- Updates validation, visualization, tests, demos, and API documentation.
Paths below are relative to dnn-providers/hip-kernel-provider/rocke/.
| File | Description |
|---|---|
style/API_DESIGN_STANDARD.md |
Updates the K-pairing validation signature. |
platform/python/rocke/tests/helpers/tiling/visualization/test_layout_render.py |
Migrates rendering tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/visualization/test_auto_pipeline.py |
Tests recorded operand dtypes and mixed-input rendering. |
platform/python/rocke/tests/helpers/tiling/traits/test_mma_traits.py |
Tests IR-typed catalog selection and diagnostics. |
platform/python/rocke/tests/helpers/tiling/test_transforms.py |
Supplies canonical layouts to K-pairing tests. |
platform/python/rocke/tests/helpers/tiling/test_scheduling.py |
Migrates scheduling tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/test_lds_descriptor_addressing.py |
Migrates addressing tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/test_layout_optimizer.py |
Supplies partner canonical layouts to optimizer tests. |
platform/python/rocke/tests/helpers/tiling/test_interleave_all_dense.py |
Checks mixed-width interleaving rejection. |
platform/python/rocke/tests/helpers/tiling/test_fragment.py |
Migrates fragment test setup to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/test_emit_wide_access.py |
Migrates wide-access tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/test_coalescing_probe.py |
Migrates coalescing tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_validation_recipe.py |
Updates dtype and canonical-layout validation inputs. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_operation.py |
Tests dtype validation, pinning, and mixed selection. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_operand_feed.py |
Tests TF32 and packed operand feeding. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_k_coupling.py |
Tests canonical K coupling and its consumers. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_issue.py |
Tests recorded instruction issue and result dtypes. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_issuable_pairs.py |
Sweeps issuable dense instruction variants. |
platform/python/rocke/tests/helpers/tiling/mma/test_mma_dtypes.py |
Tests 8-bit vector accesses and visualization labels. |
platform/python/rocke/tests/helpers/tiling/mma/test_layout_styles.py |
Migrates layout-style tests to IR dtypes. |
platform/python/rocke/tests/helpers/tiling/mma/test_layout_oracle.py |
Migrates oracle selection to IR dtypes. |
platform/python/rocke/helpers/tiling/visualization/layout_render.py |
Uses operand-specific labels and canonical pairing diagnostics. |
platform/python/rocke/helpers/tiling/visualization/auto_pipeline.py |
Reads operand widths from recorded MMA dtypes. |
platform/python/rocke/helpers/tiling/transforms/observers.py |
Validates K alignment at coupled positions. |
platform/python/rocke/helpers/tiling/transforms/_core.py |
Derives K coupling from canonical layouts. |
platform/python/rocke/helpers/tiling/traits/README.md |
Documents typed selection and coverage-driven issue. |
platform/python/rocke/helpers/tiling/traits/mma_traits.py |
Accepts IR dtypes and improves selection diagnostics. |
platform/python/rocke/helpers/tiling/tiling_recorder.py |
Records each MMA operand’s dtype. |
platform/python/rocke/helpers/tiling/README.md |
Updates the introductory example to IR dtypes. |
platform/python/rocke/helpers/tiling/mma/plan.py |
Resolves typed plans and recorded backend operations. |
platform/python/rocke/helpers/tiling/mma/mma_operation.py |
Exposes operand and result dtypes. |
platform/python/rocke/helpers/tiling/mma/driver.py |
Feeds storage registers and handles distinct result types. |
platform/python/rocke/helpers/tiling/memory.py |
Clarifies storage-unit vector-width documentation. |
platform/python/rocke/helpers/tiling/lds_conflict.py |
Updates width handling and rejects packed byte-address analysis. |
platform/python/rocke/helpers/tiling/layout_optimizer.py |
Requires canonical references for partner K validation. |
platform/python/rocke/helpers/tiling/kernels/tiling_gemm_interleaved_demo.py |
Migrates the GEMM demo to IR dtypes. |
platform/python/rocke/helpers/tiling/kernels/tiling_gemm_crc_demo/crc_interleaved_gemm.py |
Migrates both operand-order paths to IR dtypes. |
platform/python/rocke/helpers/tiling/fragments.py |
Defines and validates dtype-specific fragment storage. |
platform/python/rocke/helpers/tiling/docs/visualization_api_surface.md |
Clarifies visualization dtype sources. |
platform/python/rocke/helpers/tiling/docs/tiling_interleaving_design.md |
Documents canonical K coupling and optimizer arguments. |
platform/python/rocke/helpers/tiling/docs/tiling_api_surface.md |
Documents storage types, typed MMA calls, and pinning. |
platform/python/rocke/helpers/tiling/docs/mma_is_machinery.md |
Explains mixed-width and repeated-M-block coupling. |
platform/python/rocke/helpers/tiling/docs/api_proposals/PROPOSAL_c_transpose_crossed_ab.md |
Updates proposal examples to IR dtypes. |
platform/python/rocke/helpers/tiling/analysis/soundness.py |
Clarifies canonical coupling in soundness documentation. |
platform/python/rocke/helpers/tiling/analysis/geometry.py |
Derives element widths from dtype metadata. |
platform/python/rocke/helpers/tiling/.claude/skills/layout-viz/SKILL.md |
Updates visualization and pairing API guidance. |
platform/dsl_docs/optimization/arch/gfx942.md |
Documents TF32 truncation and pre-conversion. |
Review effort: Balanced
cgmillette
force-pushed
the
users/cgmillette/rocke/mma-catalog
branch
from
October 28, 2026 23:06
c20aad0 to
59ac8e1
Compare
cgmillette
force-pushed
the
users/cgmillette/rocke/tiling-dtypes
branch
from
October 28, 2026 23:09
8d461df to
2660d12
Compare
Contributor
|
I reviewed this PR again. The following review threads are still unresolved, or comments have no author reply:
If you have already fixed these, please reply on each comment to say so and resolve the threads. |
Contributor
Author
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14537 — IR dtypes in TileMma, a storage type under every Fragment, and MMA issue from the measured backend coverage (cgmillette, AICK-2576) — 🟠 First-round: not merge-ready — the stack must rebase onto develop (11 conflicting files, ABI 5); this PR's own commits hold up at IR level, apart from an fp6 index overflow
Utility: 🟢 Positive — typed, storage-backed fragments and record-driven MMA issue; IR-verified, hardware numerics pending. Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted First-round review: a local qwen3.6 model made the independent first pass. The orchestrator (Claude Opus 5.5) verified it. The checks were a merge-tree against develop, with each conflict and both ABI changes traced to their PRs, and the CI logs at head, with a cp1252 replay and single-test runs of the failing test on head and develop. They also included IR probes against the base |
Merged
1 task done
cgmillette
force-pushed
the
users/cgmillette/rocke/mma-catalog
branch
from
October 30, 2026 14:17
59ac8e1 to
a5e14fd
Compare
cgmillette
force-pushed
the
users/cgmillette/rocke/tiling-dtypes
branch
from
October 30, 2026 14:17
2660d12 to
423dae4
Compare
Contributor
Author
|
Restacked on the new #14329 head ( Automated review 10-29
Checks (head
|
Contributor
Author
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14537 — IR dtypes in TileMma, a storage type under every Fragment, and MMA issue from the measured backend coverage (cgmillette, AICK-2576) — 🟢 Follow-up: fp6 overflow fixed, CI pending
Net-new: none. Utility: 🟢 Positive — typed, storage-backed fragments and record-driven MMA issue; fp6 fix verified. Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: range-diff since our review, merge-trees against develop, an IR probe of our N1/N2, |
A Fragment's dtype is the logical number format; its value now holds the registers in Fragment.storage_type, derived from the dtype: the logical vector for every arithmetic dtype, i32 for tf32, and whole i32 words for a packed sub-byte dtype (fp4, fp6, i4), packed low bit first by core.storage.register_packing. A value of another type is refused; a logical tf32 vector names the bitcast to i32 that fixes it, and a packed fragment names the verbs that build its words. load_fragment, store_fragment and fill_fragment move storage units, in global memory and LDS. Element and unit widths come from core.dtypes.dtype_info, replacing the byte-width table. For every byte-or-wider dtype a unit is an element, so their IR is unchanged. A packed access moves whole 32-bit words: each lane's registers must form whole-word runs along the tensor's stride-1 axis, starting on a word boundary, and a clipped packed window must be bounded on whole words; a clipped global tile masks one whole-word group at a time. A packed LDS buffer holds i32 words, sized from the descriptor; any other buffer, and an LDS swizzle of packed data, is refused. A packed access's word index is its element index divided by the elements of one whole-word group, then multiplied by that group's words, so it is exact and stays in the i32 range wherever the element index does. A u8 fragment is held as vec<u8 x n> and stores like any byte dtype. fill_fragment and load_fragment refuse it, because they start from a zero vector of the storage type and core's zero_vec has no u8 form; the message names the route that works: an I8 fragment whose value is bitcast to vec<u8 x n>. transform_fragment reorders the storage registers: the elements of a byte-or-wider dtype (i32 for tf32; a u8 fragment through the i8 registers of the same bits) and the words of a packed fragment. A packed reorder must move whole word groups, the fewest elements that fill whole words, in order onto another group; one that would move elements within a word is refused, since repacking bits is not implemented. The byte-addressed views (the LDS bank model) refuse a sub-byte dtype instead of mis-sizing it. Tests cover the storage table, every refusal and its message, the word index of the largest whole-group element index i32 holds, in i32 arithmetic, transform_fragment on tf32, u8, fp4 and fp6 fragments, and on gfx90a bit-exact round trips of a tf32 tile through LDS, of fp4, fp6 and i4 operand tiles in gfx950's canonical f8f6f4 layout through global memory and an i32 LDS buffer, checking the loaded words against FragmentPacking.pack, plus a clipped packed window, and of tf32, u8 and fp4 tiles moved to another register order by transform_fragment. Co-Authored-By: Claude <noreply@anthropic.com>
TileMma, TileMmaPlan and MmaTraitsCatalog.select take their dtypes as
scalar IR types from rocke.core.ir (F16, F32, FP8E4M3, ...) instead of
strings. A string, or a composite type such as a vector, is refused with
the constant to pass, e.g. "a= must be a scalar IR type such as
rocke.core.ir.F16, not the string 'f16' -- pass rocke.core.ir.F16".
A new d= names the result dtype; omitted, it equals the accumulator c,
as for every accumulate-in-place instruction. Selection keys on it, so
it tells apart two instructions that differ only in their result dtype.
The plan and TileMma expose a_dtype, b_dtype, c_dtype and d_dtype as IR
types; atom_override derives them from the row through the new
MmaTraits.ir_dtypes(). The driver compares fragment dtypes with them
directly.
When no instruction matches, select names the query and lists what the
target offers in IR type names, e.g. "no MMA instruction found for
a=f16, b=f16, c=f32, shape=(13, 16, 16), family='dense', target='gfx90a'
-- available on 'gfx90a' (a x b -> c, MxNxK): ['f16 x f16 -> f32,
16x16x16', ...]".
MmaTee.from_mma labels its inputs and result from the MMA's IR dtypes;
mma_input_label is the one home of the input label ("f16", or
"fp8e4m3/bf8e5m2" when A and B differ). Labels and widths are
unchanged.
Every call site, example and doc in the tiling package passes IR types.
New tests cover the refusals (including the constant name of every
scalar IR type), the derived and default dtypes, selection by IR types,
that gfx950 has no tf32 instruction, the fp8 and bf8 wave tiles lowering
to one MFMA per atom on gfx942, and the tee's widths and labels.
Co-Authored-By: Claude <noreply@anthropic.com>
…A/B dtypes TileMma(atom_override=...) is the one way to pin an instruction by name. Its a/b/c/d are now optional checks rather than ignored: each dtype given must be one the instruction takes (for an integer row with selectable signedness that includes the unsigned twin, kept as given), and each one left out is taken from the instruction, the result dtype included. A mismatch names every wrong dtype and the constant to pass, e.g. "MMA instruction dtype mismatch for 'mfma_f32_16x16x16f16' -- a=bf16 but it takes A=f16; c=f16 but it takes C=f32 -- pass a=rocke.core.ir.F16, c=rocke.core.ir.F32, or omit a/b/c to take them from the instruction". An instruction the target does not list is refused as "MMA instruction not available on target". Tiling(atom_shape=...) takes only an (M, N, K) shape; an instruction name there is refused with the atom_override call to make instead. Without a pinned instruction, a/b/c are required. A and B may now differ: selection finds a mixed instruction such as the gfx942 fp8 x bf8 MFMA, and a pair with no instruction fails with the usual "no MMA instruction found". The recorder stores the MMA's A and B dtypes on its op, and the compute and epilogue views read operand widths and the input label from there instead of from an LDS read, so a mixed MMA is drawn with each operand at its own width and labelled "fp8e4m3/bf8e5m2". The docs show the atom_override forms and the mixed example. Tests cover the derived and checked dtypes, the mismatch, wrong-target and Tiling-name refusals, mixed and integer selection, the recorded dtypes, the views refusing an op without them, and a recorded fp8 x bf8 MMA's compute tee. Co-Authored-By: Claude <noreply@anthropic.com>
…age says TileMmaPlan.emit_op() now returns the backend op the backend coverage record names for the plan's (instruction, target) pair, fetched from core by that op id instead of searched for by operand dtypes and shape, which several backend ops can share. A pair the record does not class issuable raises NotImplementedError naming its class. An integer instruction with selectable operand signedness passes the plan's A and B dtypes, so u8 operands issue the unsigned op; a scaled instruction takes scale_format=, which the record requires when it holds more than one scale format. TileMma.emit_op() forwards scale_format. emit_op reads the committed record through committed_backend_coverage, which loads and validates it once per process; load_backend_coverage still reads a given path on every call. The driver checks, before any IR and in this order: operand dtypes; that the instruction's family is one it feeds (dense; a scaled or sparse instruction raises NotImplementedError); that an instruction whose result dtype differs from its accumulator takes one instruction along K per C subtile; that rocke issues the instruction; then the layout validation. Its result Fragment is typed by the instruction's result dtype, and the grid walk assembles a result of another dtype into a zero vector of that dtype's storage type. The plan no longer refuses such an instruction, so TileMma(..., d=BF16) on gfx1250 and atom_override="wmma_bf16f32_16x16x32_bf16" now drive it. Tests: emit_op of every issuable record entry, per variant, returns the recorded op; emit_op loads the committed record once; the refusals and their texts; the dense op chosen where scaled ops share its operands; the bf16-result tiles; WMMA, gfx11 16-bit-result, fp8 x bf8 and f32 16x16x4 tiles lower to their intrinsics, compiled where the host clang targets the arch. The traits README and the API surface doc say how issue follows the record. Co-Authored-By: Claude <noreply@anthropic.com>
The TileMma driver now slices each instruction's A and B operand out of the fragment's storage registers and hands b.mma the vector type the instruction's A or B dtype takes (mma_io.mma_operand_ir_type), bitcasting only when the two differ. A packed sub-byte operand is sliced as the whole 32-bit words that hold its elements (packed_word_range); a TF32 operand held as i32 is reinterpreted as tf32. A u8 operand is sliced as the i8 registers of the same bits, since core's zero_vec has no u8 form, and handed over as u8. Every other dtype is sliced and passed exactly as before, so its IR is unchanged. A TF32 instruction also reads F32 fragments, dropping their low mantissa bits (_MMA_TRUNCATING_SOURCES), so F32 tensors feed xf32 MFMAs with no conversion. The operand dtype error names that accepted source. Docs: the TileMma section states the operand feed and TF32 rule and how a hand-written b.mma call passes operands; the gfx942 arch doc's atom catalog gains the note on when to convert to TF32 first (round to nearest even instead of truncation). Tests: xf32 tiles on gfx942 from F32 and from TF32 tensors issue one MFMA per atom and compile; an f16 fragment on a TF32 plan is refused; a two-instruction gfx950 fp4 tile feeds words [0, 4) and [4, 8) to the f8f6f4 instruction and compiles; u8 x i8 gfx1250 tiles of two K steps, four C subtiles and both issue every instruction with a u8 A operand. Co-Authored-By: Claude <noreply@anthropic.com>
The K check compared A and B slot for slot. That is how every same-format MMA instruction pairs its operands, but not a mixed-width f8f6f4 one: its 8-bit operand holds K in two runs of 16 per lane and its 6/4-bit operand in one run of 32. The slot rule therefore refused the canonical fp8 x fp4 pair and accepted an fp4 B loaded with fp8's map, which multiplies the wrong K. mma_pair_k_aligned now takes the instruction's canonical A and B layouts as required arguments. It reads from them which A and B positions hold the same hardware K, moves B's K labels into A's slot order through that coupling and runs the unchanged three-tier comparison. For a same-format instruction the coupling is the identity, so every result and message stays as it was. mma_pair_compatible forwards its canonicals, so the driver's issue check, verify_mma_soundness and the MmaTee diagnostic all pair K through the coupling. The MmaTee diagnostic moves into MmaTee.pairing_diagnostic() with one branch: a tee without canonicals has no coupling and judges no pairing. evaluate_transform and optimize_layout take k_partner_canon and refuse a k_partner without both canonicals. An instruction whose lane holds several M blocks of A multiplies each block with the same B registers. The coupling pairs every M block of A with the lane's one B column through K, and the driver's atom-contiguity check takes such a lane's atom block as that many runs of one M row each, all in one K step. A whole wave tile compared with too few atoms, whose B lane holds several columns, still gets no coupling. The interleaved construction lays A and B out with one K map, so the dense interleave sweep now proves its mixed-width f8f6f4 pairs K-misaligned instead of passing them. Tests: the mixed-width cases on gfx950 16x16x128 canonicals through both checks; a driven gfx950 fp8 x fp4 tile that issues and compiles; the soundness gate on a recorded fp8 x fp4 pipeline; the MmaTee diagnostic; the optimizer refusal; and one test that drives every pair the backend coverage record issues for a dense instruction (each operand-dtype variant, unsigned ones included) and checks the tile lowers to the instruction's intrinsic and returns the instruction's result dtype, the instruction whose lane holds two M blocks of A included. For that instruction: the canonical pair passes both checks, a K mismatch on the second M block alone fails them, and a tile of two atoms along M, N and K issues all eight. Where the current GPU is the target, the driven gfx950 fp8 x fp4 tile and that gfx1250 tile also run on random K-dependent operands and must equal an independent numpy product exactly; elsewhere they skip. Docs: mma_is_machinery.md condition 3 (with the coupling of several M blocks), its consequences and trap 1; the interleaving design, the layout-viz skill and the API design standard show the new signatures. Co-Authored-By: Claude <noreply@anthropic.com>
… stores Add fp8e4m3, bf8e5m2 and i8 to the storage elements the tiling memory verbs move with vector accesses. A register run of these 8-bit types that is contiguous along the tensor's stride-1 axis is now read from global memory, read from LDS and written to LDS as one <N x i8> access instead of one byte at a time; the comment on the set names those three users. Tests: an fp8 or bf8 32x32x64 wave tile loads its operands as eight 8-byte global loads with no single-byte load, and an fp8, bf8 or i8 fragment staged through an LDS buffer uses only vector LDS loads and stores and compiles for gfx942 (compile-verified, not run). Co-Authored-By: Claude <noreply@anthropic.com>
cgmillette
force-pushed
the
users/cgmillette/rocke/mma-catalog
branch
from
October 31, 2026 11:18
a5e14fd to
0d432f2
Compare
cgmillette
force-pushed
the
users/cgmillette/rocke/tiling-dtypes
branch
from
October 31, 2026 11:18
423dae4 to
ec941cd
Compare
Contributor
Author
|
Restacked on the new #14329 head ( |
Contributor
Author
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14537 — IR dtypes in TileMma, a storage type under every Fragment, and MMA issue from the measured backend coverage (cgmillette, AICK-2576) — 🟡 Follow-up: restack exact, base conflicts again
Net-new: none. Utility: 🟢 Positive — typed, storage-backed fragments and record-driven MMA issue; commits unchanged, CI green. Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: range-diff since our review, merge-trees of head, base and 14328 against develop, tests at head (Copilot items, issuable pairs with compile, catalog checker), an IR probe of our N1/N2, CI logs at head. Flags whether a closer look is needed; not an approval or rejection. |
cgmillette
merged commit Oct 31, 2026
385f5f0
into
users/cgmillette/rocke/mma-catalog
92 checks passed
1 task done
shumway
pushed a commit
that referenced
this pull request
Nov 2, 2026
…R dtypes through the tiling layer (#14329) ## Motivation The tiling layer's MMA catalog (`helpers/tiling/traits/data/mma_traits.json`) is the table the tiling layer selects MMA instructions from and lays out their registers with. It had three problems: - **Rows clashed.** Rows were keyed by `op_id`, so the multi-format f8f6f4 builtins loaded "last one wins". - **Operand types were name tokens** (`fp8bf8`, `iu8`, ...), some of them wrong, instead of real dtypes. - **Layouts disagreed with themselves.** Many rows stored two register layouts that did not match each other, or that described a K order the hardware does not use. On top of the catalog, the tiling layer named MMA dtypes with strings, held every Fragment in its logical vector type, and found the backend instruction by searching on dtypes and shape, which could pick a different instruction from the catalog row it had selected. As a result: - sub-byte (fp4, fp6, i4) and tf32 operands could not be moved or fed; - mixed A/B dtypes were refused; - integer rows with selectable signedness issued the signed instruction for unsigned data; - a row whose result dtype differs from its accumulator could not be issued correctly. This PR does both halves: - **Catalog.** Each row describes one instruction variant exactly, with one hardware-exact layout, real dtypes, and real targets and intrinsics. It adds a compiler cross-check tool for the catalog, the unscaled gfx950 f8f6f4 MFMA rows, and a committed record of which rows rocke's backend issues. CTest runs the tiling package's tests. - **Dtypes.** The tiling layer speaks IR dtypes end to end. Every Fragment gets a storage type under its logical dtype, and every MMA instruction is issued as the measured backend coverage record says. **Stacked on #14328** (backend completeness); review only the last fourteen commits. The dtypes half (commits 8 to 14) came from #14537 and is now part of this one. ## Technical Details ### Catalog (commits 1 to 7) 1. **`fix(rocke): describe one MMA instruction variant per catalog row`** - Catalog schema version 2: - per-operand dtypes `a_dtype`, `b_dtype`, `c_dtype` and `d_dtype`, in canonical `core.dtypes` names; - integer operand signedness, stated as selectable or signed; - one row per (A format, B format) for each f8f6f4 builtin; - integer dims and one `layout_params` object; - a scale contract on scaled rows; - the index layout on sparse rows; - `auto_select: false` on pin-only rows. - The loader is strict: any malformed row fails with a message naming the row. A dtype must be one `core.dtypes.dtype_info` knows, spelled by its canonical name, so an alias, a typo or a retired token (`iu8`) fails. - `select()` takes `a=`, `b=`, `c=`, an optional `d=`, and `scale_block_k=`. The selection key is (target, family, A, B, C, D, shape, scale block size). - `TileMma` refuses a row whose result dtype differs from its accumulator dtype; commit 11 lifts this. - Multi-block rows now load. The encoder refuses them by name. 2. **`fix(rocke): store hardware-exact register layouts for every MMA catalog row`** - Each row stores one layout, the same in both of its forms. - The values follow the independent arbiter where one exists: the public Matrix Instruction Calculator, the public CDNA4 ISA tables, or rocKE core's lane maps. Otherwise they follow Composable Kernel's parameters. 3. **`fix(rocke): list the real gfx targets and LLVM intrinsic of every MMA instruction`** - Every row lists real target ids, never family names, and every listed target is a backend target. - Every row names its LLVM intrinsic. - Targets that are excluded are recorded with a reason. 4. **`feat(rocke): add a compiler cross-check tool for the MMA catalog`** - `tools/check_mma_catalog.py` asks the installed clang four questions about the catalog: on which targets each row's builtin compiles and links, which LLVM intrinsic it lowers to, which scale formats a scaled builtin accepts, and whether every matrix builtin that compiles on a backend target has a row. - It exits 1 on any disagreement. It names what the clang cannot judge without failing. - Its unit tests run no compiler and stay in the source tree. The README's edit procedure runs the tool. 5. **`feat(rocke): add the unscaled gfx950 f8f6f4 MFMA variants to the MMA catalog`** - One dense row per (A format, B format) pair at 16x16x128 and 32x32x64. They name the scaled builtin, which the compiler issues unscaled when both scales are zero. - Their layouts equal their scaled twins'. 6. **`feat(rocke): record which MMA catalog instructions rocke's backend issues`** - `traits/data/backend_coverage.json` records one measured class and the matched backend op for every (row, target) pair. It is read through `load_backend_coverage()`. - Only `issuable` means the backend issues the row's instruction there. The other classes say why not, for example `no_backend_op`, `other_instruction`, `multi_block`, `operand_form_words_only` or `lane_map_differs`. - `helpers/mma_io.mma_operand_ir_type` is the one home of the operand-form rule. - A test measures the record again and requires it to equal the committed file. 7. **`test(rocke): run the tiling package's tests in CTest`** - New CTest entry `rocke_tiling_pytest`. It runs `rocke/tests/helpers/tiling` from the installed package. It is listed in the quick and ffm-quick categories. - The interleaved GEMM demo's launching tests skip unless the current GPU is a gfx90a. This includes a host whose HIP runtime loads but sees no device; before, they failed there. ### IR dtypes through the tiling layer (commits 8 to 14) 8. **`feat(rocke): hold Fragment registers in the storage type of their dtype`** - Storage per dtype: the logical vector for arithmetic dtypes, i32 for tf32, and whole i32 words for packed sub-byte dtypes, packed low bit first. - Load, store and fill move storage units in global memory and LDS; a packed access moves whole 32-bit words. - Refusals name the fix. - IR for byte-or-wider dtypes is unchanged. 9. **`feat(rocke): take IR dtypes for A, B, C and the result in TileMma`** - `a/b/c` and a new `d=` are scalar IR types, e.g. `rocke.core.ir.F16`; a string is refused with the constant to pass. - A selection miss lists what the target offers. - Every in-tree `TileMma` call, including the gfx942 and gfx950 target coverage develop added to the MMA operation tests, passes IR types. 10. **`feat(rocke): pin MMA instructions by full signature and accept mixed A/B dtypes`** - `atom_override=` is the one way to pin a row. Its dtypes are optional checks; any left out are taken from the row. - Mixed A/B instructions (e.g. gfx942 fp8 x bf8) select and draw at each operand's own width. 11. **`feat(rocke): issue MMA instructions as rocke's measured backend coverage says`** - `emit_op()` returns the backend op the coverage record names for the (row, target) pair, using the author's A/B dtypes for selectable-signedness rows and `scale_format=` for multi-format scaled rows. - A pair that is not issuable raises with its class. - The driver types its result by the row's result dtype. This lifts commit 1's refusal of rows whose result dtype differs from the accumulator. 12. **`feat(rocke): feed MMA operands from their fragments' storage registers`** - Operands are sliced from storage registers and bitcast only to the instruction's operand form: packed words for sub-byte dtypes, i32 reinterpreted as tf32. - F32 fragments feed xf32 MFMAs, which truncate the low mantissa bits; the docs say when to convert to TF32 first. 13. **`fix(rocke): pair A and B K through the instruction's canonical coupling`** - Mixed-width f8f6f4 instructions hold K differently in A and B. The K check now pairs positions through the instruction's canonical layouts, so the canonical fp8 x fp4 pair is accepted and a mis-loaded operand is refused. - Same-format instructions are unchanged. 14. **`feat(rocke): move fp8, bf8 and i8 fragments with vector loads and LDS stores`** - A contiguous register run of these 8-bit types moves as one `<N x i8>` access instead of one byte at a time, in global memory and LDS. No core or C++ engine file changes, and no emitted kernel IR changes, so byte identity is unaffected. ## Test Plan - **Catalog load:** a strict-schema test for every refusal, including an unknown dtype name and a retired token; selection tests on every target, including the scaled block-size key and the multi-block refusal. - **Layouts:** a layout check for every row (the two forms agree and match their arbiter); the interleave sweep over every dense row. - **Targets:** every listed target is a backend target, excluded targets never appear, and every intrinsic is an `llvm.amdgcn.*` name. - **Cross-check tool:** unit tests that classify hand-written compiler answers against real catalog rows. - **Backend coverage:** the record is measured again and compared with the committed file; operand counts, the scaled pairs, the result-dtype pairs and the reader's errors are pinned. - **CTest:** the tiling tree run from a fresh `cmake --install` with the entry's command and `PYTHONPATH`, with no GPU visible. - **Storage:** the storage table and every refusal; bit-exact GPU round trips on gfx90a for tf32 through LDS and for fp4, fp6 and i4 in gfx950's canonical f8f6f4 layout through global memory and LDS, including a clipped packed window. - **Dtypes:** IR-dtype refusals with the constant name of every scalar type, derived and default dtypes, selection by IR type on gfx90a, gfx942 and gfx950, wrong-target pins refused on gfx90a and gfx942, mixed and integer selection. - **Issue:** `emit_op` of every issuable coverage-record entry, per operand-dtype variant, returns the recorded op; every refusal text. - **Drive:** a single-instruction TileMma for every issuable dense (row, target) pair lowers to the row's intrinsic, compiled where the host clang targets the arch. This covers WMMA, gfx11 16-bit result, fp8 x bf8, xf32 from F32 and TF32, fp4 words, and bf16-result tiles. - **K coupling:** canonical mixed pairs accepted, mis-loaded operands refused. A numeric check of a driven gfx950 fp8 x fp4 tile runs where the current GPU is a gfx950; the gfx1250 tile's numeric check is skipped until it has run on gfx1250 hardware. ## Test Result - Byte identity green at llvm20, llvm22, llvm23 and llvm24. - Differential goldens (ll at every flavor, canonical IR) and the representative IR golden: no change. - ctest passes. - Portable IR against a freshly built shared engine passes. - The touched test files, the whole tiling test tree and the in-tree test files develop's gfx942 and gfx950 coverage change touched pass with both engines in strict mode. - Full platform suite: no failures. The other platform test files: only the ones develop also fails. - Library smoke: no new failures. - pre-commit passes. - Every commit builds, passes ctest and passes its own tests on its own. ISSUE ID : AICK-2572 ISSUE ID : AICK-2576 ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The tiling layer named MMA dtypes with strings, held every Fragment in its logical vector type, and found the backend instruction by searching on dtypes and shape. That last part could pick a different instruction from the catalog row it had selected. As a result:
This PR makes the tiling layer speak IR dtypes end to end. Every Fragment gets a storage type under its logical dtype, and every MMA instruction is issued as the measured backend coverage record says.
Stacked on #14329 (MMA catalog). Review only the last seven commits.
ISSUE ID : AICK-2576
Technical Details
Seven commits:
feat(rocke): hold Fragment registers in the storage type of their dtypefeat(rocke): take IR dtypes for A, B, C and the result in TileMmaa/b/cand a newd=are scalar IR types, e.g.rocke.core.ir.F16; a string is refused with the constant to pass.feat(rocke): pin MMA instructions by full signature and accept mixed A/B dtypesatom_override=is the one way to pin a row. Its dtypes are optional checks; any left out are taken from the row.feat(rocke): issue MMA instructions as rocke's measured backend coverage saysemit_op()returns the backend op the coverage record names for the (row, target) pair, using the author's A/B dtypes for selectable-signedness rows andscale_format=for multi-format scaled rows.feat(rocke): feed MMA operands from their fragments' storage registersfix(rocke): pair A and B K through the instruction's canonical couplingfeat(rocke): move fp8, bf8 and i8 fragments with vector loads and LDS stores<N x i8>access instead of one byte at a time, in global memory and LDS.No core or C++ file changes, so byte identity is unaffected.
Test Plan
emit_opof every issuable coverage-record entry, per operand-dtype variant, returns the recorded op; every refusal text.Test Result
Submission Checklist
🤖 Generated with Claude Code