Repository navigation
feat(rocke): add independent block-scale formats through MMA contracts - #12166
Merged
tenpercent merged 40 commits intoOct 15, 2026
Merged
Conversation
github-actions
Bot
added
documentation
project: hip-kernel-provider
rocke: platform
6 tasks done
tenpercent
changed the title
tenpercent
changed the base branch from
users/mpodkory/rocke/fp6-scale-formats
to
users/mpodkory/rocke/mixed-scaled-gemm
September 17, 2026 22:39
This was referenced Sep 17, 2026
Report complete quant type aliases while preserving scalar quantization restrictions.
Keep empty descriptors valid while rejecting fragment loads before IR emission in both engines. Cover empty shapes and nonempty controls in Python and native tests.
Validate complete static fragment displacements before emitting i32 offsets in both engines. Resolve CTest readiness from registered commands and cover multi-config execution and failure propagation.
Size temporary load-pointer storage by addressable units so padded slots cannot overrun the array. Preserve IR order and add mirrored transport coverage for a layout requiring more loads than logical elements. Simplify scale-width queries while retaining the shared packing adapter, and clarify the contiguous-K matrix layout contract.
tenpercent
marked this pull request as ready for review
October 14, 2026 00:10
Preserve generated gfx1250 catalogs alongside gfx950 scaled MFMA lookup. Carry independent scale references into the shared verifier and retain complete differential enumeration coverage.
cgmillette
approved these changes
Oct 14, 2026
cgmillette
left a comment
cgmillette
left a comment
Contributor
There was a problem hiding this comment.
PR #12166 — add independent block-scale formats through MMA contracts (tenpercent, AICK-2284) — 🟢 ACCEPT: well-gated; two nits
| Prior item | Status |
|---|---|
Copilot (10-09): --output-dtype ignored |
🟢 FIXED f11189fa — passes args.output_dtype (verified) |
| Copilot (missed): invalid CLI combos traceback | 🟢 FIXED a1495640 — is_valid_spec → parser.error |
| Copilot: README support summary | 🟢 FIXED — thread resolved |
Copilot (Rule 9): limit only in run_diff |
🟢 FIXED 738b6ff6 — shared config_limit() in both drivers |
Copilot (missed): ir_artifact_diff lacks gfx1250 |
🟡 OPEN — no gfx1250 prefix/candidate (pre-existing); PR enlarges loop |
Copilot (10-13): new fields break positional prefetch |
🟢 FIXED 5027f0de — appended after prefetch; tests pass (ran) |
| Copilot (10-14): fixture treats unknown scale as E5M3 (Rule 4) | 🟡 OPEN — no reply; decode_scale rejects it later |
| Bots / CI | 🟢 Superbuild + pre-commit green; TheRock pending; merge clean, no new failures (ran) |
| # | Sev | file:line | Finding |
|---|---|---|---|
| N1 | 🟡 | arch/data.cpp, arch_specs.json, test_mma_catalog_lookup.cpp (+2) |
Conflicts with open #13588 in 5 files; agree land order |
Utility: 🟢 Positive — legal per-operand scale formats, generated catalogs, exhaustive tests.
Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good
CongMa13
approved these changes
Oct 14, 2026
tenpercent
enabled auto-merge (squash)
October 14, 2026 22:42
14 of 17 tasks
CongMa13
added a commit
that referenced
this pull request
Oct 15, 2026
Brings in the independent block-scale formats for gfx1250 scaled WMMA (#12166), which landed on develop first. Three conflicts, all beside the catalog rework that change made: - cpp/core/arch/data.cpp: develop moved rocke_arch_mma_c_frag_len below the generated gfx1250 catalog. The emulated-atom lookup (an emulated atom borrows its physical atom's fragment row, one hop) now lives in that relocated function; the row-lookup helper stays next to the fragment table. - tests/core/test_mma_catalog_lookup.cpp: the layout canary and the three emulated-row tests are kept next to develop's scaled-WMMA test; the duplicate include block is dropped. - tests/core/test_mma_frag_tables.py: the import list is the union of both sides. ROCKE_ABI_VERSION is untouched by that change (2 on develop), so the rocke_mma_op_t append and the bump to 3 stand as they were. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
TheRock Submodule Bump ActivityNewest first
|
18 of 23 tasks
cgmillette
pushed a commit
that referenced
this pull request
Nov 4, 2026
…12167) Add an untuned, reusable packed-input NVFP4 GEMM builder and a focused correctness example for gfx1250. The kernel consumes E2M1 FP4 matrices, E4M3 block scales for each group of 16 values along K, and runtime FP32 tensor dequantization factors; it produces BF16 or FP16 output. This PR delivers: - The exported `NvFp4GemmSpec` and `build_nvfp4_gemm` API, with validation, grid and signature helpers, built on the shared block-scaled GEMM implementation. - A standalone example and GPU verifier that launch the builder directly and compare outputs with an independent NumPy reference. - Tensor-factor application in the FP32 epilogue, with an optimization barrier that preserves FP32 rounding before FP16 conversion, plus matching Python/native HIP inline-assembly validation and paired coverage for isolated tensor epilogues. ### Builder and verification flow ```mermaid flowchart TD S["NvFp4GemmSpec: shape and output dtype"] --> B["build_nvfp4_gemm: FP4 / E4M3 / SCALE16"] B --> K["KernelDef with tensor-scale epilogue"] K --> L["LLVM lowering: Python / native C++"] K --> H["HIP source lowering: Python"] L --> C["COMGR compilation"] H --> HC["HIP compilation"] C --> O["HSACO"] HC --> O O --> V["Standalone verifier: pack arguments and launch"] I["Packed E2M1 matrices, E4M3 block scales, FP32 tensor factors"] --> V I --> R["Independent NumPy reference"] V --> G["BF16 / FP16 GPU output"] G --> Q["Compare outputs"] R --> Q ``` The factors are runtime launch arguments. The kernel computes their product in FP32, applies it to the FP32 accumulator, and then converts to the output dtype. In the validated GPU campaigns, COMGR checks Python/native LLVM byte identity; the HIP route compiles Python-generated HIP source. Isolated native HIP barrier acceptance and malformed-attribute rejection are covered separately by the host CTest and paired emitters. Complete native NVFP4 HIP compilation remains outside the supported API scope. ### Scope The delivered entry points are the exported builder and standalone example/verifier. GEMM dispatcher registration, request selection, and production routing are intentionally deferred: this kernel is not tuned, and the validation establishes correctness only for the stated bounded fixtures. Dispatcher integration requires separate work on the packed-input request contract, tuning, and selection-path validation. This PR makes no performance claim. ### Implementation details Tensor scaling is admitted only for FP4/E4M3/SCALE16. The launcher and reference validate factor shape and real numeric dtype before FP32 conversion, then require finite converted values. String, complex, boolean and object arrays are rejected; the launcher rejects missing, malformed or unexpected factors before GPU allocation. The generic block verifier rejects tensor-enabled specs before device discovery or compilation; the NVFP4 verifier supplies the factors and tensor-aware reference. Unsupported LLVM emission flavors are checked with backend-compatible exception assertions and the expected diagnostic. Merge develop's completed scale-format prerequisite #12166, retaining its canonical scale-aware catalog, shared matrix/scale packing, independent operand storage and default-off prefetch. NVFP4 HIP compilation uses Python-generated source; complete native NVFP4 HIP compilation remains outside this API's supported scope. Python and native HIP now agree on single-output/void inline assembly, including boolean sideeffect validation, string clobbers, and an absent or empty clobber for pure optimization barriers. These isolated lowering checks do not establish complete native NVFP4 compilation or new GPU numerical coverage. ### Current coverage and lint follow-up at `e7f2ac9f6a1a` `3532d6e59e29` addresses N1 and the related N3 in the subsequent review; `e7f2ac9f6a1a` clears the CMake lint findings exposed by its new registrations. GCC 13 reproduced a C-builder evaluation-order difference in HIP emitter configurations 0–7: the conversion was recorded before the thread ID, unlike Python. Both builder calls are now explicitly sequenced. The new `rocke_optimization_barrier_hip_pair` CTest executes the actual prebuilt C emitter and the selected Python emitter for all 12 HIP configurations and their range boundary. It is registered in build-tree and installed layouts and enrolled in quick/FFM categories. Binding comparisons now use the existing exact engine-parity helpers. - [x] The former GCC binary fails the new comparison at configuration 0; the corrected binary passes all 12 configurations and the range boundary. GCC 13 and Clang 24 CMake builds pass the real-emitter CTest; the relocated installed CTest also passes. - [x] Source checks: **790 passed per backend**, in Python, strict C++ and strict `both`. Minimal checks: **771 passed / 19 optional-dependency skips per backend**. - [x] All four LLVM parity lanes, older-lane companions and representative goldens pass. Native CTests: **35 passed**, plus one named architecture-domain compiler-provenance skip. - [x] Kernel builders and lowering emission are unchanged. The four NVFP4 fixtures retain exactly the same names, LLVM23/24 hashes and Python HIP hashes as `5acc3fa12d92`; no golden was re-blessed. - [x] **Full PR-range `pre-commit` and whitespace checks pass**, including `cmake-lint`. All 207 findings were fixed without disabling rules: private build variables use compliant names and long commands/strings are wrapped. The generated installed CTest manifest is byte-for-byte unchanged, and all 36 source CTest names, commands and properties are preserved. GPU-reference and configuration-contract checks passed (21 tests, one capability skip); source and relocated real-emitter tests passed again under GCC and Clang. - [ ] The earlier full-runner baseline failures remain unresolved. The lint and coverage gates are green; this is not a merge-readiness claim. N2 shared reference/runner refactoring and N4 shared validation/error-style suggestions are acknowledged and deferred to separately scoped maintenance/API changes, not claimed fixed. See the [explicit author dispositions](#12167 (comment)). #### Additional gfx1250 evidence at `5acc3fa12d92` A completed SSH validation campaign using matched ROCm `10.3.0a20261102` compiler/HIP-runtime components returned **16 NVFP4 campaigns / 448 exact comparisons**, with zero skips, across LLVM23/24 emission, COMGR/HIP, BF16/FP16 output and the bounded 16×16×128 / 32×48×256 fixtures. It also passed **24 scalar/barrier numerical checks**, the focused 783-test host cohort in each backend mode, native CTests and the 12 HIP emitter pairs. COMGR checks Python/native LLVM identity before compilation; the HIP route compiles Python-generated source, not complete native NVFP4 HIP. Required dynamically loaded runtime components were added before accepting this campaign. Existing ROCm installations and default symlinks were not replaced. The completion status and declared assertion summary were retained, but the remote workspace disappeared before detailed logs could be recovered; this limits the available replay artifacts. This GPU evidence belongs to `5acc3fa12d92`, not a new GPU execution of the test/build-wiring-only follow-up. The completed campaign on the passing gfx1250 setup supplies the requested bounded hardware acceptance evidence. A separate setup returned launch faults and is retained only as a diagnostic result, not as a failure of the passing campaign or a claim that the PR caused those faults. Its older-ROCm control passed four COMGR campaigns (112 exact comparisons) and 12 scalar/barrier checks but failed the four HIP campaigns. Neither agreement nor this bounded acceptance establishes arbitrary-input correctness or universal environment compatibility. ### Host validation at `5acc3fa12d92` (before the additional SSH campaign) Follow-up to `9b690231cf47`: `912d776b8a33` fixes HIP template escaping, optional host-test dependencies and numerical-verifier subprocess identity; `5acc3fa12d92` separately hardens the pre-existing LLVM `sideeffect` admission gap. No existing golden was changed in these follow-ups. Both HIP formatters preserve assembly bytes in C++ string literals for void and single-output forms, with paired multiline/quote/backslash coverage and direct/serialized control-character regressions. Both LLVM handlers reject present non-boolean `sideeffect` attributes for void, single-output and multi-output assembly, while preserving absent/false/true behavior. The NVFP4 host tests use a scoped `ml_dtypes` fixture, not a module-wide skip; malformed-factor checks stay enabled without that dependency. Required gfx1250 numerical validation still fails on a missing dependency. The numerical subprocess uses the selected-package launcher; its host regression executes the real launch prefix against a shadowing working directory without claiming a GPU comparison. Fresh Release engine/binding and standalone installed/relocated checks used ROCm `10.3.0a20261102` / LLVM24, Python 3.12.12, NumPy 2.5.4, pytest 9.2.0 and ml_dtypes 0.6.0, without Torch. The minimal environment used the same NumPy/pytest versions and no ml_dtypes. Source commands ran from `platform/`; relocated checks used fresh platform/library wheels and the matching installed native binding, with no editable source imports. - [x] Focused source fixes: **783 passed per backend**, zero skips, in Python, strict C++ and strict `both`. The broader NVFP4/barrier/scale-format/prefetch/registry/golden cohort passed **1,766 tests plus 37 subtests per backend**, zero skips. - [x] Real minimal source and relocated environments: **764 passed / 19 optional-dependency skips per backend**. Builder, identity, ABI-admission, IR and malformed-factor tests remain enabled. Wheel-only default fallback: **595 passed / 188 explicitly unavailable-native skips**, with the native binding absent. - [x] Representative golden gates passed at LLVM20/22/23/24. Canonical Python/native byte identity passed at every flavor, including the older-lane LLVM23 companions: LLVM20/22 each cover 89 families / 2,141 configurations, and LLVM23/24 each cover 94 families / 2,228 configurations; zero mismatches. - [x] Full-runner LLVM24 corpus acceptance: **364/364 modules compiled and linked**, no new failures. - [x] Native CTests: **34 passed / zero failed**, plus one named architecture-domain reproducibility skip because the measured compiler identity differs. - [x] **Twelve Python/C HIP mini-pairs** match byte-for-byte. All 24 emitted source variants compile and link for gfx950/gfx1250; four deliberately unescaped multiline controls fail as expected. All four complete Python-generated NVFP4 HIP fixtures compile. - [x] Local **gfx90a** GPU smoke checks: four multiline void/single-output payload checks from Python/native HIP sources preserve all 64 lane payloads exactly, including floating special-value bit patterns. The existing 12 scalar/barrier numerical checks pass in each of Python, strict C++ and strict `both`. These are not gfx1250 NVFP4 numerical results. - [x] Relocated focused replay: **1,764 passed plus 37 subtests per backend**, zero skips, with exactly two named unrelated example-asset tests deselected. The original broad selection reproduces both failures, and the unchanged starting-head test file reproduces them independently. Installed barrier/golden CTests: **2/2 passed**. - [x] Full pinned-base-to-head repository hooks and whitespace checks passed. - [ ] The full default runner remains **RED**: strict C++ reports **11 failed / 6,922 passed / 851 skipped**; strict `both` reports **2 failed / 6,922 passed / 851 skipped**. Every failing selector reproduces on an independent refreshed build of starting head `9b690231cf47` in the same environment: nine C++ F64 exception-type subtests, the strict-fallback test and the explicit-COMGR compiler-selection test. No new failing case was found. - [ ] No current-head gfx1250 NVFP4 numerical campaign ran. Earlier bounded gfx1250 results remain attributed to their recorded source revision. Complete native NVFP4 HIP compilation remains outside this API's supported scope; the known native `0f` formatting limitation is not changed by these follow-ups. #### Shared symbol-rename inventory The preceding shared `BlockScaledGemmSpec` rename intentionally reaches **MXFP4/6/8, mixed operands, alternate scale formats, legacy WMMA and fused-MoE's block-scaled GEMM stages**, not only NVFP4. At this head, independent comparisons against pinned develop confirm unchanged normalized LLVM/HIP for the **82 affected focused golden cases plus two legacy-WMMA controls**, and all four MXFP8 representative cases at LLVM23/24. The change is symbol-only for those checked paths; no new numerical or performance claim is inferred. Already regenerated committed artifacts are the focused [scaled-WMMA](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_scaled_wmma_llvm23.json), [mixed-scaled-WMMA](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_mixed_scaled_wmma_llvm23.json), [scale-format](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_scale_formats_llvm23.json) goldens and the [representative GEMM golden shard](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/golden/rocke_representative_ir_sha256/gemm.json). Any externally generated name-keyed sources, binaries or manifests from before the rename must be rebuilt to use the new entry-point names. These remediation commits do not rename kernels again or re-bless those goldens. ### Validation of `9b690231cf47` (prior integration) Merged develop `141e2e47b38b`. The LLVM24 golden has its own four NVFP4 records, with no LLVM24-to-LLVM23 alias. Kernel symbols use codegen specialization identity rather than workload-size tags: M changes only the grid; baked N addressing and K unrolling remain distinguished. All affected goldens were refreshed after verifying symbol-only changes, including 82 sibling cases. Commands ran from `platform/` with a fresh Release engine/binding, ROCm `10.3.0a20261102` / LLVM24, Python 3.12.12, NumPy 2.4.6, pytest 9.2.0 and ml_dtypes 0.6.0. - [x] Focused source suites: **1,231 tests plus 37 subtests per backend**, zero skips, in Python, strict C++ and strict `both`. - [x] Representative golden and byte-identity gates at LLVM20/22/23/24, including older-lane companion checks. - [x] LLVM24 corpus acceptance: **364/364 modules compiled and linked**. - [x] Eight Python/C HIP mini-pairs match byte-for-byte and compile for gfx950/gfx1250. All four complete Python-generated NVFP4 HIP fixtures compile. - [x] Fresh installed/relocated replay: **1,229 tests plus 37 subtests per backend**, zero skips; two unrelated example-asset tests were deselected after reproducing their failures with the original test file. Installed golden/barrier CTests: **2/2 passed**. - [x] Native CTests: **33 passed, zero failed**, with one named architecture-domain reproducibility skip because the measured compiler revision differs. - [x] PR-range repository hooks and whitespace checks. - [ ] Full default runner remains **RED**: strict C++ reports **11 failed / 6,395 passed / 851 skipped**; strict `both` reports **2 failed / 6,395 passed / 851 skipped**. All remaining failures reproduce on an independent fresh build of the pre-fix integration `3be7531f`: nine C++ F64 exception-type subtests, the strict-fallback test and the explicit-COMGR compiler-selection test. No new failing case remains. - [ ] Complete native NVFP4 HIP compilation still fails on the existing `0f` literal formatting issue and remains outside the supported API scope. Isolated native barrier support is not substituted for complete-kernel acceptance. - [ ] No new current-head gfx1250 GPU numerical campaign ran. Earlier bounded GPU evidence remains attributed to its recorded source revision; host hashes and compilation do not establish GPU correctness. ### Historical native HIP scope reduction at `ad36142c7dbedebf026fa5bd6f01b294cb480d50` Removed this PR's native HIP optimization-barrier support and its HIP paired-emitter mode, optional pytest wrapper, test-specific environment variable, extra CMake target/install wiring, and runner plumbing. Python HIP compilation and Python/native LLVM lowering remain. The existing `rocke_optimization_barrier` CTest now requires native HIP rejection of the valid FP16 tensor epilogue, directly and after serialization, on gfx950/gfx1250 with absent/empty clobbers; the same epilogue must still lower through LLVM at every supported flavor. These results are retained only as historical evidence for this revision; the current isolated-native-HIP behavior and validation are recorded above. - [x] Fresh Release native engine/binding build with ROCm `10.3.0a20261029` / AMD Clang LLVM24, Python 3.12.12, NumPy 2.4.6, pytest 9.1.1, and optional `ml_dtypes`. - [x] Native CTests: **33 passed, zero failed**. The separate architecture-domain reproducibility gate was excluded, not rerun. - [x] Focused source NVFP4/barrier/example/runner/registration/representative-golden checks: **246 passed plus four subtests, zero skips**, in each of Python, strict C++ and strict `both`. - [x] Standalone installed/relocated package: native rejection CTest passed; **209 focused tests plus four subtests, zero skips**, in each backend mode. Imports resolved from the relocated package and binding. - [x] Canonical LLVM byte-identity gates passed at LLVM20/22/23/24. Older lanes cover 88 eligible families / 1,985 configurations each (1,714 identical, 271 both-rejected), with their LLVM23 companion lanes; LLVM23/24 cover 93 families / 2,072 configurations each (1,900 identical, 172 both-rejected). Zero mismatches. NVFP4 builder agreement is covered separately by the strict registry tests. - [x] All four NVFP4 names, LLVM hashes and Python HIP hashes are unchanged from the preceding implementation. Full FP16 NVFP4 serialized IR now rejects native HIP with the expected inline-assembly diagnostic. The new rejection test fails against the old native implementation at the expected assertion, establishing a negative control. Existing BF16 native lowering behavior is not disabled; it is not the verifier's supported HIP compilation route. - [x] Documented direct `tests/test_rocke.py` command passed: **368 tests, 132 skips** on its Python-only path. File hooks, full pinned-develop-to-head PR-range hooks, relative-path guard, and whitespace checks passed. No goldens changed. - [ ] No full default runner, compiler-acceptance corpus, architecture-domain reproduction, or GPU numerical campaign was rerun for this scope reduction. Prior bounded GPU results remain attributed to their original source revisions. ### Develop integration at `94475ba54d94934251490a807b917d587c0cd9f0` Merged develop `bec4b98a286f68ef37c4bb1036d78d7f88719180` into `f903028e5eadbdeabecf4bc70e83727a85624727`. The sole conflict was the representative golden. Regeneration from the complete merged registry preserves every develop record and all four NVFP4 outcomes; the four focused NVFP4 LLVM23 hashes are unchanged. LLVM23 has 335 successful cases; LLVM20/22 each have 307 successful cases and 28 expected rejections. LLVM24 shares the explicit LLVM23 emission-profile alias from develop. - [x] Fresh Release native engine and Python binding build with `rocm-nightly/latest`, resolved to `10.3.0a20261029`, using AMD Clang/COMGR LLVM24. - [x] Focused registry/NVFP4/barrier/golden-comparator tests: 233 passed plus four subtests in each of Python, strict C++ and strict `both`. The eight initially skipped direct HIP text comparisons passed separately with the fresh native barrier emitter selected. - [x] Fresh representative golden check passed at LLVM20, LLVM22, LLVM23 and LLVM24. - [x] Python/native LLVM parity: LLVM20/22 each cover 85 eligible families and 1,825 configurations (1,584 byte-identical, 241 both-rejected); LLVM23/24 each cover all 88 families and 1,894 configurations (1,753 byte-identical, 141 both-rejected). Zero mismatches. The older lanes exclude `conv_direct_dgrad_wmma`, `conv_direct_fwd_wmma` and `conv_fwd_small_wmma`, whose fixtures require LLVM23 operations; all three are covered by the full LLVM23/24 runs. The unfiltered older-lane checks also establish matching rejection for the two forward families. - [x] Native CTest: 33 passed, one architecture-domain reproducibility skip, zero failed. PR-range repository hooks, relative-path guard and whitespace checks passed. - [ ] Full default runner, compiler-acceptance corpus, installed/relocated replay and GPU numerical execution were not rerun for this golden refresh. Earlier results remain attributed to their recorded revisions. ### Documentation follow-up at `2f004ba50231ca099a47ee83ab43c0af19d1eee0` Added the [NVFP4 API reference](https://github.com/ROCm/rocm-libraries/blob/2f004ba50231ca099a47ee83ab43c0af19d1eee0/dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/gemm.md#gfx1250-nvfp4-gemm) and instance-catalog entry. The reference covers all five exported APIs, packed input/scale layouts, runtime tensor-factor ABI, shape/dtype constraints, LLVM emission flavor, and standalone scope, and links the detailed example input contract. This commit changes only two documentation files. - [x] All four introduced relative links and anchors resolve; documentation whitespace checks pass. - [x] The documented Python example builds; package exports, the ten-argument ABI, and launch grid/block agree with the source API. - [x] Repository hooks passed on the two changed files and over the complete pinned-develop-to-head PR range; PR-range `git diff --check` passed. No native build, compiler acceptance, installed replay, full runner, or GPU numerical lane was rerun for this documentation-only commit. Prior host/native/GPU results remain attributed to the source revisions below, including the reported broad baseline failures and unrun lanes. ### Develop integration at `9ae4868fe069ed91aac0de1e71e2c79fbb4f3c9b` Merged develop `ed481508f54b3ba6b3141d06e8dcd8bdce2a8c8f`. Resolved the test README conflict by retaining the representative-versus-paired-emitter coverage explanation alongside upstream inventory/runner updates. Regenerated the representative golden from the merged registry and regenerated the four focused NVFP4 hashes. Every current develop record and all four prior NVFP4 outcomes are preserved; the focused hashes are unchanged. - [x] Fresh Release native engine/binding build; **31/31 CTests passed**, including architecture-domain reproducibility. - [x] Focused source tests in Python, strict C++, and strict `both`: **223 passed plus four registry subtests per mode, zero skips**. - [x] Representative golden gate at every flavor: **288 successful LLVM23 cases; 278 successful cases and 10 expected rejections at LLVM20/22**. - [x] Canonical LLVM20/22 byte-identity gates, including their companion LLVM23-only lanes, passed. Complete LLVM23 dashboard: **84 families / 1,854 configurations**, 1,737 byte-identical and 117 both-rejected, zero mismatches. NVFP4 builder agreement is covered separately through strict serialized-IR tests. - [x] The documented direct test-file command passed: **362 tests, 124 skips** (native checks unavailable on that direct command's Python-only path and optional Torch). - [x] Relative-path guard, emitted-IR validity and architecture-domain warning gate passed. IR validity reports **22 unvalidated cases**: 10 require newer LLVM and 12 an untargetable architecture. - [x] Repository pre-commit hooks over `origin/develop..HEAD` on this committed head and PR-scope whitespace checks passed. - [ ] The full default runner is **RED**: its strict C++ pytest pass reports **15 failed / 5,357 passed / 848 skipped**; its strict `both` pass reports **6 failed / 5,357 passed / 848 skipped**. Every failing item reproduces on a separate fresh build of pinned develop with the same environment and working directory. These are baseline failures, not an NVFP4 regression. - [ ] Current-head GPU numeric execution and installed-artifact replay were not run. This host has no `/dev/kfd`; earlier gfx1250 GPU evidence remains attributed to its original source revision below. Commands ran from `platform/`, with Python 3.12.12, NumPy 2.4.6, pytest 9.1.1, optional `ml_dtypes` available for the focused/full runner suites, and no Torch. The host COMGR resolves LLVM20; the direct/default runner commands use that compatible automatic flavor. Offline NVFP4 emission tests explicitly use LLVM23. ```sh PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python:../library ROCKE_BACKEND=python python tests/test_rocke.py # Full runner, with no stages disabled; the build-root argument selects the fresh build. PYTHONDONTWRITEBYTECODE=1 \ PYTHONPATH=python:../library:<fresh-build>/cpp/bindings:<ml-dtypes-location> \ ROCKE_CPP_STRICT=1 CMAKE_BUILD_PARALLEL_LEVEL=8 \ OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 \ python tests/run_all.py --build-root <fresh-build> ``` The baseline failures are nine HIP F64 exception-type subtests in C++ mode, a fallback test under strict-mode settings, two stale gfx1250 enumeration expectations, a subprocess import with the runner's relative `PYTHONPATH`, MX GEMM concrete-IR JSON replay, and an installed-environment ordering expectation. The latter six fail in both backend modes; the nine F64 subtests pass in `both`. All were replayed on develop `ed481508f54b`. The host example test exercises `nvfp4_gemm.main`, and the GPU test invokes `nvfp4_gemm_verify` via subprocess. Their presence does not establish current-head GPU execution. The field-manual follow-up adds the NVFP4 API and input contract to the GEMM reference and instance catalog; it changes documentation only. Validation in this develop-integration section remains attributed to `9ae4868fe069`, including the broad baseline failures and GPU/installed replay limitations. Review-thread disposition is recorded in the individual replies. ### Develop integration at `599f89a22801faf43f1ad9a7bee88faab236b37c` Merged develop `3ebcd68817f0af480ee9926d5b12174edc2a970d`. The only textual conflict was the representative golden summary; regenerating from the merged registry preserves every develop record and all four prior NVFP4 outcomes at each flavor. Shared lowerers merged automatically. The documentation now gives the focused host commands, distinguishes representative binding coverage from the paired-emitter dashboard, and clarifies compiler-flavor and GPU evidence requirements. - [x] Fresh Release native engine/binding build; 30/30 CTests passed. - [x] Focused source suite in Python, strict C++, and strict `both`: 575 passed and 5,342 subtests passed per mode; 14 optional Torch tests skipped per mode. - [x] Representative golden gate: LLVM23 has 272 successful cases; LLVM20/22 each have 264 successful cases and eight expected rejections. All four NVFP4 cases emit at LLVM23 and reject at older flavors. - [x] Full paired-emitter differential gate: 83 families / 1,830 configurations per flavor, zero mismatches. LLVM20/22 each have 1,624 identical and 206 both-rejected configurations; LLVM23 has 1,721 identical and 109 both-rejected. NVFP4 builder agreement is established separately by strict serialized-IR registry tests. - [x] Direct README test-file entry point passed: 362 tests reported, with 124 skips (110 native-extension checks and 14 optional Torch tests). The native-enabled focused matrix above covers the binding checks. - [x] Runner relative-path guard and all native CTests passed with the other runner stages explicitly disabled, as shown below. - [x] Repository hooks over the full pinned-develop-to-head PR range and whitespace checks passed. The direct test-file command, run from `platform/`, was: ```sh PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python:../library ROCKE_BACKEND=python python3 tests/test_rocke.py ``` The runner command, with a fresh full native build and its binding on `PYTHONPATH`, was: ```sh python3 tests/run_all.py --no-gate --no-pytest --no-both --no-ir-validity --no-arch-domain --build-root <fresh-build> ``` These commands do not claim a full default `python tests/run_all.py` run: pytest and all-flavor differential checks ran separately; the default corpus compilation/domain stages and installed-artifact replay were not rerun. No new GPU campaign ran for this merge. The earlier bounded gfx1250 results remain attributed to `9ef74bfa2d93f06de24d5a6cf475bed096eda55a`, ROCm 10.1 / LLVM24 with rocKE's `llvm23` emission flavor, rather than to the current head. See the earlier validation block for that campaign's scope and provenance. Live review-thread readback before this update found all 11 inline threads resolved, including both yraparti findings. The latest review contains no FAIL rows; its remaining information requests were based on truncated material. The example host test calls `nvfp4_gemm.main`, and the GPU test invokes `nvfp4_gemm_verify` as a subprocess. The PR title and issue reference were confirmed through the live GitHub API. ### Develop integration at `5689189316cd05a4a3dc882db4f37b07e9e11de3` Merged develop `3df7b66cc91f2ebb37c629b5fe30df6354b9e95d`. The sole textual conflict was the representative golden summary. Regenerating it from the merged registry preserves every develop record and all four prior NVFP4 records. The shared Python/native HIP lowerers retain develop additions alongside this PR's typed optimization-barrier admission. - [x] Fresh native engine/binding build; 28/28 CTests passed. - [x] Focused source tests: 223 passed, zero skips, plus four registry subtests in each of Python, strict C++, and strict `both` modes. - [x] Representative golden gate: 250 LLVM23 cases; 242 cases and eight expected rejections at LLVM20/22. - [x] Full standalone differential gate: 81 families / 1,740 configurations per flavor, zero mismatches. LLVM20/22: 1,551 byte-identical and 189 both-rejected; LLVM23: 1,648 byte-identical and 92 both-rejected. NVFP4 agreement remains covered separately through strict serialized-IR lowering tests. - [x] Installed/relocated golden gate passed all three flavors; 209 focused installed tests passed with zero skips, plus four subtests in each backend mode. - [x] Full PR-range repository hooks, relative-path guard and whitespace checks passed. These are fresh host and emission checks for the merged source. No new GPU campaign was run; earlier bounded GPU evidence remains attributed to its recorded revision. ### Validation of `69457fb215c538e721e890d59e15bf7090a9393a` This follow-up registers four representative NVFP4 cases: BF16/FP16 output × K128/K256 at M32/N48, through `NvFp4GemmSpec` and the exported builder. The coverage regression builds those registered cases, checks their tensor-scale ABI and lowers them through the public backend entry point. Strict boolean admission for `tensor_scale` and `prefetch` is retained from `7aeb5c40`. - [x] Fresh native engine and binding build; all 20 CTests passed. - [x] Focused source tests: 223 passed, zero skips, plus four registry subtests in each of Python, strict C++, and strict `both` modes. - [x] Representative golden gate: 204 successful cases at LLVM23; 196 successful cases and eight expected rejections at LLVM20/22. The four new NVFP4 cases emit at LLVM23 and reject with the expected diagnostic at older flavors. Every pre-existing record remains unchanged. - [x] Standalone paired-emitter differential gate: 78 families / 1,295 configurations per flavor, all GREEN with zero mismatches. LLVM20/22: 1,151 byte-identical and 144 both-rejected; LLVM23: 1,248 byte-identical and 47 both-rejected. This dashboard discovers paired emitters independently of the representative registry; NVFP4 Python/C++ agreement is established by the strict `both` registry and focused tests through serialized-IR lowering. - [x] Installed and relocated package: representative golden gate passed all three flavors; 209 focused registry/NVFP4/barrier tests passed, zero skips, plus four subtests in each backend mode. Installed native barrier test passed. Two unrelated example-data lookup tests fail in the standalone installed layout when the whole static-test file is selected; they remain outside this coverage fix. - [x] Repository pre-commit hooks passed over the full PR range; relative-path guard and whitespace checks passed. No new GPU campaign was run for this test/fixture-only follow-up. The bounded GPU evidence below remains attributed to its recorded source revision. `fuzz_diff.py` was omitted because its current samplers cover universal GEMM, batched GEMM, elementwise and reduce, rather than NVFP4/block-scaled specs or HIP admission; running it unchanged would not validate this new contract. Existing recursive installation and `rocke_pytest` / `rocke_golden_static` registration cover the added assets without per-file CMake or tier-YAML changes. Validation of `9ef74bfa2d93f06de24d5a6cf475bed096eda55a`: - [x] Repository pre-commit hooks passed over the develop-to-head range; relative-path guard passed. - [x] Focused host suite: 1,239 passed, zero skips, 30 subtests passed in each of Python, strict C++, and strict `both` modes. Explicit C++ checks cover LLVM20/22 rejection; `both` alone rejects those flavors in Python first. - [x] Fresh native build and all 19 CTests passed, including direct and serialized malformed-clobber and malformed-sideeffect rejection and tensor-epilogue HIP lowering. - [x] Full Python/native LLVM byte identity: all 77 families / 1,266 configurations GREEN at LLVM20, LLVM22 and LLVM23, zero mismatches. - [x] Sixteen non-tensor controls and four NVFP4 controls retain the prior head's names, signatures and LLVM hashes. All 1,266 configuration verdicts and LLVM hashes at every flavor match the previous head. Twelve valid HIP controls (general inline assembly and NVFP4) retain identical source text. No goldens changed in the remediation. - [x] Installed and relocated CMake package: 147 focused tests passed in each of Python, strict C++, and strict `both`, zero skips. Imports resolved from the relocated prefix; installed native barrier test passed. - [x] Exact-source gfx1250 GPU campaigns on ROCm 10.1 / LLVM24 using rocKE's `llvm23` emission format: 20/20 passed, zero skips/errors/failures: eight NVFP4 campaigns with 224 exact `bad=0` comparisons, plus 12 scalar-barrier bit-preservation, predicate and FP32 rounding checks. COMGR/HIP × BF16/FP16 × 16×16×128 and 32×48×256, with an independent NumPy reference and explicit FP32 tensor-factor rounding. The node's fresh native build passed 19 CTests and 147 focused host tests in each backend mode. COMGR checks Python/native LLVM bytes; the HIP route compiles Python-generated HIP source. The source archive SHA-256 was `87e0632904ee462ca1276c8cdf18907bfc43f1f2dc4966a871fbb697dd99a7e1`. The GPU compiler revision was `a7525a1bd6f3bf2a8eb8a8ce35fd75ed1fd6a2ca`; HIP and COMGR came from the same ROCm installation. `llvm23` identifies rocKE's emitted intrinsic ABI and supports this LLVM24 compiler; it is not an exact compiler-major requirement. The previous head's local LLVM20 compilation gate found no new failures, with eight cases unvalidated because they require a newer compiler or an untargetable architecture. These exact bounded fixtures do not establish arbitrary-input accumulation behavior or performance. Quantization, foreign scale-layout conversion, partial tiles, a dynamic K loop, generalized tensor scaling, and a full C++ spec-to-kernel builder remain outside this packed-input example. ISSUE ID : AICK-2254 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
cycheng
pushed a commit
that referenced
this pull request
Nov 6, 2026
…12167) Add an untuned, reusable packed-input NVFP4 GEMM builder and a focused correctness example for gfx1250. The kernel consumes E2M1 FP4 matrices, E4M3 block scales for each group of 16 values along K, and runtime FP32 tensor dequantization factors; it produces BF16 or FP16 output. This PR delivers: - The exported `NvFp4GemmSpec` and `build_nvfp4_gemm` API, with validation, grid and signature helpers, built on the shared block-scaled GEMM implementation. - A standalone example and GPU verifier that launch the builder directly and compare outputs with an independent NumPy reference. - Tensor-factor application in the FP32 epilogue, with an optimization barrier that preserves FP32 rounding before FP16 conversion, plus matching Python/native HIP inline-assembly validation and paired coverage for isolated tensor epilogues. ### Builder and verification flow ```mermaid flowchart TD S["NvFp4GemmSpec: shape and output dtype"] --> B["build_nvfp4_gemm: FP4 / E4M3 / SCALE16"] B --> K["KernelDef with tensor-scale epilogue"] K --> L["LLVM lowering: Python / native C++"] K --> H["HIP source lowering: Python"] L --> C["COMGR compilation"] H --> HC["HIP compilation"] C --> O["HSACO"] HC --> O O --> V["Standalone verifier: pack arguments and launch"] I["Packed E2M1 matrices, E4M3 block scales, FP32 tensor factors"] --> V I --> R["Independent NumPy reference"] V --> G["BF16 / FP16 GPU output"] G --> Q["Compare outputs"] R --> Q ``` The factors are runtime launch arguments. The kernel computes their product in FP32, applies it to the FP32 accumulator, and then converts to the output dtype. In the validated GPU campaigns, COMGR checks Python/native LLVM byte identity; the HIP route compiles Python-generated HIP source. Isolated native HIP barrier acceptance and malformed-attribute rejection are covered separately by the host CTest and paired emitters. Complete native NVFP4 HIP compilation remains outside the supported API scope. ### Scope The delivered entry points are the exported builder and standalone example/verifier. GEMM dispatcher registration, request selection, and production routing are intentionally deferred: this kernel is not tuned, and the validation establishes correctness only for the stated bounded fixtures. Dispatcher integration requires separate work on the packed-input request contract, tuning, and selection-path validation. This PR makes no performance claim. ### Implementation details Tensor scaling is admitted only for FP4/E4M3/SCALE16. The launcher and reference validate factor shape and real numeric dtype before FP32 conversion, then require finite converted values. String, complex, boolean and object arrays are rejected; the launcher rejects missing, malformed or unexpected factors before GPU allocation. The generic block verifier rejects tensor-enabled specs before device discovery or compilation; the NVFP4 verifier supplies the factors and tensor-aware reference. Unsupported LLVM emission flavors are checked with backend-compatible exception assertions and the expected diagnostic. Merge develop's completed scale-format prerequisite #12166, retaining its canonical scale-aware catalog, shared matrix/scale packing, independent operand storage and default-off prefetch. NVFP4 HIP compilation uses Python-generated source; complete native NVFP4 HIP compilation remains outside this API's supported scope. Python and native HIP now agree on single-output/void inline assembly, including boolean sideeffect validation, string clobbers, and an absent or empty clobber for pure optimization barriers. These isolated lowering checks do not establish complete native NVFP4 compilation or new GPU numerical coverage. ### Current coverage and lint follow-up at `e7f2ac9f6a1a` `3532d6e59e29` addresses N1 and the related N3 in the subsequent review; `e7f2ac9f6a1a` clears the CMake lint findings exposed by its new registrations. GCC 13 reproduced a C-builder evaluation-order difference in HIP emitter configurations 0–7: the conversion was recorded before the thread ID, unlike Python. Both builder calls are now explicitly sequenced. The new `rocke_optimization_barrier_hip_pair` CTest executes the actual prebuilt C emitter and the selected Python emitter for all 12 HIP configurations and their range boundary. It is registered in build-tree and installed layouts and enrolled in quick/FFM categories. Binding comparisons now use the existing exact engine-parity helpers. - [x] The former GCC binary fails the new comparison at configuration 0; the corrected binary passes all 12 configurations and the range boundary. GCC 13 and Clang 24 CMake builds pass the real-emitter CTest; the relocated installed CTest also passes. - [x] Source checks: **790 passed per backend**, in Python, strict C++ and strict `both`. Minimal checks: **771 passed / 19 optional-dependency skips per backend**. - [x] All four LLVM parity lanes, older-lane companions and representative goldens pass. Native CTests: **35 passed**, plus one named architecture-domain compiler-provenance skip. - [x] Kernel builders and lowering emission are unchanged. The four NVFP4 fixtures retain exactly the same names, LLVM23/24 hashes and Python HIP hashes as `5acc3fa12d92`; no golden was re-blessed. - [x] **Full PR-range `pre-commit` and whitespace checks pass**, including `cmake-lint`. All 207 findings were fixed without disabling rules: private build variables use compliant names and long commands/strings are wrapped. The generated installed CTest manifest is byte-for-byte unchanged, and all 36 source CTest names, commands and properties are preserved. GPU-reference and configuration-contract checks passed (21 tests, one capability skip); source and relocated real-emitter tests passed again under GCC and Clang. - [ ] The earlier full-runner baseline failures remain unresolved. The lint and coverage gates are green; this is not a merge-readiness claim. N2 shared reference/runner refactoring and N4 shared validation/error-style suggestions are acknowledged and deferred to separately scoped maintenance/API changes, not claimed fixed. See the [explicit author dispositions](#12167 (comment)). #### Additional gfx1250 evidence at `5acc3fa12d92` A completed SSH validation campaign using matched ROCm `10.3.0a20261102` compiler/HIP-runtime components returned **16 NVFP4 campaigns / 448 exact comparisons**, with zero skips, across LLVM23/24 emission, COMGR/HIP, BF16/FP16 output and the bounded 16×16×128 / 32×48×256 fixtures. It also passed **24 scalar/barrier numerical checks**, the focused 783-test host cohort in each backend mode, native CTests and the 12 HIP emitter pairs. COMGR checks Python/native LLVM identity before compilation; the HIP route compiles Python-generated source, not complete native NVFP4 HIP. Required dynamically loaded runtime components were added before accepting this campaign. Existing ROCm installations and default symlinks were not replaced. The completion status and declared assertion summary were retained, but the remote workspace disappeared before detailed logs could be recovered; this limits the available replay artifacts. This GPU evidence belongs to `5acc3fa12d92`, not a new GPU execution of the test/build-wiring-only follow-up. The completed campaign on the passing gfx1250 setup supplies the requested bounded hardware acceptance evidence. A separate setup returned launch faults and is retained only as a diagnostic result, not as a failure of the passing campaign or a claim that the PR caused those faults. Its older-ROCm control passed four COMGR campaigns (112 exact comparisons) and 12 scalar/barrier checks but failed the four HIP campaigns. Neither agreement nor this bounded acceptance establishes arbitrary-input correctness or universal environment compatibility. ### Host validation at `5acc3fa12d92` (before the additional SSH campaign) Follow-up to `9b690231cf47`: `912d776b8a33` fixes HIP template escaping, optional host-test dependencies and numerical-verifier subprocess identity; `5acc3fa12d92` separately hardens the pre-existing LLVM `sideeffect` admission gap. No existing golden was changed in these follow-ups. Both HIP formatters preserve assembly bytes in C++ string literals for void and single-output forms, with paired multiline/quote/backslash coverage and direct/serialized control-character regressions. Both LLVM handlers reject present non-boolean `sideeffect` attributes for void, single-output and multi-output assembly, while preserving absent/false/true behavior. The NVFP4 host tests use a scoped `ml_dtypes` fixture, not a module-wide skip; malformed-factor checks stay enabled without that dependency. Required gfx1250 numerical validation still fails on a missing dependency. The numerical subprocess uses the selected-package launcher; its host regression executes the real launch prefix against a shadowing working directory without claiming a GPU comparison. Fresh Release engine/binding and standalone installed/relocated checks used ROCm `10.3.0a20261102` / LLVM24, Python 3.12.12, NumPy 2.5.4, pytest 9.2.0 and ml_dtypes 0.6.0, without Torch. The minimal environment used the same NumPy/pytest versions and no ml_dtypes. Source commands ran from `platform/`; relocated checks used fresh platform/library wheels and the matching installed native binding, with no editable source imports. - [x] Focused source fixes: **783 passed per backend**, zero skips, in Python, strict C++ and strict `both`. The broader NVFP4/barrier/scale-format/prefetch/registry/golden cohort passed **1,766 tests plus 37 subtests per backend**, zero skips. - [x] Real minimal source and relocated environments: **764 passed / 19 optional-dependency skips per backend**. Builder, identity, ABI-admission, IR and malformed-factor tests remain enabled. Wheel-only default fallback: **595 passed / 188 explicitly unavailable-native skips**, with the native binding absent. - [x] Representative golden gates passed at LLVM20/22/23/24. Canonical Python/native byte identity passed at every flavor, including the older-lane LLVM23 companions: LLVM20/22 each cover 89 families / 2,141 configurations, and LLVM23/24 each cover 94 families / 2,228 configurations; zero mismatches. - [x] Full-runner LLVM24 corpus acceptance: **364/364 modules compiled and linked**, no new failures. - [x] Native CTests: **34 passed / zero failed**, plus one named architecture-domain reproducibility skip because the measured compiler identity differs. - [x] **Twelve Python/C HIP mini-pairs** match byte-for-byte. All 24 emitted source variants compile and link for gfx950/gfx1250; four deliberately unescaped multiline controls fail as expected. All four complete Python-generated NVFP4 HIP fixtures compile. - [x] Local **gfx90a** GPU smoke checks: four multiline void/single-output payload checks from Python/native HIP sources preserve all 64 lane payloads exactly, including floating special-value bit patterns. The existing 12 scalar/barrier numerical checks pass in each of Python, strict C++ and strict `both`. These are not gfx1250 NVFP4 numerical results. - [x] Relocated focused replay: **1,764 passed plus 37 subtests per backend**, zero skips, with exactly two named unrelated example-asset tests deselected. The original broad selection reproduces both failures, and the unchanged starting-head test file reproduces them independently. Installed barrier/golden CTests: **2/2 passed**. - [x] Full pinned-base-to-head repository hooks and whitespace checks passed. - [ ] The full default runner remains **RED**: strict C++ reports **11 failed / 6,922 passed / 851 skipped**; strict `both` reports **2 failed / 6,922 passed / 851 skipped**. Every failing selector reproduces on an independent refreshed build of starting head `9b690231cf47` in the same environment: nine C++ F64 exception-type subtests, the strict-fallback test and the explicit-COMGR compiler-selection test. No new failing case was found. - [ ] No current-head gfx1250 NVFP4 numerical campaign ran. Earlier bounded gfx1250 results remain attributed to their recorded source revision. Complete native NVFP4 HIP compilation remains outside this API's supported scope; the known native `0f` formatting limitation is not changed by these follow-ups. #### Shared symbol-rename inventory The preceding shared `BlockScaledGemmSpec` rename intentionally reaches **MXFP4/6/8, mixed operands, alternate scale formats, legacy WMMA and fused-MoE's block-scaled GEMM stages**, not only NVFP4. At this head, independent comparisons against pinned develop confirm unchanged normalized LLVM/HIP for the **82 affected focused golden cases plus two legacy-WMMA controls**, and all four MXFP8 representative cases at LLVM23/24. The change is symbol-only for those checked paths; no new numerical or performance claim is inferred. Already regenerated committed artifacts are the focused [scaled-WMMA](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_scaled_wmma_llvm23.json), [mixed-scaled-WMMA](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_mixed_scaled_wmma_llvm23.json), [scale-format](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/core/gfx1250_scale_formats_llvm23.json) goldens and the [representative GEMM golden shard](https://github.com/ROCm/rocm-libraries/blob/5acc3fa12d92/dnn-providers/hip-kernel-provider/rocke/platform/tests/golden/rocke_representative_ir_sha256/gemm.json). Any externally generated name-keyed sources, binaries or manifests from before the rename must be rebuilt to use the new entry-point names. These remediation commits do not rename kernels again or re-bless those goldens. ### Validation of `9b690231cf47` (prior integration) Merged develop `141e2e47b38b`. The LLVM24 golden has its own four NVFP4 records, with no LLVM24-to-LLVM23 alias. Kernel symbols use codegen specialization identity rather than workload-size tags: M changes only the grid; baked N addressing and K unrolling remain distinguished. All affected goldens were refreshed after verifying symbol-only changes, including 82 sibling cases. Commands ran from `platform/` with a fresh Release engine/binding, ROCm `10.3.0a20261102` / LLVM24, Python 3.12.12, NumPy 2.4.6, pytest 9.2.0 and ml_dtypes 0.6.0. - [x] Focused source suites: **1,231 tests plus 37 subtests per backend**, zero skips, in Python, strict C++ and strict `both`. - [x] Representative golden and byte-identity gates at LLVM20/22/23/24, including older-lane companion checks. - [x] LLVM24 corpus acceptance: **364/364 modules compiled and linked**. - [x] Eight Python/C HIP mini-pairs match byte-for-byte and compile for gfx950/gfx1250. All four complete Python-generated NVFP4 HIP fixtures compile. - [x] Fresh installed/relocated replay: **1,229 tests plus 37 subtests per backend**, zero skips; two unrelated example-asset tests were deselected after reproducing their failures with the original test file. Installed golden/barrier CTests: **2/2 passed**. - [x] Native CTests: **33 passed, zero failed**, with one named architecture-domain reproducibility skip because the measured compiler revision differs. - [x] PR-range repository hooks and whitespace checks. - [ ] Full default runner remains **RED**: strict C++ reports **11 failed / 6,395 passed / 851 skipped**; strict `both` reports **2 failed / 6,395 passed / 851 skipped**. All remaining failures reproduce on an independent fresh build of the pre-fix integration `3be7531f`: nine C++ F64 exception-type subtests, the strict-fallback test and the explicit-COMGR compiler-selection test. No new failing case remains. - [ ] Complete native NVFP4 HIP compilation still fails on the existing `0f` literal formatting issue and remains outside the supported API scope. Isolated native barrier support is not substituted for complete-kernel acceptance. - [ ] No new current-head gfx1250 GPU numerical campaign ran. Earlier bounded GPU evidence remains attributed to its recorded source revision; host hashes and compilation do not establish GPU correctness. ### Historical native HIP scope reduction at `ad36142c7dbedebf026fa5bd6f01b294cb480d50` Removed this PR's native HIP optimization-barrier support and its HIP paired-emitter mode, optional pytest wrapper, test-specific environment variable, extra CMake target/install wiring, and runner plumbing. Python HIP compilation and Python/native LLVM lowering remain. The existing `rocke_optimization_barrier` CTest now requires native HIP rejection of the valid FP16 tensor epilogue, directly and after serialization, on gfx950/gfx1250 with absent/empty clobbers; the same epilogue must still lower through LLVM at every supported flavor. These results are retained only as historical evidence for this revision; the current isolated-native-HIP behavior and validation are recorded above. - [x] Fresh Release native engine/binding build with ROCm `10.3.0a20261029` / AMD Clang LLVM24, Python 3.12.12, NumPy 2.4.6, pytest 9.1.1, and optional `ml_dtypes`. - [x] Native CTests: **33 passed, zero failed**. The separate architecture-domain reproducibility gate was excluded, not rerun. - [x] Focused source NVFP4/barrier/example/runner/registration/representative-golden checks: **246 passed plus four subtests, zero skips**, in each of Python, strict C++ and strict `both`. - [x] Standalone installed/relocated package: native rejection CTest passed; **209 focused tests plus four subtests, zero skips**, in each backend mode. Imports resolved from the relocated package and binding. - [x] Canonical LLVM byte-identity gates passed at LLVM20/22/23/24. Older lanes cover 88 eligible families / 1,985 configurations each (1,714 identical, 271 both-rejected), with their LLVM23 companion lanes; LLVM23/24 cover 93 families / 2,072 configurations each (1,900 identical, 172 both-rejected). Zero mismatches. NVFP4 builder agreement is covered separately by the strict registry tests. - [x] All four NVFP4 names, LLVM hashes and Python HIP hashes are unchanged from the preceding implementation. Full FP16 NVFP4 serialized IR now rejects native HIP with the expected inline-assembly diagnostic. The new rejection test fails against the old native implementation at the expected assertion, establishing a negative control. Existing BF16 native lowering behavior is not disabled; it is not the verifier's supported HIP compilation route. - [x] Documented direct `tests/test_rocke.py` command passed: **368 tests, 132 skips** on its Python-only path. File hooks, full pinned-develop-to-head PR-range hooks, relative-path guard, and whitespace checks passed. No goldens changed. - [ ] No full default runner, compiler-acceptance corpus, architecture-domain reproduction, or GPU numerical campaign was rerun for this scope reduction. Prior bounded GPU results remain attributed to their original source revisions. ### Develop integration at `94475ba54d94934251490a807b917d587c0cd9f0` Merged develop `bec4b98a286f68ef37c4bb1036d78d7f88719180` into `f903028e5eadbdeabecf4bc70e83727a85624727`. The sole conflict was the representative golden. Regeneration from the complete merged registry preserves every develop record and all four NVFP4 outcomes; the four focused NVFP4 LLVM23 hashes are unchanged. LLVM23 has 335 successful cases; LLVM20/22 each have 307 successful cases and 28 expected rejections. LLVM24 shares the explicit LLVM23 emission-profile alias from develop. - [x] Fresh Release native engine and Python binding build with `rocm-nightly/latest`, resolved to `10.3.0a20261029`, using AMD Clang/COMGR LLVM24. - [x] Focused registry/NVFP4/barrier/golden-comparator tests: 233 passed plus four subtests in each of Python, strict C++ and strict `both`. The eight initially skipped direct HIP text comparisons passed separately with the fresh native barrier emitter selected. - [x] Fresh representative golden check passed at LLVM20, LLVM22, LLVM23 and LLVM24. - [x] Python/native LLVM parity: LLVM20/22 each cover 85 eligible families and 1,825 configurations (1,584 byte-identical, 241 both-rejected); LLVM23/24 each cover all 88 families and 1,894 configurations (1,753 byte-identical, 141 both-rejected). Zero mismatches. The older lanes exclude `conv_direct_dgrad_wmma`, `conv_direct_fwd_wmma` and `conv_fwd_small_wmma`, whose fixtures require LLVM23 operations; all three are covered by the full LLVM23/24 runs. The unfiltered older-lane checks also establish matching rejection for the two forward families. - [x] Native CTest: 33 passed, one architecture-domain reproducibility skip, zero failed. PR-range repository hooks, relative-path guard and whitespace checks passed. - [ ] Full default runner, compiler-acceptance corpus, installed/relocated replay and GPU numerical execution were not rerun for this golden refresh. Earlier results remain attributed to their recorded revisions. ### Documentation follow-up at `2f004ba50231ca099a47ee83ab43c0af19d1eee0` Added the [NVFP4 API reference](https://github.com/ROCm/rocm-libraries/blob/2f004ba50231ca099a47ee83ab43c0af19d1eee0/dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/gemm.md#gfx1250-nvfp4-gemm) and instance-catalog entry. The reference covers all five exported APIs, packed input/scale layouts, runtime tensor-factor ABI, shape/dtype constraints, LLVM emission flavor, and standalone scope, and links the detailed example input contract. This commit changes only two documentation files. - [x] All four introduced relative links and anchors resolve; documentation whitespace checks pass. - [x] The documented Python example builds; package exports, the ten-argument ABI, and launch grid/block agree with the source API. - [x] Repository hooks passed on the two changed files and over the complete pinned-develop-to-head PR range; PR-range `git diff --check` passed. No native build, compiler acceptance, installed replay, full runner, or GPU numerical lane was rerun for this documentation-only commit. Prior host/native/GPU results remain attributed to the source revisions below, including the reported broad baseline failures and unrun lanes. ### Develop integration at `9ae4868fe069ed91aac0de1e71e2c79fbb4f3c9b` Merged develop `ed481508f54b3ba6b3141d06e8dcd8bdce2a8c8f`. Resolved the test README conflict by retaining the representative-versus-paired-emitter coverage explanation alongside upstream inventory/runner updates. Regenerated the representative golden from the merged registry and regenerated the four focused NVFP4 hashes. Every current develop record and all four prior NVFP4 outcomes are preserved; the focused hashes are unchanged. - [x] Fresh Release native engine/binding build; **31/31 CTests passed**, including architecture-domain reproducibility. - [x] Focused source tests in Python, strict C++, and strict `both`: **223 passed plus four registry subtests per mode, zero skips**. - [x] Representative golden gate at every flavor: **288 successful LLVM23 cases; 278 successful cases and 10 expected rejections at LLVM20/22**. - [x] Canonical LLVM20/22 byte-identity gates, including their companion LLVM23-only lanes, passed. Complete LLVM23 dashboard: **84 families / 1,854 configurations**, 1,737 byte-identical and 117 both-rejected, zero mismatches. NVFP4 builder agreement is covered separately through strict serialized-IR tests. - [x] The documented direct test-file command passed: **362 tests, 124 skips** (native checks unavailable on that direct command's Python-only path and optional Torch). - [x] Relative-path guard, emitted-IR validity and architecture-domain warning gate passed. IR validity reports **22 unvalidated cases**: 10 require newer LLVM and 12 an untargetable architecture. - [x] Repository pre-commit hooks over `origin/develop..HEAD` on this committed head and PR-scope whitespace checks passed. - [ ] The full default runner is **RED**: its strict C++ pytest pass reports **15 failed / 5,357 passed / 848 skipped**; its strict `both` pass reports **6 failed / 5,357 passed / 848 skipped**. Every failing item reproduces on a separate fresh build of pinned develop with the same environment and working directory. These are baseline failures, not an NVFP4 regression. - [ ] Current-head GPU numeric execution and installed-artifact replay were not run. This host has no `/dev/kfd`; earlier gfx1250 GPU evidence remains attributed to its original source revision below. Commands ran from `platform/`, with Python 3.12.12, NumPy 2.4.6, pytest 9.1.1, optional `ml_dtypes` available for the focused/full runner suites, and no Torch. The host COMGR resolves LLVM20; the direct/default runner commands use that compatible automatic flavor. Offline NVFP4 emission tests explicitly use LLVM23. ```sh PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python:../library ROCKE_BACKEND=python python tests/test_rocke.py # Full runner, with no stages disabled; the build-root argument selects the fresh build. PYTHONDONTWRITEBYTECODE=1 \ PYTHONPATH=python:../library:<fresh-build>/cpp/bindings:<ml-dtypes-location> \ ROCKE_CPP_STRICT=1 CMAKE_BUILD_PARALLEL_LEVEL=8 \ OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 \ python tests/run_all.py --build-root <fresh-build> ``` The baseline failures are nine HIP F64 exception-type subtests in C++ mode, a fallback test under strict-mode settings, two stale gfx1250 enumeration expectations, a subprocess import with the runner's relative `PYTHONPATH`, MX GEMM concrete-IR JSON replay, and an installed-environment ordering expectation. The latter six fail in both backend modes; the nine F64 subtests pass in `both`. All were replayed on develop `ed481508f54b`. The host example test exercises `nvfp4_gemm.main`, and the GPU test invokes `nvfp4_gemm_verify` via subprocess. Their presence does not establish current-head GPU execution. The field-manual follow-up adds the NVFP4 API and input contract to the GEMM reference and instance catalog; it changes documentation only. Validation in this develop-integration section remains attributed to `9ae4868fe069`, including the broad baseline failures and GPU/installed replay limitations. Review-thread disposition is recorded in the individual replies. ### Develop integration at `599f89a22801faf43f1ad9a7bee88faab236b37c` Merged develop `3ebcd68817f0af480ee9926d5b12174edc2a970d`. The only textual conflict was the representative golden summary; regenerating from the merged registry preserves every develop record and all four prior NVFP4 outcomes at each flavor. Shared lowerers merged automatically. The documentation now gives the focused host commands, distinguishes representative binding coverage from the paired-emitter dashboard, and clarifies compiler-flavor and GPU evidence requirements. - [x] Fresh Release native engine/binding build; 30/30 CTests passed. - [x] Focused source suite in Python, strict C++, and strict `both`: 575 passed and 5,342 subtests passed per mode; 14 optional Torch tests skipped per mode. - [x] Representative golden gate: LLVM23 has 272 successful cases; LLVM20/22 each have 264 successful cases and eight expected rejections. All four NVFP4 cases emit at LLVM23 and reject at older flavors. - [x] Full paired-emitter differential gate: 83 families / 1,830 configurations per flavor, zero mismatches. LLVM20/22 each have 1,624 identical and 206 both-rejected configurations; LLVM23 has 1,721 identical and 109 both-rejected. NVFP4 builder agreement is established separately by strict serialized-IR registry tests. - [x] Direct README test-file entry point passed: 362 tests reported, with 124 skips (110 native-extension checks and 14 optional Torch tests). The native-enabled focused matrix above covers the binding checks. - [x] Runner relative-path guard and all native CTests passed with the other runner stages explicitly disabled, as shown below. - [x] Repository hooks over the full pinned-develop-to-head PR range and whitespace checks passed. The direct test-file command, run from `platform/`, was: ```sh PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python:../library ROCKE_BACKEND=python python3 tests/test_rocke.py ``` The runner command, with a fresh full native build and its binding on `PYTHONPATH`, was: ```sh python3 tests/run_all.py --no-gate --no-pytest --no-both --no-ir-validity --no-arch-domain --build-root <fresh-build> ``` These commands do not claim a full default `python tests/run_all.py` run: pytest and all-flavor differential checks ran separately; the default corpus compilation/domain stages and installed-artifact replay were not rerun. No new GPU campaign ran for this merge. The earlier bounded gfx1250 results remain attributed to `9ef74bfa2d93f06de24d5a6cf475bed096eda55a`, ROCm 10.1 / LLVM24 with rocKE's `llvm23` emission flavor, rather than to the current head. See the earlier validation block for that campaign's scope and provenance. Live review-thread readback before this update found all 11 inline threads resolved, including both yraparti findings. The latest review contains no FAIL rows; its remaining information requests were based on truncated material. The example host test calls `nvfp4_gemm.main`, and the GPU test invokes `nvfp4_gemm_verify` as a subprocess. The PR title and issue reference were confirmed through the live GitHub API. ### Develop integration at `5689189316cd05a4a3dc882db4f37b07e9e11de3` Merged develop `3df7b66cc91f2ebb37c629b5fe30df6354b9e95d`. The sole textual conflict was the representative golden summary. Regenerating it from the merged registry preserves every develop record and all four prior NVFP4 records. The shared Python/native HIP lowerers retain develop additions alongside this PR's typed optimization-barrier admission. - [x] Fresh native engine/binding build; 28/28 CTests passed. - [x] Focused source tests: 223 passed, zero skips, plus four registry subtests in each of Python, strict C++, and strict `both` modes. - [x] Representative golden gate: 250 LLVM23 cases; 242 cases and eight expected rejections at LLVM20/22. - [x] Full standalone differential gate: 81 families / 1,740 configurations per flavor, zero mismatches. LLVM20/22: 1,551 byte-identical and 189 both-rejected; LLVM23: 1,648 byte-identical and 92 both-rejected. NVFP4 agreement remains covered separately through strict serialized-IR lowering tests. - [x] Installed/relocated golden gate passed all three flavors; 209 focused installed tests passed with zero skips, plus four subtests in each backend mode. - [x] Full PR-range repository hooks, relative-path guard and whitespace checks passed. These are fresh host and emission checks for the merged source. No new GPU campaign was run; earlier bounded GPU evidence remains attributed to its recorded revision. ### Validation of `69457fb215c538e721e890d59e15bf7090a9393a` This follow-up registers four representative NVFP4 cases: BF16/FP16 output × K128/K256 at M32/N48, through `NvFp4GemmSpec` and the exported builder. The coverage regression builds those registered cases, checks their tensor-scale ABI and lowers them through the public backend entry point. Strict boolean admission for `tensor_scale` and `prefetch` is retained from `7aeb5c40`. - [x] Fresh native engine and binding build; all 20 CTests passed. - [x] Focused source tests: 223 passed, zero skips, plus four registry subtests in each of Python, strict C++, and strict `both` modes. - [x] Representative golden gate: 204 successful cases at LLVM23; 196 successful cases and eight expected rejections at LLVM20/22. The four new NVFP4 cases emit at LLVM23 and reject with the expected diagnostic at older flavors. Every pre-existing record remains unchanged. - [x] Standalone paired-emitter differential gate: 78 families / 1,295 configurations per flavor, all GREEN with zero mismatches. LLVM20/22: 1,151 byte-identical and 144 both-rejected; LLVM23: 1,248 byte-identical and 47 both-rejected. This dashboard discovers paired emitters independently of the representative registry; NVFP4 Python/C++ agreement is established by the strict `both` registry and focused tests through serialized-IR lowering. - [x] Installed and relocated package: representative golden gate passed all three flavors; 209 focused registry/NVFP4/barrier tests passed, zero skips, plus four subtests in each backend mode. Installed native barrier test passed. Two unrelated example-data lookup tests fail in the standalone installed layout when the whole static-test file is selected; they remain outside this coverage fix. - [x] Repository pre-commit hooks passed over the full PR range; relative-path guard and whitespace checks passed. No new GPU campaign was run for this test/fixture-only follow-up. The bounded GPU evidence below remains attributed to its recorded source revision. `fuzz_diff.py` was omitted because its current samplers cover universal GEMM, batched GEMM, elementwise and reduce, rather than NVFP4/block-scaled specs or HIP admission; running it unchanged would not validate this new contract. Existing recursive installation and `rocke_pytest` / `rocke_golden_static` registration cover the added assets without per-file CMake or tier-YAML changes. Validation of `9ef74bfa2d93f06de24d5a6cf475bed096eda55a`: - [x] Repository pre-commit hooks passed over the develop-to-head range; relative-path guard passed. - [x] Focused host suite: 1,239 passed, zero skips, 30 subtests passed in each of Python, strict C++, and strict `both` modes. Explicit C++ checks cover LLVM20/22 rejection; `both` alone rejects those flavors in Python first. - [x] Fresh native build and all 19 CTests passed, including direct and serialized malformed-clobber and malformed-sideeffect rejection and tensor-epilogue HIP lowering. - [x] Full Python/native LLVM byte identity: all 77 families / 1,266 configurations GREEN at LLVM20, LLVM22 and LLVM23, zero mismatches. - [x] Sixteen non-tensor controls and four NVFP4 controls retain the prior head's names, signatures and LLVM hashes. All 1,266 configuration verdicts and LLVM hashes at every flavor match the previous head. Twelve valid HIP controls (general inline assembly and NVFP4) retain identical source text. No goldens changed in the remediation. - [x] Installed and relocated CMake package: 147 focused tests passed in each of Python, strict C++, and strict `both`, zero skips. Imports resolved from the relocated prefix; installed native barrier test passed. - [x] Exact-source gfx1250 GPU campaigns on ROCm 10.1 / LLVM24 using rocKE's `llvm23` emission format: 20/20 passed, zero skips/errors/failures: eight NVFP4 campaigns with 224 exact `bad=0` comparisons, plus 12 scalar-barrier bit-preservation, predicate and FP32 rounding checks. COMGR/HIP × BF16/FP16 × 16×16×128 and 32×48×256, with an independent NumPy reference and explicit FP32 tensor-factor rounding. The node's fresh native build passed 19 CTests and 147 focused host tests in each backend mode. COMGR checks Python/native LLVM bytes; the HIP route compiles Python-generated HIP source. The source archive SHA-256 was `87e0632904ee462ca1276c8cdf18907bfc43f1f2dc4966a871fbb697dd99a7e1`. The GPU compiler revision was `a7525a1bd6f3bf2a8eb8a8ce35fd75ed1fd6a2ca`; HIP and COMGR came from the same ROCm installation. `llvm23` identifies rocKE's emitted intrinsic ABI and supports this LLVM24 compiler; it is not an exact compiler-major requirement. The previous head's local LLVM20 compilation gate found no new failures, with eight cases unvalidated because they require a newer compiler or an untargetable architecture. These exact bounded fixtures do not establish arbitrary-input accumulation behavior or performance. Quantization, foreign scale-layout conversion, partial tiles, a dynamic K loop, generalized tensor scaling, and a full C++ spec-to-kernel builder remain outside this packed-input example. ISSUE ID : AICK-2254 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds separate A/B scale-format selectors to gfx1250 block-scaled GEMM in Python and C++. Callers can use E4M3 or E5M3 on an FP4 operand while the other operand uses E8M0. Both engines resolve the complete MMA contract, pack its operands, and emit byte-identical LLVM IR.
ISSUE ID : AICK-2284
Supported combinations
FP4 × FP4 requires matching scale formats. Both K32/SCALE and K16/SCALE16 follow this table. The catalog contains 86 scaled atoms: 43 supported matrix/scale combinations × 2 group sizes, adding 36 alternate-scale atoms to develop's 50 E8M0 atoms. Native atoms are 16 × 16 × 128 with FP32 accumulation; GEMM stores BF16 by default or FP16 when requested.
Catalog admission and both backends apply the supported-combination check from LLVM commit 7597c0c938b3 — [AMDGPU] Validate WMMA scale/format combination. That commit adds validation to intrinsic lowering and the assembler. rocKE applies the same rule on every compiler version: older compiler acceptance does not establish support for the excluded combinations. The public machine-readable ISA describes SCALE/SCALE16 operand widths and block sizes but omits the complete scale-format legality table.
Implementation
flowchart TD J["Compact JSON family declaration"] --> P["Python expansion and supported-combination filter"] C["Mirrored C++ family facts"] --> N["Compile-time generation and matching filter"] P --> PC["Python catalog: 86 scaled atoms"] N --> NC["Immutable native catalog: 86 scaled atoms"] S["GEMM spec: A/B matrix types, A/B scale types, block K"] --> Q["Resolve complete MMA contract"] PC --> Q NC --> Q Q --> V["Backend validation and operand packing"] V --> PY["Python LLVM lowering"] V --> CPP["C++ LLVM lowering"] PY --> B["Byte-identity gate"] CPP --> Bscale_dtype_aandscale_dtype_boverride the sharedscale_dtypefallback.i8remains an E8M0 alias. Unsupported combinations are absent from catalog queries, rejected by builders, and independently rejected before backend lowering. CLI invalid-spec requests exit through argparse with code 2, before compilation.64,32to32,32; instruction bodies are unchanged. Representative goldens regenerate identically. Exhaustive host/native tests cover all 450 candidate tuples: 86 admitted and 364 rejected.Current merge validation
The merge refresh preserves the generated gfx1250 catalog and develop's gfx950 scaled MFMA lookup. Independent scale decoding and fixtures now live in develop's shared reference module, with regression coverage for the gfx950 E8M0 fallback and K=64/128/256. Differential enumeration retains develop's 256-case default and the gfx1250-specific 512-case bound.
tests/run_all.py, and the remaining installed pytest CTest entries were not rerun for this merge. GPU results below remain evidence for their stated earlier head.Scale fixture guard validation
Direct
make_case_inputs()calls now reject unknown scale formats before selecting fixture bytes, for either A or B. The guard preserves E8M0/I8 handling and the existing resolved aliases; it does not change catalog admission or kernel emission.Test plan
88dde8e1e008: fresh native/pybind build and strict dual-backend execution on gfx1250 with ROCm 10.1.0 / Clang24 pass 319 tests, with zero failures/errors/skips, 3,969 successful fixture reports, and all 40 fixed-byte selector tests. COMGR and HIP routes cover both SCALE/SCALE16 and BF16/FP16 output where parametrized. The complete supported suite runs without external selection or deselection. After the GPU run,e6c65de3aa52updates three overview documents. The positional-API repair at5027f0de3dd8restores the originalprefetchargument position and corrects a stale host prefetch fixture; all emitted parity LLVM hashes remain identical to the GPU-validated implementation. The GPU suite was not rerun for this constructor-order repair.5027f0de3dd8: the new A/B scale fields follow the existingprefetchfield. Regression coverage exercises the original positional signature with prefetch on/off across WMMA, SCALE, and SCALE16 and checks equality with keyword construction and its LLVM. 834 focused tests plus 25 subtests pass in strict dual-backend mode from both source and a relocated install; the regression fails all six cases against the pre-fix constructor. All-flavor parity passes and every emitted LLVM hash matches the preceding run.tests/run_all.py, the remaining installed pytest CTest entries, and exhaustive special-value/rounding coverage were not run.Numeric fixtures include independent non-unit A/B scales, mixed scales, and isolated K groups. E8M0 inputs include powers of two; E4M3/E5M3 include 0.5, 0.75, 1.25, 1.5, and 1.75. Fixed-byte probes compare raw scale bytes
0x3cand0x7cagainst literal predictions while changing the FP4 selector, in both operand positions and both instruction modes. The numeric scope is bounded finite inputs; it makes no arbitrary-input, special-value, or performance claim.Native/pybind validation uses Release builds with warnings treated as errors and LTO disabled. COMGR exercises Python/C++ LLVM identity; HIP tests compile Python-generated HIP. The emission flavor is
llvm23, including when the consuming compiler is LLVM24.