Repository navigation
feat(hipdnn): support LLVM generic GPU targets in kernel descriptors - #13129
Merged
BrianHarrisonAMD merged 67 commits intoOct 15, 2026
Merged
Conversation
…harriso/hkp-packaging-probes
…ARCH ...) line per arch Derive keeps one UKD per compile group (rocke: builder; hip: source+build) among UKDs matching ARCH using the packer's arch rules, writes expect.json, and is write-if-different so a reconfigure does not repack. Assert takes expect.json and a provenance rules table keyed by origin kind. KIND/KDP/INSTANCE are removed. Docs and skills updated.
…-packaging-probes
tools/hkp_probe_kinds.py holds one entry per supported kernel_source.kind (compile-group fields, provenance checks). hsaco, packed by develop since #12729, is the third entry. Remove the production-tree count pin.
…d_source Assert dispatches on the expected kind's output type: kpack kinds keep the archive checks, pass-through kinds are checked for an unchanged kernel_source, arch and provenance. Archive checks run only when a kpack UKD is expected. expect.json carries the authored kernel_source for pass-through kinds.
Add the shared gpu_generic_targets.json (LLVM provenance block), its configure-time validator and generated constexpr table, the SDK tier helpers (archTier, archesCompete, expanded archOverlaps/archCovers) and the Python mirror in hkp_pack.generic_targets, with golden vectors.
- hkp_add_packaging_probe gains UKDS: the probe packs exactly the listed UKDs (by name); without it, one UKD per compile group. rocke groups by (source, builder); a missing group_by field fails derive. - check_probe_junit reads the lane's full ctest junit and checks manifest names only; disabled or not-run probes fail. Probes run once in 'Run tests'; the check runs whenever tests ran. 'Build packaging probes' step removed. - kpack UKDs must ship arch == [ARCH]; NAME/ARCH/UKDS validated; probe call guard enforced; derive tooling is a configure dependency. - Tests: helpers moved out of conftest, new UKDS/derive/assert cases, packer-only test removed. Docs: usage and extension guide, runbook final step.
- UKDS "" (e.g. an empty variable) fails configure instead of falling back to the default sample. - ARCH must be one concrete target, gfx[0-9a-f]+, as the packer selects them. - --ukd mode reports a listed UKD whose arch is not a subset of its KDP's instead of dropping it; tests for that and for equal kept names. - Runbook final step builds hkp_packaging_probes before ctest; README credits the junit check for CI; unused test import and wrapper removed.
…k loader Tuple uniqueness competes per arch tier, admitted kernels sharing a tuple are shadowed by the best tier per device, the loader warns on generic names absent from the table, and KpackModuleCache::selectArch prefers the exact device key before a generic archive key containing the device.
The packer validates generic arch entries (S5/S6/unknown/rocKE), compiles a generic entry once under its literal spelling and materialises it in every selected member target's folder. The arch probe, desk check and descriptor context follow the same tier rule via the shared table.
…tools Config validation, dedup and the generated census follow the loader's tier rule; verify_variant_sets reads the shared table through hkp_pack. agreement.overlap is removed with its last caller.
Desk check follows the loader's same-tier rule and inheritance; a standalone UKD with an empty arch is only valid under an empty-arch KDP; the table validator, its negative fixtures and ctests are tightened; generator and packer share the generic arch rules under a parity test; tests renamed and categorised to repo conventions.
…JOBS check_probe_junit.py gains an opt-in --min-probes N so a lane that must always have a probe fails when arch discovery declares none. Both superbuild lanes pass --min-probes 1. hkp_add_packaging_probe passes PACK_JOBS to hkp_wire_pack_target only when set, avoiding the CMP0174 empty-value warning.
…-packaging-probes # Conflicts: # .github/workflows/hipdnn-superbuild-ci.yml
Develop moved packed UKD provenance out of the descriptor into a per-descriptor sidecar. hkp_probe_assert.py now attaches it with hkp_pack.provenance_sidecar, reporting provenance-sidecar when it is missing or does not match, and the probe tests read and write packed trees through the sidecar.
…eric-target-matching # Conflicts: # .github/workflows/hipdnn-superbuild-ci.yml # dnn-providers/hip-kernel-provider/descriptor-packaging/README.md # dnn-providers/hip-kernel-provider/descriptor-packaging/python/hkp_pack/descriptor_context.py # dnn-providers/hip-kernel-provider/descriptor-packaging/python/hkp_pack/desk_check.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/pack_helpers.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/rocke/test_desk_check_rocke_corpus.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/rocke/test_hkp_pack_layout_rocke.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/rocke/test_hkp_pack_rocke.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/test_hkp_pack_hip.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/test_hkp_pack_hsaco.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/test_hkp_pack_layout.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tests/test_specialization_agreement.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tools/hkp_desk_check.py # dnn-providers/hip-kernel-provider/src/compilation/KpackModuleCache.hpp # dnn-providers/hip-kernel-provider/src/tests/compilation/TestKpackModuleCache.cpp # projects/hipdnn/plugin_sdk/include/hipdnn_plugin_sdk/ingestor/DescriptorLoader.hpp # projects/hipdnn/plugin_sdk/include/hipdnn_plugin_sdk/ingestor/KernelIngestorStateManager.hpp # projects/hipdnn/plugin_sdk/tests/ingestor/TestDescriptorLoader.cpp # projects/hipdnn/tools/IngestorGenerator/README.md # projects/hipdnn/tools/IngestorGenerator/tools/verify_variant_sets.py
…bes' into users/bharriso/generic-target-matching # Conflicts: # .github/workflows/hipdnn-superbuild-ci.yml # dnn-providers/hip-kernel-provider/descriptor-packaging/tools/hkp_probe_assert.py
…-packaging-probes
…o/generic-target-matching
Contributor
|
From AI: Plugin SDK
hip-kernel-provider
IngestorGenerator
|
adickin-amd
pushed a commit
that referenced
this pull request
Oct 14, 2026
…#13061) ## Summary The superbuild CI builds only gfx942 (Linux) and gfx1151 (Windows), so packaging for an architecture neither lane builds, today gfx950, can break with no check going red. This adds packaging probes to both lanes. A probe packs a bounded sample of the production descriptors for an architecture the lane does not build, then checks the output. Architectures come from the descriptors, so a new one needs no edit. Nothing is installed, so TheRock is unaffected. JIRA ID : ALMIOPEN-2833 ## Risk Assessment Risk level 2. Probes sit behind `HIPKERNELPROVIDER_ENABLE_PACKAGING_PROBES`, which defaults to OFF and is ON only in the two superbuild lanes. With it OFF, targets and tests match `develop`. With it ON, a probe failure fails the lane. A probe compiles a sample, not every descriptor (17 of 840 for gfx950 dense attention), alongside the C++ build. ## ASIC Coverage Host-side packaging only. PR CI is sufficient. ## Testing Summary - Probe tooling tests cover architecture discovery, kernel selection, every output check, and end-to-end packs with the real packer and hipcc. - Mutations that turn a probe red: an invalid spec, a metadata disagreement, a missing or empty output, a broken kernel. - With `HIPKERNELPROVIDER_ENABLE_ROCKE=OFF`, probes still run for the non-rocKE kinds without a rocKE toolchain. This is covered by an end-to-end test and a scratch CMake configure, build and ctest. ## Testing Checklist - [x] Probe tooling tests - `pytest descriptor-packaging/probes/tests` - Status: Passed - [x] Junit check tests - `pytest .github/scripts/tests/test_check_probe_junit.py` - Status: Passed - [x] Packer suites - `hip-kernel-provider-hkp-pack-quick`, `hip-kernel-provider-hkp-pack` - Status: Passed - [ ] PR CI - Linux and Windows superbuild lanes - Status: Pending (rerun after the latest push) ## Technical Changes - **Architectures:** every concrete `arch` a production descriptor names, minus the lane's `GPU_TARGETS`. - Wildcard descriptors are excluded; each lane's own pack covers them. - Generic targets and TheRock family names are skipped with a configure STATUS line. Generic support comes with #13129. - **Kernel selection:** per compile group, enough rocKE kernels that every value of each varying `spec` field compiles once (combinations are not guaranteed), and one kernel for other kinds. An integration can add a named probe listing specific kernels in `probes/probes.cmake`. - **rocKE OFF:** probes read and pack the root under the same excluded folders and disabled kinds as the production pack, so rocKE content is neither packed nor expected and other kinds are probed as usual. An architecture with only rocKE content gets no probe, and a named probe that lists a rocKE UKD is skipped with a STATUS line. Tests that pack rocKE live in `probes/tests/rocke/`, which OFF builds do not collect. - **Tooling:** `hkp_probe_derive_root.py` builds each probe's descriptor tree at configure using the packer's own loader and arch rules. `hkp_probe_assert.py` checks the packed output. `hkp_probe_kinds.py` holds the per-kind rules both share. - **CI:** both lanes configure with the option ON. A `Check packaging probes ran` step fails the lane if a probe was skipped or failed, since ctest counts a disabled test as passing. - **Docs:** "Packaging probes" in the descriptor-packaging README, plus the ingestor-engine and superbuild skills.
Base automatically changed from
users/bharriso/hkp-packaging-probes
to
develop
October 14, 2026 03:29
…eric-target-matching # Conflicts: # dnn-providers/hip-kernel-provider/descriptor-packaging/README.md # dnn-providers/hip-kernel-provider/descriptor-packaging/cmake/HkpPackagingProbes.cmake # dnn-providers/hip-kernel-provider/descriptor-packaging/probes/probes.cmake # dnn-providers/hip-kernel-provider/descriptor-packaging/probes/tests/probe_support.py # dnn-providers/hip-kernel-provider/descriptor-packaging/probes/tests/rocke/test_hkp_probe_assert_rocke.py # dnn-providers/hip-kernel-provider/descriptor-packaging/probes/tests/test_hkp_probe_derive_root.py # dnn-providers/hip-kernel-provider/descriptor-packaging/probes/tests/test_hkp_probe_end_to_end.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tools/hkp_probe_assert.py # dnn-providers/hip-kernel-provider/descriptor-packaging/tools/hkp_probe_derive_root.py
…matching; add order-independent merge atomicity test
dpeabody-amd
added a commit
that referenced
this pull request
Oct 16, 2026
…eneric) (#13150) Builds on #12710 (merged as `198636dae1d`); this branch is rebased onto `develop` and contains only its own commits. Turns the gfx1151 SDPA kernels into one kernel set per RDNA generation: `gfx11-generic` code objects for gfx1100, gfx1101, gfx1102, gfx1103, gfx1150, gfx1151, gfx1152 and gfx1153, and `gfx12-generic` code objects for gfx1200 and gfx1201. It then widens what hipDNN can route to them without adding per-shape kernels. ## Does rocKE need to learn generic targets first? Only for gfx12. rocKE IR never names a processor: the LLVM module carries the plain `amdgcn-amd-amdhsa` triple, and only the comgr ISA name selects the processor. `gfx11-generic` was already a catalog row with ISA backends in both engines. What was gfx1151-specific was in the SDPA layer: string gates (`arch == "gfx1151"`, `arch == "gfx1201"`), dispatch arches, CU lookup and packaging. This PR: 1. **rocKE targets** (`feat(rocke): catalog gfx12-generic ...`) - Adds `gfx12-generic` to both catalogs and both backend registries. - Regenerates the llvm20/22/23 intrinsic arch-domain tables. The new column equals gfx1201 cell for cell. The regeneration also records the previously unswept `raw.ptr.buffer.load.f16` row. - Adds `generic_targets` to `arch_specs.json`, plus `generic_arch_from_target_id()` / `rocke_generic_arch_from_target_id()`, mapping device names to generic targets. `base_arch_from_target_id()` keeps processors that have their own catalog row, so `arch="gfx1151"` and `arch="gfx1201"` still build processor-specific code; other members (gfx1100, gfx1200) resolve to their generic row. 2. **SDPA kernels and dispatch** (`feat(rocke): run RDNA WMMA attention as gfx11/gfx12 generic kernel sets`) - The WMMA atom now comes from the target's MMA catalog in both engines: the split-K gfx12 atom when listed, otherwise the gfx11 atom. - Transposed QK relies on the gfx11 operand duplication, so it is accepted on any gfx11 row and rejected on gfx12. Every other feature runs on both generations: dtypes, D16..D256, masks, windows, score features, LSE, additive bias, dense/BHSD/ragged/paged layouts, FP8 KV, output-column tiling, and distinct V widths. - Dispatch admits `gfx11-generic`, `gfx1151`, `gfx12-generic` and `gfx1201`, and reads live CUs from device 0 when it runs the requested object (any family member for a generic request, the named processor otherwise). - The selection policy is retuned for generic objects (see Results). Processor-specific gfx1151 requests keep #12710's wide-launch and D256 tile choices. - GPU tests and the benchmark compile for the device's generic target. Parity configs 136-142 replay representative configs at `gfx11-generic`; their golden hashes equal the replayed gfx1151 configs. 3. **hip-kernel-provider** (`feat(hip-kernel-provider): package RDNA WMMA attention as generic objects`) - rocKE UKDs accept `kernel_source.target: generic`. The packer compiles for the shard's generic processor, and shards stay concrete. - The compile target is recorded in observations and provenance, and the agreement check binds the two. - The IngestorGenerator expresses the target and per-group arch lists. - The catalog now ships to all 10 shards, with transposed-QK variants on gfx11 shards only. The census covers every shard. - The standard variants carry `scheduler_strategy: max-ilp`, the spec public dispatch selects for the same requests, so packaged objects get the generic-object tuning too. - The native score now prefers the 32-key single-wave transposed geometry. 4. **BSHD long-sequence fix** (`perf(rocke): decode heads fastest for head-interleaved dense attention`) - With head-interleaved K/V (BSHD, as from PyTorch's `view(B,S,H,D).transpose(1,2)`), the q-tile-fastest CTA order put one head's rows, a full token apart, in flight. On gfx1151 this lost cache and translation locality at long sequences. - Dense kernels now choose the CTA decode at runtime from the strides: heads fastest when K is head-interleaved, q-tiles fastest otherwise. The grid, ABI and object count are unchanged. 5. **Causal mask plus `window_left`** (`feat(rocke): accept a causal mask with an explicit window_left`): the cuDNN spelling of a causal sliding window now runs as the window `(window_left, 0)` instead of being rejected. 6. **Runtime head dims** (`feat(rocke): runtime head dims for RDNA WMMA attention`) - An opt-in `runtime_head_dims` spec flag lets one object serve every head width up to its compile-time `head_size` (a bucket maximum). Q/K and V/O widths (multiples of 16) and a V head count independent of K's arrive as kernel arguments. - Dense objects read K/V through bounded buffer resources at constant tile offsets, which keeps register pressure at the exact-size kernel's level. Dead Q/K tiles contribute exact zeros; dead V/output columns are never stored. - With the flag off, every existing config emits unchanged IR. The attention ABI is bumped to v6. 7. **Wider hipDNN routing** (`feat(hip-kernel-provider): route more SDPA graphs to RDNA WMMA attention`) - The pack now matches any Q/K and V/O head width that is a multiple of 16 up to 256, and V head counts dividing the query heads. - It also matches an additive FP32 or Q-dtype attention bias (`attn_mask`) with broadcast strides, and every hipDNN scale form: a value, a pass-by-value tensor (constant, or read at launch), or absent = 1.0. - Catalog per dtype: - runtime-head-dims buckets 64/96/128: no-mask, band, band+bias f32/q; - buckets 160/256: no-mask, causal, window, window+bias f32/q; - exact 192 and 256: no-mask, causal, window, kept where the exact kernel is measurably faster; - the 32-key transposed objects on gfx11. - That is **64 objects per gfx11 shard and 56 per gfx12 shard**. Previously, 40 objects covered only D64/96/128/256 with no bias. - Scores prefer transposed, then exact, then the tightest bucket, then mask-specialized objects. The 64-key transposed objects were dropped because the score never selected them. 8. **Tuned registry candidates** (`feat(rocke): register RDNA WMMA attention as tuned registry candidates`) - Each RDNA arch gets an opt-in tuned candidate built with `make_tuned_candidate`, like the gfx950/gfx942 dense candidates: `attention_<arch>_wmma_fwd`, pinned by algorithm `wmma_fmha_fwd` and spec_id `<arch>_wmma_fwd`. Tuning ids and `tuning_knobs` round-trip. - `wmma_rules.py` and `_AXES[(arch, "wmma")]` expose `v_lds_stage`, `causal_tile_skip`, `value_tile_size` and `scheduler_strategy`, plus `block_n` × `num_waves` for transposed QK on gfx11 only. `WmmaFmhaFwdSpec.resolved_scheduler_strategy()` names the unset scheduler so the knob can be projected into metadata. - `AttentionRequest.runtime_head_dims` selects the bucket objects. Only the RDNA WMMA candidates accept it, and it skips the fp16 D64 small-head `v_lds_stage` heuristic so the default spec matches the shipped bucket objects. - The existing `attention_gfx1151_wmma` candidate stays the public auto route and shares the policy and knob space. Public dispatch is unchanged for 29,940 requests, and IR is unchanged for all 1,648 WMMA specs and 163 parity configs. - `KernelId.request_hash` changes for every attention request because `normalized()` now includes `runtime_head_dims` (the same precedent as `emit_lse`); `spec_hash` is unchanged. - With #13593's `registry_enumerator` (`space: slice`, `num_cus` fixed), gfx11-generic and gfx12-generic manifests regenerate exactly the shipped 64 and 56 kernel specs and metadata (names and UUIDs ignored). The full knob space is 238 / 206 instances. The manifests stay out of tree until #13593 lands and the generic-target model is settled with #13129. ## Toolchain, runtime and hardware coverage - **Compile:** LLVM 20 or newer comgr (rocKE's oldest flavor). The LLVM 20 clang of ROCm 7.1 accepts `gfx11-generic`, `gfx12-generic`, `gfx1152` and `gfx1153`. Generic objects never need the member's name, so an older toolchain that cannot name gfx1152/gfx1153 still builds them. - **Load:** ROCm 7.0 or newer. ROCm 7.0.2 (gfx1100), 7.2.0, core 7.13 and core 10.0 load the generic objects. ROCm 6.4.0 on a gfx1151 rejects them (`hipErrorNoBinaryForGpu`), while it loads a code object v6 processor-specific gfx1151 object from the same toolchain. No gfx12 device with a runtime older than 7.2 was available. - **Hardware:** gfx1100, gfx1101, gfx1150, gfx1151, gfx1200 and gfx1201 were run. gfx1102, gfx1103, gfx1152 and gfx1153 are covered only by the LLVM generic-processor definition and were not run. ## Validation - **Device numerics** (rocKE GPU suites, generic objects, current head): | Device | Passed | Skipped | |---|---:|---:| | gfx1100 | 182 | 0 | | gfx1151 | 182 | 0 | | gfx1201 | 174 | 8 (transposed-QK tests) | The suites include the runtime-head-dims tests: - d16-d240 within buckets, dq != dv, and Hv in {1, Hk, Hq}; - bias with masks; - NaN-padded inputs proving no padded column is read into or written from the result; - rows ending exactly at the allocation end. An earlier round also ran gfx1101, gfx1150 and gfx1200 (134/134/126 before the new tests). - **Benchmark correctness:** the 54-case inference benchmark passes 54/54 against its FP64 oracle on every device, in every timed run. - **Code objects:** `gfx11-generic` objects are code object v6, machine 0x054, generic version 1. `gfx12-generic` objects are code object v6, machine 0x059. Both load on every device listed. - **C++/Python parity:** byte-identity is GREEN for all 73 families at llvm20, llvm22 and llvm23; `gfx1151_wmma_fmha_fwd` now has 157 configs (14 runtime-head-dims configs added). The native arch-identity CTest matches Python. - **Provider end-to-end** (gfx1100;gfx1151;gfx1201 shards, current catalog): - Census: 15/15. - `TestGfx1151WmmaAttentionMatchers`: 25/25. - hipDNN `quick_SdpaFwd*` integration bundles: **21 passed** (GPU reference), 20 skipped, 0 failed on each of gfx1100, gfx1151 and gfx1201. #12710's gfx1151-only build gives 17 passed and 24 skipped. - **Python suites:** rocKE CPU suites, IngestorGenerator (862 passed) and descriptor-packaging on a ROCm node (512 passed) all pass. The descriptor-packaging run has one failure, `test_enabled_user_site_runtime_preserved`, which fails identically on #12710's tree in the same cluster Python environment. - pre-commit passes. Golden hashes: `run_diff.py --check-golden` reports unblessed changes in this environment on `develop` itself (638 `ll`, 542 `ir_canonical` at `341b0b606e1`). The WMMA families' entries are re-recorded for the changed configs and report 0 unblessed. ## Results ### Selection policy (generic objects) Scope: relative timing of this selection policy against #12710's selection policy, measured before the BSHD decode and runtime-head-dims commits (both arms then shared the same kernels). The baseline is #12710's gfx1151-tuned profiles applied to the same kernels, with transposed QK limited to gfx11. Workload: the 54-case `benchmarks/gfx1151/attention` inference suite through public dispatch. Method: - Both arms compile for the device's generic target unless noted, and both arms pass the oracle gate in every run. - Two runs per arm, in ABBA order, in one exclusive allocation per device. - Each run times HIP-graph replays of 32 launches and takes the median of 7 batches. - Reported: the geomean of per-case ratios (this / #12710; lower is faster), with the per-case range. One device per processor. | Device | Target | Geomean | Per case | |---|---|---:|---:| | gfx1100 | gfx11-generic | 0.84 | 0.43-1.03 | | gfx1101 | gfx11-generic | 0.82 | 0.37-1.02 | | gfx1150 | gfx11-generic | 0.87 | 0.39-1.08 | | gfx1151 | gfx11-generic | 0.85 | 0.39-1.16 | | gfx1151 | gfx1151 | 0.90 | 0.49-1.03 | | gfx1200 | gfx12-generic | 0.69 | 0.32-1.03 | | gfx1201 | gfx12-generic | 0.63 | 0.29-1.08 | On gfx1151, `gfx11-generic` objects with this policy measured 0.90 (0.48-1.05) against processor-specific gfx1151 objects with #12710's policy, and 0.99 (0.71-1.07) against processor-specific objects with this policy. With #12710's policy alone, generic objects measured 1.06 (0.78-2.26) against processor-specific ones. The gap traced to scheduling under the generic register model, and the `max-ilp` scheduler closes it. Caveats: - Results hold at these measured configurations only, for inference forward attention. - The largest gains are on standard-path cases (tails, packed layouts, score features, decode, and every dense case on gfx12). - On gfx11, aligned dense transposed-QK cases change little apart from wide launches. ### BSHD long sequences B1 H16 bf16 noncausal, S=8192, dense, through public dispatch; median of 5×10 launches. | Device | Kernel | BSHD before | BSHD after | BHSD | |---|---|---:|---:|---:| | gfx1151 | d64 transposed QK | 82-96 ms | 29-42 ms | 15 ms | | gfx1151 | d64 standard | 97 ms | 26 ms | 21 ms | | gfx1151 | d128 transposed QK | 196-201 ms | 151-165 ms | 24 ms | | gfx1100 | d128 transposed QK | 23 ms | 15 ms | 22 ms | BHSD timings and gfx1201 are unchanged within noise. The remaining d128 gap on gfx1151 is not CTA ordering. At d128 with 16 heads each K/V row is exactly 4 KiB, so each 32-row tile touches 32 pages. My inference is that this exhausts the APU's address translation; discrete gfx1100 does not show it. ### Runtime head dims Timing of each runtime-head-dims object against the exact-size object for the same head width (lower is faster), bf16 noncausal, prefill B1 H16 S2048 and decode B8 Hq32 Hkv8 Sq1 Sk4096. Outputs match the exact object bit for bit. | d | Shipped object | gfx1151 prefill | gfx1151 decode | gfx12 prefill | gfx12 decode | |---:|---|---:|---:|---:|---:| | 64 | bucket 64 | 0.94 | 0.93 | 0.99 | 0.96 | | 80 | bucket 96 | 1.12 | 1.02 | 0.98 | 1.06 | | 96 | bucket 96 | 1.02 | 0.97 | 0.93 | 1.01 | | 128 | bucket 128 | 0.80 | 0.85 | 0.91 | 0.99 | | 160 | bucket 160 | 0.84 | 0.81 | 1.16 | 0.96 | | 192 | exact 192 (a 192 bucket would be 1.65 / 2.20 / 0.85 / 0.79) | 1.00 | 1.00 | 1.00 | 1.00 | | 256 | exact 256 (bucket 256: 1.56 / 1.44 / 1.07 / 1.06) | 1.00 | 1.00 | 1.00 | 1.00 | gfx12 columns: gfx1201 up to d128, gfx1200 from d160. These measurements chose the catalog. Up to 128, a bucket is within 2% of the exact kernel, or faster, when d equals the bucket; the one in-bucket width measured, d80 on bucket 96, costs 0.98-1.12×. Bucket 160 is faster than exact on gfx1151 and costs up to 1.16× on gfx12 prefill. Above that, gfx11 kernels sit at the register limit, so exact 192/256 objects stay; they are 1.4-2.2× faster than a runtime bucket on gfx1151. Widths with no exact object (176, 208, 224, 240) run on bucket 256 instead of being declined. Not done here: - A gfx12 transposed-QK variant (the gfx12 standard path already serves those shapes). - Split-KV decode for small batches with long contexts. - The gfx1151 d128 BSHD translation gap above. - Renaming the `gfx1151` module, candidate and pack identifiers (e.g. `hipkernel:Gfx1151WmmaAttention`, which now ships to every RDNA3/3.5/4 shard). - Deriving `TestGfx1151WmmaAttentionMatchers`' standard-catalog selection test from the shipped descriptor instead of its own copy of the catalog. ## Rebase onto develop and third-party review (head `e06957a02bd`) The seven feature commits are replayed onto `develop` (`e581a923f2b`), which already carries #12710 including its review round 2. Conflict resolutions keep develop's #12710 behavior: - The FP8 vector-alignment and byte-extent guards in the gfx1151 binding (`_check_vector_aligned`, `_check_byte_extent_i32`) stay in force with runtime head dims. - `emit_lse` replaces `return_lse` everywhere. - The RDNA census registers each wired arch only when `product` ships `hipkernel:Gfx1151WmmaAttention` for it (`hkp_product_ships_engine_for_arch()`). An independent review of the rebased diff found these issues, now fixed: - `91718c676b9`: the gfx12-generic catalog row didn't match develop's widened memory capabilities. `arch_specs.json` lacked `async_lds_max_dwords`, so `get_target("gfx12-generic")` raised `KeyError`, and the C++ row used the old 3-field initializer, which no longer compiles. Other gaps fixed in the same commit: - The gfx12-generic cells of develop's new `maxnum.f64`/`minnum.f64` arch-domain rows were measured on clang 20 (ROCm 7.1.0), 22 (7.2.0) and 23 (core-7.13); all are `ok`. - `generic_targets` is now the single gfx11-generic membership list. `target.py` reads it, and gfx1154 (already in develop's Python list) is mirrored to C++. - Known-arch expectations include gfx12-generic. - `e430c2da6ba`: a causal mask with `window_left=-1` resolved to an unbounded two-sided window and ran unmasked. The causal diagonal is now kept, with a regression test. - `b5bd11bf9e8` / `e06957a02bd`: the numeric tests use `emit_lse`, and the representative IR goldens are regenerated. Validation on this head: - Windows CPU lanes, rocKE `library/tests` + `platform/tests` (no GPU): the failure set is **identical to develop's** (comgr, `ml_dtypes` and Windows-only environment failures). The full-lane comparison (3,790 passed vs develop's 3,765) ran before the causal-bound fix and the llvm23 arch-domain cells; the dispatch, arch-identity and gfx1250-foundation suites re-run on the final head pass (719 passed, 0 failed). - Arch-domain coverage test passes for the llvm23 column; the llvm20 host-flavor gap for develop's `cluster.*`/prefetch keys exists on develop too. - `run_diff --check-golden`, C engine built from develop and from this head on Alola. The `ll` comparison used `ROCKE_LLVM_FLAVOR=llvm20`, the flavor the golden was blessed with. - No anchor that passes on develop fails here: `ll` passes 253 on develop and 280 here; `ir_canonical` passes 315 on develop and 336 here. - All 157 `gfx1151_wmma_fmha_fwd` configs match their goldens in both modes, and the 6 `gfx1201_wmma_fmha_fwd` anchors that are stale on develop now pass. - The C-vs-Python `ir` DRIFT families are identical on develop. - `pre-commit run --from-ref origin/develop --to-ref HEAD` passes. - Not re-run on this head: GPU numerics, the hipDNN integration bundles, and the 54-case benchmark (the device results above are from the pre-rebase head). This round's code changes are the catalog row, the causal window bound and test wiring. The emitted kernels are unchanged: all 157 `gfx1151_wmma_fmha_fwd` goldens match. ## Independent review, round 2 (head `feb7b7d7ee4`) Fresh review of the full diff against merge base `97fc1832835`, covering the arch catalog, kernel emission (both engines), dispatch, and hip-kernel-provider/IngestorGenerator. Fixed in this PR: - `34f8b95fe6e`: C++ `rocke_generic_arch_alias` kept its own gfx11 member list beside `rocke_ati_generic_targets`. It now reads the table, so `generic_targets` is the only membership list in both engines. Behavior unchanged; the native `rocke_arch_target_identity` binary passes and `test_cpp_target_identity_matches_python` passes against it (43/43, Linux build). - `feb7b7d7ee4`: `_resolve_num_cus` lent a sibling's CU count to a processor-specific request (`arch="gfx1151"` on a gfx1100 host took gfx1100's count for the output-tiling gate). Generic requests still read any family member, but processor requests now require that processor on device 0. A regression subtest fails before the fix and passes after it. The README now says only device 0 is read, and that 64-column D256 noncausal tiles apply only to gfx11-generic objects. Reviewed with no defect found: the gfx12-generic rows (JSON vs C++, field by field; arch-domain column equals gfx1201 in all three tables); runtime-head-dims memory safety (dead Q/K tiles zeroed after load, so NaN padding cannot leak; bounded rsrc extents within the i32 byte-extent guard); BSHD decode bijection; the kernarg order vs the native `kernelSignature()`/`launchWith()`; matcher legality vs the binder; yaml vs checked-in kdp.json (identical expansion). Fresh validation on `0db16e5c758` (new clones and venvs; the code changes since then are host-side selection/arch-alias code that GPU tests don't exercise): | Device | ROCm / LLVM | fmha numeric | dispatch numeric | fp8 decode | benchmark | |---|---|---:|---:|---:|---:| | gfx1100 | 7.2.0 / llvm22 | 41 | 141 (+27 subtests) | 4 | 54/54 | | gfx1100 | 7.0.2 / llvm20 | 41 | 141 (+27) | 4 | 54/54 | | gfx1150 | 7.2.0 / llvm22 | 41 | 141 (+27) | 4 | 54/54 | | gfx1201 | 7.2.0 / llvm22 | 33 (+8 transposed-QK skips) | 141 (+27) | 4 | 54/54 | - No gfx1151 node was usable: one was drained, and the other has ROCm 6.4 only. - Descriptor-packaging on gfx1100: `test_hkp_pack_rocke.py` passes 54/54 (base: 40/40). The full suite gives 618 passed and 1 failed. The failure is `test_enabled_user_site_runtime_preserved`; it fails identically on the base because the cluster system Python contains a nameless `argcomplete` dist-info. - Windows CPU, rocKE `library/tests/dispatch` + `platform/tests/core` + spec/multiarch: the failure set (14) is identical to the base, with 1925 passed vs 1891. - IngestorGenerator: 873 passed. - Golden changes are confined to `gfx1151_wmma_fmha_fwd` (126 per mode) and 6 previously stale `gfx1201_wmma_fmha_fwd` `ll` anchors. - pre-commit passes on the changed files. Left for follow-ups (not this PR): the remaining items under "Not done here"; device-0-only CU lookup (multi-GPU hosts with a mixed device 0); the descriptor-packaging probe's robustness to nameless dist-info. ## Review round 3 (AnaghaRaoAMD), head `601a18c94fa` - `ef7698617c1`: `gfx1200` requests resolve to `gfx12-generic`; before this, no RDNA candidate matched them. Both engines fold any `generic_targets` member that has no row of its own. The processor-specific live-CU check compares base archs. - `289619c0d15`, `601a18c94fa`: the WMMA GPU suites run from `rocke_library_gpu_pytest`, and the CPU spec, selection and tuning tests from `rocke_library_attention_pytest`, which is labelled for CI `quick` selection. - `90660821818`: on-device `causal + window_left=-1` test. - `2934d00bfc4`: an IngestorGenerator test keeps the WMMA catalog `target: generic`, which the native score assumes. - `bc71331224e`: the engine declares graph schema 1.2.0, so runtime pass-by-value scale graphs reach the matcher; before this, every `SdpaFwdRuntimeScale` graph was declined. Override-shape graphs are declined. Validation: - rocKE GPU suites (`2934d00bfc4`, from fresh clones): fmha 41 (33 + 8 transposed-QK skips on gfx12), dispatch 143, fp8 decode 4, benchmark 54/54, on gfx1100 (llvm22 and llvm20), gfx1150, gfx1201 and gfx1200. - hipDNN forward SDPA, all tiers (engine pinned, GPU reference; `bc71331224e` code): 88 passed / 14 skipped / 0 failed on gfx1100, gfx1201 and gfx1200. The skips are FP8 or varlen graphs. - gfx120X Multi-Arch CI dispatch on `601a18c94fa` (https://github.com/ROCm/rocm-libraries/actions/runs/38698822336): `hipkernelprovider` and `hipdnn-integration-tests` pass on Linux gfx1200 and Windows gfx120X. The WMMA suites ran on the gfx1200 runner.
This was referenced Oct 16, 2026
6 of 8 tasks
CongMa13
added a commit
that referenced
this pull request
Oct 16, 2026
…t tests PR #13129 made the generic-target table a required argument of `load_flat_input()` and `run_pipeline()`. PR #13150 then added tests for `kernel_source.target` that still call the old signatures. Each PR passed CI alone. Together they fail 9 tests in `test_hkp_pack_rocke.py`, and the `hip-kernel-provider-hkp-pack-quick` ctest fails on Linux and Windows. Pass `GENERIC_TARGETS` to `load_flat_input()` in three tests. Call the `pack_helpers.run_pipeline()` wrapper in the fourth. The wrapper supplies `generic_targets_json`. A reviewer of #13806 proposed this change. Verified with the `rocm_kpack` commit that CI pins: - Before: 9 failed, 41 passed in the quick tests of this file. - After: 50 passed. - Over the whole quick selection of `descriptor-packaging`, the change fixes exactly those 9 test ids and breaks none. - No other caller of the two functions uses the old signatures. Not run here: the hipcc and comgr tests of this file. They need CI. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
CongMa13
added a commit
that referenced
this pull request
Oct 16, 2026
…develop (#13826) # Fix three test breaks from merge races on develop Branch: `users/congma13/fix/develop-merge-race-tests` JIRA ID : AICK-2516 --- # Summary — For human reviewers ## The short version Three checks fail on develop after PRs #13121, #13129 and #13150 merged. Each PR passed CI on its own branch. The code change and the tests that call it came from different PRs. No CI run covered them together. This PR updates the three stale call sites. It changes only test and gate code. ## What broke 1. **`rocke_library_attention_pytest` (ctest #41).** `test_wmma_fmha_fwd_spec.py` imports `_wmma_op_id_for_arch`. #13150 removed that helper. Pytest stops at collection with an `ImportError`. All four `hipkernelprovider` jobs fail, on Linux and Windows. 2. **`hip-kernel-provider-hkp-pack-quick` (HipDNN Superbuild).** #13129 made the generic-target table a required argument of `load_flat_input()` and `run_pipeline()`. #13150 then added tests that call the old signatures. Nine test ids fail, on Linux and Windows. 3. **gfx1201 binding parity.** #13150 changed `cfgs_gfx1151_wmma_fmha_fwd()` to return `(config, arch)` pairs. The gfx1201 family slices that list and passes the pairs to the binding. All 12 configs fail. CI does not build the pybind module, so it never runs this gate. ## What changes - **Attention test:** call `_wmma_attn_op_id(ArchTarget.from_gfx(arch), dtype)`. This is the change from #13806, cherry-picked with its author. #13806 can close after this PR merges. Its own CI fails on break 2. - **hkp-pack tests:** pass `GENERIC_TARGETS` to `load_flat_input()` in three tests. Call the `pack_helpers.run_pipeline()` wrapper in the fourth. The wrapper supplies `generic_targets_json`. - **Parity family:** keep each config and drop its arch. ## How this was tested - [x] **ctest #41 test files:** 75 tests and 438 subtests pass. - [x] **hkp-pack quick tests of `test_hkp_pack_rocke.py`:** 9 failed and 41 passed before. 50 pass after. The run used the `rocm_kpack` commit that CI pins. - [x] **Whole `quick` selection of `descriptor-packaging`:** the change fixes exactly those 9 ids and breaks none. - [x] **gfx1201 binding parity:** 0 of 12 configs matched before. 12 of 12 match after. - [x] **Byte identity of both FMHA families, at both LLVM flavors:** GREEN. gfx1201 has 12 configs and gfx1151 has 157. - [x] `pre-commit run --from-ref origin/develop --to-ref HEAD` passes. - [ ] **Not run locally: the hipcc and comgr tests of `test_hkp_pack_rocke.py`.** They need CI. - [ ] **Not run locally: the full HipDNN Superbuild and Multi-Arch CI.** They run on this PR. ## Reviewing this The branch has three commits. It changes 3 files: +14 and −8. One problem remains on develop. This PR does not cause it and does not fix it. 23 of 157 gfx1151 binding-parity configs mismatch (cfg71–93). --- # Summary — For AI reviewers ## Change surface Paths are relative to `dnn-providers/hip-kernel-provider/`. ``` descriptor-packaging/tests/rocke/test_hkp_pack_rocke.py +5 -5 rocke/library/tests/test_wmma_fmha_fwd_spec.py +5 -2 rocke/platform/cpp/bindings/prove_parity_binding.py +4 -1 ``` No kernel, engine or emitter code changes. ## Properties to verify 1. **Each call site matches the current signature.** `GENERIC_TARGETS` comes from `tests/pack_helpers.py`, and the file already imports it. `run_pipeline` is the `pack_helpers` wrapper. It forwards `**kwargs` and supplies `generic_targets_json`. Confirm that no caller of `load_flat_input` or `pipeline.run_pipeline` omits the new argument. A search of `descriptor-packaging` and `projects/hipdnn` finds none. 2. **The attention test calls the helper that `is_valid_spec` calls.** `_wmma_attn_op_id(target, dtype)` takes an `ArchTarget`. The four (arch, dtype) cases keep their expected op ids. 3. **The gfx1201 family receives plain dicts.** `FAMILIES` sets its arch to `"gfx1201"`. The harness loop treats each config as a dict when `fam_arch` is not `None`. The gfx1151 family sets `fam_arch` to `None` and holds pairs. 4. **The first 12 rows still mirror the gfx1201 emitters.** Byte identity reports `gfx1201_wmma_fmha_fwd configs=12 bad=0`. ## Repo rules that apply - **Byte identity.** No emission code changes. The gate is GREEN at both LLVM flavors. - **Relative paths; cross-platform.** The change adds no absolute path and no script. - **Compliance (`platform/AGENTS.md`).** The text names hardware by gfx identifier only. It has no product name, internal link or performance figure. - **Matching tests.** Every changed file is a test or a gate. ## Known-weak points - **The `[:12]` slice couples gfx1201 to gfx1151.** A reorder of the first 12 gfx1151 rows breaks the match. Binding parity then reports DIFFs, not an import error. - **The attention test imports a private helper.** `_wmma_attn_op_id` starts with an underscore. A rename breaks the test again. ## Scope calibration - A finding about the 23 gfx1151 parity mismatches is out of scope. They reproduce on develop without this PR. - A finding that this PR duplicates #13806 is deliberate. #13806 cannot pass CI until break 2 is fixed. ## What review cannot establish from the diff - **The hipcc and comgr tests.** They need CI. - **Binding parity.** It needs a C++ build and the pybind module. The claim is 12 of 12 validated for gfx1201. --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
TheRock Submodule Bump ActivityNewest first
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Kernel descriptors can name a target GPU by exact id (
gfx1151) or leavearchempty. This adds LLVM generic targets (gfx11-generic,gfx12-generic) to the kernel ingestor and the HKP descriptor packer.On each device, an
archentry ranks: explicit id > generic target containing the device > empty list. Kernels with the same metadata tuple are resolved by that rank, so a member-specialized kernel beats a generic one and the generic remains the fallback on other members. Two kernels with the same tuple at the same rank are still an error. A generic entry is compiled once and packed under the generic's spelling into each selected member's folder.JIRA ID : ALMIOPEN-2877
Risk Assessment
Risk level 3. This changes plugin SDK arch matching used by every ingestor pack, adds a required CMake input to HKP packaging, and tightens pack-time and load-time validation. Packs that name only concrete ids or leave
archempty behave as before, with two exceptions:archunder a KDP that has anarchis now a pack-time error. No in-repo descriptor is affected.At load, a pack with an unknown generic name, a list holding a generic and one of its members, or a UKD that disagrees with its KDP's generics is dropped with an ERROR log. Other packs still load.
ASIC Coverage
Matching, shadowing, selection, and packing are host-side and arch-independent. Because they change how kernels are chosen at load, the production lanes that ship descriptors (gfx942, gfx950) should run PR CI before merge.
Verified locally: Linux builds for gfx1151 (a member of
gfx11-generic) and gfx942 (a nonmember), and a TheRock build for gfx1151 and gfx950. Device runs on gfx1100 and gfx1151 (both members ofgfx11-generic) pass. Windows andgfx12-genericon gfx1200/gfx1201 are left to PR CI and are not covered locally.Testing Summary
KpackModuleCachearchive selection.gfx11-generic->gfx1100), and a compile error in the generic source turns the probe red.Testing Checklist
hipdnn_plugin_sdk_tests- Status: Passed (682 passed, 7 GPU skips)ctest -R hipdnn_gpu_generic_targets_- Status: Passed (8/8; superbuild only, not staged into TheRock)hip-kernel-provider-hkp-pack-quick,hip-kernel-provider-hkp-pack- Status: Passed (gfx1151 and gfx942 builds)hip_kernel_provider_tests,hip-kernel-provider-hkp-census- Status: Passed (882 passed, 51 device skips; census 5/5)pytest projects/hipdnn/tools/IngestorGenerator/tests- Status: Passedhkp-probe-ctests - Status: Passed (3/3 on gfx942)IntegrationGpuKernelIngestor*on gfx1100 and gfx1151 - Status: Passed (20/20 on each, unmodified and withpointwise_addpacked asgfx11-generic; the device is served by the generic archive)Technical Changes
plugin_sdk/data/gpu_generic_targets.jsonmaps each generic to its member processors, with LLVM provenance. CMake validates it at configure time, C++ reads a generated constexpr header, and the packer, probes, and IngestorGenerator read the same JSON.entryTier(entry, device)drives ranking, admission, andKpackModuleCache. Tuple uniqueness competes per tier, and the best tier wins per device.KpackModuleCache::selectArchprefers the exact device key, then a generic key containing the device.--generic-targets-jsonis a required option, andhkp_add_packagingfails clearly when the table is missing.generic_passesinpipeline.pylets the probe tooling pack and assert generic entries. A*-genericname absent from the table fails configure, as it fails the packer.verify_variant_sets, and the desk check follow the same tier rule.