Repository navigation
feat(hipdnn,hip-kernel-provider): hip and hsaco multi-launch kernels, launch-set checks and an end-to-end test - #14215
Merged
GarbyGilbertAMD merged 45 commits intoOct 30, 2026
Conversation
…vice A kernel descriptor carrying a sources map had been exercised only in pieces: the loader and state manager on hand-built definitions, and launchKernelDefinition over one module. Nothing loaded such a kernel from a tree, selected it through an engine and ran its launches with a workspace between them. A new test descriptor set, multi_launch, packs one HIP translation unit exporting two kernels. It is staged beside the unit and integration roots rather than inside them, so no Container loads it and the engine lists of both test binaries are unchanged. At run time the test rewrites the packed descriptor into a UKD 1.1 sources kernel with a produce and a consume launch, loads that tree on its own and registers a test-only graph match and dispatch handler. On a device the engine is selected, planned and executed: produce writes the workspace, consume reads it, and the exact output is reachable only when both launches run, in order, over the same workspace. A host-only case checks that a handler declaring fewer launches drops the kernel at load and logs both launch sets. ALMIOPEN-2973
The state manager and the descriptor validator each built the set of a kernel's launch names from its sources map with their own loop, and the two must agree for the validator's stub handler to model the runtime. launchNamesOf() in the plugin SDK's descriptor header now computes the set once, empty for a single-launch kernel, and both call it. ALMIOPEN-2973
… launch sets A pack's native dispatch handler declares one set of launch names, and the state manager drops a kernel whose set differs, so a variant with another launch set belongs in its own pack. The packer accepted such a pack with exit 0 and left the mistake to the validator and the runtime. hkp_pack now checks each KDP of an arch shard before packing and fails naming the KDP and every kernel's launch set; a single-launch kernel beside a multi-launch one fails the same way, and the order of sources keys does not matter. Two comments in the pipeline described a field a launch cannot report as failing loudly, which is not the rule the code applies: each declared field must be reported by at least one launch and match wherever reported. They now say so. New tests pin behaviour that survived mutation before: compile grouping in agreement.verify by library and toc key rather than object digest, the desk-check reader taking each launch's object from its own toc key, and launch order carrying no meaning in packing, shipping and verification. The shared _restamp test helper moves to pack_helpers. ALMIOPEN-2973
The probe derive tool prefixes each launch's spec fields with its launch name and keys a compile group by launch, and both tools sort launches by name. Existing tests passed with the prefix removed, the launch dropped from the group key, or either sort removed. New cases put the varying field in the launch that sorts first, separate two kernels that differ only in launch names, and permute the authored order of sources for derive and assert. The assert tool's header now states that the kind, digest, signature and symbol checks apply to each sources entry. ALMIOPEN-2973
…unch sets The descriptor validator's stub handler declares the launch set of a pack's first kernel, so a pack whose kernels name different sets loads its engine but drops the odd kernel and fails validation. A committed fixture, multi_launch_mismatch, and a round-trip test pin that outcome and the error naming both sets. ALMIOPEN-2973
No shipped config declares launches, so the native stub's launchNames() override and the census arm that checks each kernel's sources were only ever checked as text. The real compile tests now render both for a multi-launch config, compile the stub against the plugin SDK headers, parse the census test and check that launchNames() returns exactly the configured launches. A further test checks that verify_variant_sets gives the same verdict whatever the order of a descriptor's sources keys. The README says what the stub declares and that variants needing different launch counts are split into separate engines. ALMIOPEN-2973
…ce handler RFC 0017 section 15.5 now lists the three plugin SDK members the kernel-descriptor half adds, launchNames() on the dispatch handler and sources on both the kernel definition and the kernel descriptor, together with the UKD 1.1 version bump. Section 15.4 refers to kernel slots as a separate draft RFC rather than citing a document that is not in the tree. The ingestor-engine native-pack guide gains a worked multi-launch handler that declares its launches, sizes the workspace between them, builds each launch with launchKernelDefinition and buildIngestorKernelCode, and enqueues them in order. ALMIOPEN-2973
A multi-launch kernel descriptor could author its sources entries only as rocke builds, so a HIP or prebuilt hsaco multi-stage kernel had no way through the packer. hkp_pack now accepts hip entries in the single-launch hip shape (source, entry, build) and hsaco entries in the single-launch hsaco shape (file, symbol), one kind per map. Entries that share a translation unit and build, or an hsaco file, compile or read once and share one code object and toc key, while each launch keeps its own symbol and the signature read for that symbol. role stays a rocke field and is refused on the other kinds, an hsaco map needs a UKD arch as a single-launch hsaco kernel does, and embedded_source entries stay refused. The packed descriptor records each launch's authored identity under provenance.sources with the shared origin kind on top. Consumer agreement and its published evidence remain rocke-only, so a declaration with metadata fields over a hip or hsaco map is refused, as it is for a single-launch kernel of those kinds, and the desk check reports such a kernel as unclaimed. A translation unit may export kernels no launch names. ALMIOPEN-2973
…unch The packaging probes asserted every sources launch against rocke effective-spec evidence and role bindings. A kind now states whether its launches carry that evidence; for hip and hsaco launches the assert checks archive membership, digest, symbol and the signature read from the blob, while rocke keeps its evidence checks. Derive groups hip launches by source and build and hsaco launches by launch name, as the docstrings now say. ALMIOPEN-2973
…orGenerator launches was accepted only on rocke kernel sources. A launch now inherits the kind of its kernel source, and its overrides are the fields of that kind: source, entry and build for hip, file and symbol for hsaco, and the existing rocke keys. role on a hip or hsaco launch is a configuration error and embedded_source still takes no launches. verify_variant_sets keys a hip launch by source, entry and build and an hsaco launch by file and symbol, so two launches of one translation unit no longer collide. ALMIOPEN-2973
…n device The multi_launch fixture is now authored as a UKD 1.1 hip sources kernel with produce and consume launches on one translation unit, and the end-to-end test loads what hkp_pack produced instead of rewriting a single-launch descriptor at run time. It checks that both launches share one code object and toc key with their own symbols before running them through the engine. RFC 0017 now lists rocke, hip and hsaco as authored sources kinds and states that consumer-agreement evidence stays rocke-only. ALMIOPEN-2973
github-actions
Bot
added
documentation
project: hipdnn
project: hip-kernel-provider
labels
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
…heir bugs The engine-selector revision test changed only one launch's symbol, so dropping a launch's sha256 or toc key from the digest, or keeping only the last launch, went unnoticed. It now changes each field of each launch and expects a new revision every time. A launch name that starts with an underscore gains a refusal row. The launch-definition test for two names sharing one entry now checks that each name gets its own complete copy, and a device-free case shows that each launch's recorded signature is checked against its own expected signature before its archive is opened, replacing a cache-size assertion that could not fail. launchKernelDefinition builds the single-launch definition without copying the sources map it then discarded, and a loader comment uses the RFC's wording. ALMIOPEN-2973
…rces kernels The desk check's toc-key uniqueness compared each kernel's launches as one tuple, so a single-launch kernel reusing one toc key of a multi-launch kernel passed. Kernels now count as one whenever any launch of one shares a toc key with any launch of another, while the launches of one kernel may still share a module. An unknown authored sources kind now gets the same refusal message as every other refused kind, from one helper. The prewarm failure message names every launch of a shared compile, and a loop variable no longer shadows a parameter in the agreement check. A test pins that an accessor raising AttributeError fails the observation instead of counting as unreported. README and test wording are corrected. ALMIOPEN-2973
…e gate An IngestorGenerator launch override of target or role set to an empty string was accepted, and an empty target silently made that launch native. Both are now configuration errors. verify_variant_sets gains a nesting case that differs only in a launch name, the multi-launch validator round trip asserts that no kernel was dropped for its launch set, and the README tool rows describe packed keys, sources entries in catalog_diff and per-launch inventory counts. ALMIOPEN-2973
…ch gate pending The §15.1 example lists only kind, library and symbol for each entry, so it is now marked as abridged. The loader check that every Launch is filled is described as arriving with launches[], with the handler's declared launch names standing in until then, and the state manager is named as the plugin SDK's. ALMIOPEN-2973
…ne launch set The pack-time launch-set check grouped kernels per KDP, but the loader resolves a dispatch handler from the dispatch_symbol of a KDP's UDD and compares every kernel it serves against that handler's launch names. Two KDPs that each agree with themselves but share a UDD, or name one dispatch symbol from two UDDs, therefore packed cleanly while the loader dropped one of them. The check now groups the shard's kernels by dispatch symbol and fails the pack naming the symbol, each KDP and each launch set. The probe end-to-end fixture fused one pointwise KDP into a multi-launch kernel while its siblings stayed single-launch on the same UDD, which is exactly that shape, so the fused KDP now gets a UDD and dispatch symbol of its own. The packaging README describes the check at handler scope and notes that it sees one packaging root. ALMIOPEN-2973
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #14215 +/- ##
===========================================
- Coverage 77.31% 75.93% -1.38%
===========================================
Files 2893 2884 -9
Lines 486189 481016 -5173
Branches 73466 72380 -1086
===========================================
- Hits 375882 365242 -10640
- Misses 85488 90732 +5244
- Partials 24819 25042 +223
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
…f in the launch-set check The launch-set check looked up every KDP's UDD and failed with a KeyError on a KDP whose dispatch names no UDD of the root, which the packer accepts (a null dispatch, as the minimal pack roots in the packer tests author). Such a KDP has no handler it could share, and the loader refuses its pack outright, so the check now holds it to one launch set on its own and names it in the refusal. Two KDPs that both name no UDD are not grouped. ALMIOPEN-2973
…IOPEN-2973/follow-ups
…inters unsortedDefinitions returns pointers into the state manager's own definitions rather than copies, so the multi-launch end-to-end test holds the kept kernels as pointers and dereferences the one it inspects before the state manager moves into the engine. ALMIOPEN-2973
BradPepersAMD
added a commit
that referenced
this pull request
Oct 25, 2026
RFC 0021 §3 (kernel slots, draft #13839) names "one UKD whose code object holds the dependent kernels, launched by symbol names its KMD records" as an accepted alternative to RFC 0017 §15.1 `sources`, with a split-KV kernel and its reduction as the example: FlyDSL's decode and RMSNorm backward objects. The note now says so, and that a move to hsaco `sources` entries (#14215) is optional. JIRA ID: ALMIOPEN-2953 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…IOPEN-2973/follow-ups # Conflicts: # dnn-providers/hip-kernel-provider/CMakeLists.txt # dnn-providers/hip-kernel-provider/descriptor-packaging/README.md # dnn-providers/hip-kernel-provider/descriptor-packaging/cmake/HkpPackaging.cmake # dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/test_descriptors/README.md # dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/CMakeLists.txt # projects/hipdnn/tools/IngestorGenerator/README.md
A multi-launch kernel carries its launches in `sources` and leaves `source` default, and a default KernelSource reads as an EMBEDDED_SOURCE kernel naming no file. A reader that looked at `source` alone therefore saw an embedded_source kernel. Plan saving did: it refused a multi-launch plan as having source kind 'embedded_source', and the engine's serialization behaviour note took the same wrong path to the right answer. Descriptors.hpp gains two helpers that look at `sources` first. singleLaunchSource() returns the one source of a single-launch kernel and nullptr for a multi-launch kernel, for a reader that needs exactly one code object. allLaunchSources() asks a question of every launch, whatever the kernel's shape. Plan capture, the serialization behaviour note and buildForSourceKind read through singleLaunchSource(), and the state manager's adaptability check through allLaunchSources(). Capture now refuses a multi-launch kernel for having several launches. The selector digest in MakeEngine keeps reading both fields as they are, since existing selector revisions hash them. ALMIOPEN-2973
singleLaunchSource() returns the address of the kernel's own `source` member, and its const reference parameter binds a temporary, so a caller that passes one keeps a dangling pointer. The comment now says the result is valid only while the kernel is and that a temporary must not be passed. ALMIOPEN-2973
…s launches The native-pack multi-launch sketch copied the workspace pointer into a local because launch() takes its arguments by address. launch() takes a forwarding pack and the address of its own parameters, so any expression works, as the same calls show by passing `in.ptr` and `out.ptr` directly. The sketch now passes `workspace` to both launches and says in the order comment that decode writes the slice and merge reads it. ALMIOPEN-2973
The notes under the native-pack multi-launch sketch restated the section intro (the state manager drops a kernel with another launch set, the handler owns launch order) and the LaunchKernelDefinition.hpp contract (the INVALID_VALUE cases). They now list only what neither says: that the generator refuses mismatched launch names, so another launch set needs its own engine, that the workspace is not zeroed, and that launchNames() is provisional. The helper list names `Binding`, which the sketch uses without defining, and the section points at Gfx1151WmmaAttentionNative.cpp as a shipped handler. ALMIOPEN-2973
…epare() TODO The generated native stub's multi-launch prepare() TODO said hkp_pack compiled the dialect's rocke kernels to KPACK. The branch is selected by the presence of launch names, which hip and hsaco kernels now carry as well, so the comment says that hkp_pack compiled or packed every launch to KPACK, whether rocke, hip or hsaco. ALMIOPEN-2973
… hsaco declarations The packaging README's provenance note for hip and hsaco `sources` UKDs said one declaration is both refused and reported unclaimed. The packer refuses a declaration that states `metadata_fields` over such a UKD, while the desk check reports the kernel as unclaimed for a declaration without them. The note now names the two cases separately. ALMIOPEN-2973
… its handler MultiLaunchHarness declared its handler, then its symbol scope, then its scratch tree, while its comment said the tree outlasts both. Members are destroyed in reverse order, so the tree was removed first, while the handler and the registered symbols that point at it were still alive. The tree is now declared first, so it is removed last, as the comment says. ALMIOPEN-2973
…serialization note supportsExecutionPlanSerialization() refuses a set holding a kernel with a `sources` map, as captureIngestorPlan() cannot save one, but no test built such a kernel. The new case takes a saveable single-kpack set, gives its kernel two kpack launches while leaving `source` kpack, and expects false, so a gate that read `source` instead of the launches would fail it. ALMIOPEN-2973
Edits to the IngestorGenerator README launches paragraph, the KernelSource.launches comment and the _build_launches docstring left one line far past its neighbours' width and one orphaned short line. The three passages are rewrapped to the width around them, with no change in wording. ALMIOPEN-2973
…k by _launch_signature The packer's toc_key collision check built a single-launch UKD's input signature inline, per producer, beside _launch_signature(), which builds the same tuples for each launch of a `sources` UKD. A producer or field added to one copy and not the other would let a collision through on that path. Both paths now call _launch_signature(); the tuples it returns for rocke, hip and hsaco are the ones the inline code built, so the check behaves as before. ALMIOPEN-2973
…ers from pack_helpers test_hkp_pack_sources_hip_hsaco.py defined its own `ARCH`, `_read` and `_load_kpack`, each identical to the one pack_helpers already provides to the other packer tests. The module now imports them, so a change to the shared arch or kpack loading reaches these cases too. ALMIOPEN-2973
…and-in The rocke and the hip/hsaco launch probe tests each carried their own copy of the loader that imports hkp_probe_assert and of the kpack archive stand-in its checks read, and the derive-root tests carried a third copy of the loader. probe_support, which the probe tests already import, now provides `tool_module()` and `StandInArchive`, and the three modules use them. Each test module keeps its own fixture constants, signatures and descriptor builders. ALMIOPEN-2973
…est setups once The packer refused an hsaco UKD with no `arch` in two places, one for a single launch and one for a `sources` map, with text differing by one word. _require_hsaco_arch() now holds the check, its reason and its message for both. The desk-check tests that read each launch out of a stand-in archive built the same fake kpack module twice and now share one helper, and the derive-root tests build a rocke `sources` UKD through the same _sources_ukd() as the hip and hsaco ones. ALMIOPEN-2973
GarbyGilbertAMD
marked this pull request as draft
October 27, 2026 14:56
…low-ups' into users/ggilbert/ALMIOPEN-2973/follow-ups
…IOPEN-2973/follow-ups
GarbyGilbertAMD
marked this pull request as ready for review
October 27, 2026 17:29
BrianHarrisonAMD
approved these changes
Oct 27, 2026
…IOPEN-2973/follow-ups # Conflicts: # projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md
…IOPEN-2973/follow-ups # Conflicts: # projects/hipdnn/plugin_sdk/include/hipdnn_plugin_sdk/ingestor/Descriptors.hpp
Contributor
TheRock Submodule Bump ActivityNewest first
|
adickin-amd
pushed a commit
that referenced
this pull request
Oct 30, 2026
…ward (RDNA, CDNA) (#14111) JIRA ID: ALMIOPEN-2953, ALMIOPEN-2954, ALMIOPEN-2955 Supersedes #13712, #13713 and #13714. ## Summary Adds **FlyDSL** as a second kernel source for the hip-kernel-provider's kernel ingestor, beside rocKE: a pipeline that ships prebuilt FlyDSL code objects, and eight engines on it. | Area | Engines | Targets | |---|---|---| | RMSNorm | `hipkernel:flydsl_rmsnorm` (inference and training forward), `hipkernel:flydsl_rmsnorm_bwd` (dx, dgamma) | gfx11-generic, gfx12-generic, gfx90a, gfx942, gfx950 | | SDPA forward, RDNA | `hipkernel:flydsl_sdpa_rdna`, `hipkernel:flydsl_sdpa_rdna_decode`, `hipkernel:flydsl_sdpa_aiw` (gfx12 only) | gfx11-generic, gfx12-generic | | SDPA forward, CDNA | `hipkernel:flydsl_sdpa_cdna`, `hipkernel:flydsl_sdpa_cdna_decode`, `hipkernel:flydsl_sdpa_cdna_dualwave` (gfx950 only) | gfx90a, gfx942, gfx950 | ## How it works - **Prebuilt objects, no compiler at build or run time.** FlyDSL kernels are compiled ahead of time into code objects (`.hsaco`). `flydsl/gen_descriptors.py` generates their descriptors as a `FlyDSL/` bundle, and the provider's existing packer puts them in each arch's kpack shard. Neither the build nor the runtime needs FlyDSL, a kernel compiler or Python at run time. - **Objects live in DVC.** Each `FlyDSL/<op>/<target>/` directory (1,453 objects, about 29 MB) is a DVC output on the `ingestor` remote that rocKE's descriptors use; git holds the 22 `.dvc` sidecars, the op-level descriptors, the generators and docs. Fetch with `dvc pull -r ingestor dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/FlyDSL/*/*.dvc`. The provider's configure-time DVC check covers FlyDSL: a missing directory warns and that target ships no FlyDSL kernels; `HIPKERNELPROVIDER_REQUIRE_DVC_DESCRIPTORS=ON` (CI, release) makes it an error. - **Integrity.** Each `manifest.json` records every object's SHA256 and a digest of that op's own sources. `gen_descriptors.py --check` runs before every pack and fails on any mismatch, so a source edit without regeneration fails the build; `check_shards.py` verifies every shard after the pack. Regeneration is byte-reproducible with the pinned toolchain (`REGEN.md`). - **Generic targets.** RDNA objects are built once for `gfx11-generic` / `gfx12-generic` and copied into every member arch's shard. - **Runtime arguments.** Shapes, strides, head counts, scale, epsilon, mask bounds, bias strides and the LSE output are kernel arguments, so one object serves every shape of its class. Only tiling, head-dim tiers and measured RMSNorm widths are baked. - **Declines.** Each matcher gates every graph field it reads and declines everything else, so another engine gets the plan. `flydsl/COVERAGE.md` lists every accept and decline rule with its test. ## What each engine serves - **RMSNorm** (153 objects per target): bf16, f16, f32, and bf16/f16 with an f32 scale; training forward writes a per-row f32 inv_rms; any finite epsilon, fixed or pass-by-value; the frontend's normalized-axes rule (including cuDNN's `x {B*S, D, 1, 1}` layout). Backward is FlyDSL's deterministic two-stage kernel (dgamma bitwise reproducible). Baked widths were kept only where they measured at least 15% over the generic tier on two archs. - **SDPA RDNA:** bf16/f16 at head_dim 64/96/128/256 plus generic tiers for any multiple of 8 up to 256; causal at either corner, bands, sliding windows; additive bias (f32 or the query dtype, any broadcast); attention sinks; MLA 192/128 and wide single-KV-head decode (576/512, 512/512); MHA/GQA/MQA. - **SDPA CDNA:** bf16/f16 at head_dim 64/128/256 with bucket objects between (gfx90a also 96/160/192/224); MLA 192/128 and absorbed-MLA decode (576/512); bias and sinks; BSHD, BHSD and packed QKV strides. Each class ships a default tile plus measured companion tiles. - **Same answer on every family.** RDNA, CDNA and aiw give the same result at the edge cases (keyless rows, −inf and lowest-finite bias rows, NaN rows, sinks, decode splits), matching PyTorch and hipDNN's reference (`COVERAGE.md` §5.7). - **`left_bound`** follows hipDNN's cuDNN meaning (#14483): `L` keeps the L keys ending at the diagonal; the kernels receive `L - 1`, and 0 declines. ## Build flag and CI - `HIPKERNELPROVIDER_ENABLE_FLYDSL` is **OFF by default**, like rocKE, and takes effect only with `HIPDNN_ENABLE_KERNEL_INGESTOR=ON`. A default rocm-libraries build is unchanged and needs no `dvc pull`. TheRock turns FlyDSL on through its `HIPKERNELPROVIDER_ENABLE_FLYDSL` flag (default ON, ROCm/TheRock#9615) together with the ingestor (TheRock #9375), and can turn it off there; the hipDNN superbuild CI passes `-DHIPKERNELPROVIDER_ENABLE_FLYDSL=ON`. - FlyDSL engines and tests compile whenever FlyDSL content exists, independent of `GPU_TARGETS` (TheRock's multi-arch build ships one shard's host half to every arch); device tests decide at run time whether the device's shard is staged. Packing, the shard check and the census follow the requested archs. - `THIRD_PARTY_LICENSES.md` (FlyDSL Apache-2.0, AITER and AOTriton MIT) installs under `share/doc/hip-kernel-provider/flydsl/`; TheRock packages it through the `doc` component added by ROCm/TheRock#9476. - This PR's TheRock Multi-Arch CI builds FlyDSL, runs its device tests and runs the FlyDSL integration entries on Linux gfx94X and Windows gfx110X once its TheRock pin includes ROCm/TheRock#9615. ## Changes outside `flydsl/` - Provider `CMakeLists.txt`: the option, `add_subdirectory(flydsl)` under the ingestor gate, and the DVC check extended to FlyDSL. - `HkpPackaging.cmake`: `FlyDSL` in `HKP_DESCRIPTOR_FAMILIES`. - Ingestor: one pack-table entry and module cache per engine; natives under `kernel_ingestor_engine/packs/Flydsl*`, with shared headers for RMSNorm, RDNA SDPA, CDNA SDPA and both SDPA families. - `descriptors/FlyDSL/`: op-level descriptors, `.dvc` sidecars and per-target `SOURCE.md`. - Tests: `main.cpp` adds the product shard as an additive descriptor root when FlyDSL is active; one engine suite and one census suite per engine, plus a per-object SDPA suite; `TestContainer`'s engine bound raised from 8 to 32. - Integration suite: one external integration entry per FlyDSL engine and architecture (55), pinned with `--test-engine`, checked by the GPU reference executor against the shared RMSNorm, RMSNormBackward and SdpaFwd bundles, routed into the architecture's census shard and gated on FlyDSL content. Each entry's generated TOML skips every other FlyDSL architecture, so a family package (gfx1100–gfx1103) runs each suite once. Category YAMLs `hipkernel_flydsl_*_test_categories_integration.yaml`; the two 4096-token fp16 causal bhsd SdpaFwd cases run from the comprehensive tier up. Support claims recorded for gfx1151. - rocKE test fixes for two Windows CI failures: the GPU-reference worker's temporary directory removal retries while Windows releases a killed worker's handles, and `test_lowers_and_compiles_without_scratch` gets its own 180 s limit (it ran ~54 s of 60 s). `flydsl/` holds the vendored kernel sources at pinned commits (FlyDSL, AITER, AOTriton; every local modification listed in each file's header and in the attention READMEs), the generators, `gen_descriptors.py`, `tools/check_shards.py`, the drift-guard tests, and `README.md`, `REGEN.md` and `COVERAGE.md`. ## Risks | Risk | Mitigation | |---|---| | TheRock ships FlyDSL on merge | TheRock's `HIPKERNELPROVIDER_ENABLE_FLYDSL` flag switches it off; this PR's TheRock CI built and tested FlyDSL, and a run pinned to TheRock main passed on gfx942, gfx950 and gfx110X | | Prebuilt objects drift from their sources | Per-object SHA256 and per-op source digest checked before every pack; drift-guard tests prove the checks refuse tampered trees | | An engine runs a graph it cannot compute | Every read field is gated, with one decline test per rule, mutation-checked | | One object set per generic family | Objects verified as `gfx11-generic` / `gfx12-generic`; every member shard verified after pack; gfx1150 and gfx1153 have no device run (same objects as gfx1151 and gfx1100) | | A device suite passes with no objects staged | Device cases fail rather than skip on an arch that ships their family; shard censuses fail on an empty shard | | Selection between overlapping engines | Static per-object priority today; autotune measures every applicable engine and object; UHD heuristics are a follow-up | **Known deviation:** RMSNorm backward and the two split-KV decode engines launch a second kernel compiled into the same object, named in KMD metadata (`reduce_symbol`, `merge_symbol`), instead of RFC 0017 §15.1 `sources` entries. RFC 0021 §3 names this form as an accepted alternative (`COVERAGE.md` §8). ## Testing - **Strategy:** host tests for every matcher, score and decline rule, checked against compiled mutants; GPU launches against a double-precision host reference; a per-object suite that launches every shipped object (every shape class in its `Full/` tier); shard censuses; `gen_descriptors --check` before pack and `check_shards` after. - **CI:** all required checks green at `fcb674c8766`, with FlyDSL device tests on Linux gfx94X and Windows gfx110X (FlyDSL ON by default then). - **Device runs on the shipped objects:** gfx950, gfx942, gfx90a, gfx1201, gfx1200 and gfx1100 ran the default and `Full/` tiers, launching every object of their families, with 0 failures; gfx1151 runs the default tier. With an empty descriptor root every device case of the arch's families fails. - **Local, this head (gfx1151):** 841 FlyDSL, container and ingestor gtests passed (62 skipped need other GPUs); census, FlyDSL and name-validation CTests pass; the FlyDSL integration entries pass the quick tier (rmsnorm 12, rmsnorm_bwd 5, sdpa_rdna 14, sdpa_rdna_decode 13 cases verified, 0 failed). - **After the next TheRock pin move:** a Multi-Arch run of this head with the TheRock pin moved to TheRock main plus #9615 and ROCm/TheRock#9619 (test branch, not merged) built FlyDSL and passed every hipkernelprovider, hipDNN and integration job on gfx950, gfx942 and Windows gfx110X. The FlyDSL integration entries verified 9–12 cases per SDPA engine and 12/5 per RMSNorm engine, 0 failed; only the runner's own architecture's entries ran. - **Regeneration:** every op on every target is byte-identical to the shipped objects; `diff_upstream.py` shows only recorded modifications. - **Tiers:** quick, standard and comprehensive run everything but `Full/`; the full tier adds every object on every shape class. ## Follow-ups - Support claims for the architectures beyond gfx1151 (harvest from CI once the TheRock pin includes #9615), and UHD heuristics for the FlyDSL engines. - Move the two-launch objects to `hsaco` multi-launch `sources` entries now that #14215 has landed. - RMSNorm: bias/dbias, strided rows, fusions (residual add, FP8 output, zero-centered gamma), baked widths ≤ 2048, backward above 16384. - SDPA: padding/varlen, exact head-dim 96/160/192/224 objects on gfx942/gfx950, a d512 prefill object, the deferred performance items in `COVERAGE.md` §7.5, and upstreaming the kernel modifications. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
AviralGoelAMD
added a commit
that referenced
this pull request
Nov 2, 2026
Brings in #13955 as merged (the stacked copy drops out), #14367, #14276, #14215, #14109, #14111, #13897, #14679 and the rest of develop. Resolutions: - Parity emitters: develop's #14367 added gfx942 tiled-3D configs 18 and 19; this branch's strided decode configs move from 18-27 to 20-29 in the .py and .c, and the C strided switch moves to idx >= 20. Golden re-recorded for the family only: 0-19 unchanged in every flavor, 20-29 added. - test_registry_regeneration: keep the per-engine parametrization and adopt develop's stable-id, byte-for-byte g2 and --expect-ids g3, with the shipped folder taken per engine instead of the gfx950 globals. - Representative golden: take develop's per-family shards (#14679). - CMake, IngestorPacks.hpp, READMEs: keep both sides; SUPPORT_MATRIX_OVERVIEW re-rendered from the merged claims.
BrianHarrisonAMD
added a commit
that referenced
this pull request
Nov 2, 2026
…g (ALMIOPEN-2922) (#13972) ## Summary Implements [RFC 0021](https://github.com/ROCm/rocm-libraries/blob/develop/projects/hipdnn/docs/rfcs/0021_KernelSlots.md) (kernel slots). An engine can run several kernels in a fixed order, and each position (a slot) picks its kernel independently. The first intended user is split SDPA backward (prep, dK/dV, dQ). JIRA ID : ALMIOPEN-2922 ## Risk Assessment Risk 3: a new feature path in the plugin SDK, behind `HIPDNN_ENABLE_KERNEL_INGESTOR`. That flag is OFF by default in hipDNN and ON in TheRock. Engines without slots are unchanged. Golden tests pin their knob ids, selector revision and winner records, and their packed bundles match `develop` byte for byte. Public API changes are additive, apart from the source changes listed in RFC 0021 section 10. ## ASIC Coverage The change is architecture-independent host code. It was tested on the host with the GPU hidden, and on a gfx90a device for the device-gated tests (gfx950 for an earlier head). A local build targeting gfx90a, gfx950 and gfx1151 runs every packed-load test. CI covers Linux and Windows superbuilds and TheRock gfx94X, gfx950, gfx110X and gfx125X builds; the Windows superbuild currently fails at configure on a `develop` issue (#14801). ## Testing Summary Results are for this head: the single feature commit, rebased onto `develop` f755808. Host suites (Linux, flag ON, GPU hidden): | Suite | Result | |---|---| | `hipdnn_plugin_sdk_tests` | 1637 run, 0 failed, 15 skipped (device-gated or locale) | | hipDNN ctest: SDK, validator, generator, bench and `uhd_gen` (required, not skipped) | 88/88 | | hip-kernel-provider ctest: gtest suites, `hkp_pack` pytest and packaging probes (provider-wide GPU integration tiers not run) | 67 run, 0 failed, 1 skipped (environment) | | `hkp_pack` and packaging-probe pytest, run directly | 386 and 182 passed | Device-gated tests on gfx90a: - Plugin SDK stall, benchmark and timer tests: 45/45. - `hip_kernel_provider_tests`: 1179 passed, 3 skipped (a device code-object capture, and the gfx950 and gfx1151 fixture loads, which the three-arch build runs: 10/10 packed-load tests). This includes `TestMultiLaunchEndToEnd`. - `hip_kernel_provider_integration_tests`: 35/35, including the `kernel_ingestor_engine` suites. Not tested: - Real slotted kernels; the fixtures are synthetic. - `hipdnn_bench --sweep` on a slotted engine. - The RFC section 11 separability check. Like the existing `uhd_gen` tests, the slot tests skip in CI because CI lacks pandas, pyarrow and lightgbm (#14623). They pass locally. ## Testing Checklist - [x] Host suites above - Status: Passed - [x] Device-gated SDK and HKP tests - Devices: gfx90a (gfx950 for an earlier head) - Status: Passed - [x] pre-commit - Status: Passed - [ ] hipDNN superbuild CI, Linux and Windows - Status: Linux running; Windows fails at configure on a `develop` FlyDSL census-shard error, not from this change (fix: #14801) - [ ] TheRock multi-arch CI - Status: Running on the new head ## Technical Changes - **Plugin SDK:** - UED, KDP and UHD 1.1 describe slots. - Each slot has its own catalog, ranker, `<slot>.<field>` knobs and winner records. - A slot program runs the picks over one shared workspace. `SlotProgramBenchmark` times whole programs, and `winners.jsonl` v3 records the results. - **Composition with existing features:** - Slots work with multi-launch kernels, pointer catalogs, staged applicability and the stall-restart benchmark policy. - A slot's kernels may be rocKE, hip or hsaco multi-launch kernels (#14215); `launches` and the launch-set checks apply per slot. The gfx950 hsaco fixture's `dq` slot is a two-launch hsaco `sources` kernel. - Regenerated slotted bundles keep their descriptor ids (#14109). - Slotted plans are not serialized (RFC 0021 section 11). - **RFCs:** RFC 0017 links RFC 0021 and carries the slot edits RFC 0021 lists. - **Tooling:** - `uhd_gen --slot` trains and promotes one slot's ranker. - IngestorGenerator, `hipdnn_validate_descriptors` and `hipdnn_bench` accept slotted engines. - **`hkp_pack`:** packs slotted engines and rejects invalid slot configurations it can detect statically. - **Shared rule corpus:** the `trained_against` slot-provenance rules are one JSON corpus run by the C++ parser, the packaging schema, `hkp_pack` and `uhd_gen`. - **Test packaging:** fixtures built for one arch (gfx950 hsaco, gfx1151 multi-launch) pack only when the build targets that arch. On other builds, packaging probes check that they pack. - **Skills:** the `hipdnn-ingestor-engine` skill names RFC 0017 by repository path, so `validate-skills.py` passes. - **Behavior changes visible on unslotted engines:** - Ingestor log lines now name the engine. - A cached decline logs at INFO instead of TRACE. Logging is off by default. ## Known Issues - PR CI does not build for gfx950 or gfx1151 on Linux, so the packed-load tests of the two single-arch fixtures do not run there. The gfx1151 test would run on the Windows superbuild, which currently fails at configure on a `develop` issue (#14801); both ran in a local build targeting gfx90a, gfx950 and gfx1151 for this head. Until CI covers these arches, changes to those fixtures need a build that targets them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-ups to #14158 (multi-launch
sourceskernel descriptors). Asourcesmap may now authorhipandhsacoentries as well asrocke, so a HIP or prebuilt-code-object multi-stage kernel can go through the packer. The packer refuses a pack in which kernels served by one dispatch handler name different launch sets. A device test runs a packer-built multi-launch kernel end to end through an engine, with a workspace hand-off between its launches. Saving an execution plan of a multi-launch kernel is refused for having several launches, rather than as anembedded_sourcekernel. The remaining review items from #14158 are closed with tests, tooling fixes and docs.JIRA ID : ALMIOPEN-2973
Risk Assessment
Risk 3 of 5. The change extends the packer (
hkp_pack) and IngestorGenerator with a new authoring path, adds a pack-time refusal, and adds three inline helpers to the plugin SDK ingestor headers. Single-launch descriptors and rockesourcespack and load as before; the state manager's launch-set and adaptability rules are unchanged and only move into shared helpers. The new packer refusal can fail a pack that used to succeed, but such a pack already had kernels dropped at load. Execution-plan saving refuses a multi-launch kernel with the same status as before and a corrected message, and the serialization behavior note is unchanged for every engine. Consumer-agreement evidence stays rocke-only, so hip/hsacosourceskernels are treated like their single-launch counterparts (a declaration withmetadata_fieldsover them is refused). The packer, the state manager and the validator also overlap with the open kernel-slots implementation (#13972), so whichever lands second will need a rebase.ASIC Coverage
Host-side packaging, loading, plan-save checks and tooling, plus a test-only fixture. No kernel, support surface, selection rule or default changes. Passing PR CI is sufficient. The device tests also ran and passed on gfx90a.
Testing Summary
End to end on device:
TestMultiLaunchEndToEndloads a UKD 1.1 hipsourceskernel packed byhkp_pack(two launches on one translation unit, one code object, two symbols), selects it through an engine with a test-only dispatch handler, and executes it. The first launch writes the workspace and the second reads it. The asserted output is reachable only if both launches run, in order, over one workspace. Host-only cases check that a handler declaring fewer launches drops the kernel and logs both launch sets.Plan saving: a plan over a multi-launch kernel whose launches are all kpack is refused for having several launches, and the message does not call it an
embedded_sourcekernel.Packer: hip and hsaco
sources(compile and read once per shared source, per-launch symbol and signature, provenance, desk check, refusals forrole, a missing hsaco arch, mixed kinds andmetadata_fields). Also the pack-time launch-set check, including two KDPs that share one UDD and two UDDs that name one dispatch symbol, and kills for previously surviving mutants: compile grouping by library and toc key, the desk-check reader per launch, and launch-order insensitivity.Probes: hip and hsaco launches checked by archive membership, digest, symbol and signature without rocke evidence; rocke evidence still required. Also per-launch sweep keys and group keys, and launch-order insensitivity.
IngestorGenerator: the generated multi-launch
launchNames()stub is now compiled, its census arm parsed, and its launch set checked against the config.launcheson hip and hsaco kernel sources is accepted.verify_variant_setskeys hip and hsaco launches by their own identity and ignoressourceskey order.Validator: a committed fixture whose kernels name different launch sets fails validation and names both sets.
Mixed-version check (manual): a validator built from develop just before feat(hipdnn): support multi-launch rocKE kernels in kernel descriptors #14158 skips UKD 1.1 kernels with a version warning instead of misloading them. A pack holding only 1.1 kernels is dropped; in a mixed pack its 1.0 kernel is kept.
Mutation checks: every new test was run against a targeted source mutant and failed. Examples: state manager keeping every kernel, plan passing no workspace, every launch given the first launch's symbol, signatures keyed per code object instead of per symbol, plan capture reading the kernel's
sourceinstead of its launches.Review follow-ups from feat(hipdnn): support multi-launch rocKE kernels in kernel descriptors #14158. The new tests cover:
target/roleoverrides being refused;Each was run against a targeted source mutant and failed.
Testing Checklist
ctest --test-dir build --output-on-failure --no-tests=error- Status: Passed (236 registered; the 3 skips are 2 sample tests and the rocKE arch-domain reproduction test, none of them changed here)hip_kernel_provider_tests --gtest_filter='TestMultiLaunch*:TestLaunchKernelDefinition*:TestIngestorPlanCapture*:TestRestoredIngestorPlan*:TestIngestorPlanRoundTrip*'- Devices: gfx90a - Status: Passed (26/26)ctest -R hipdnn_ingestor_generator_pytest(strict skips) - Status: Passed (1545 passed, 0 skipped)ctest -R 'hkp|probe|validate_descriptors'- Status: PassedTechnical Changes
hkp_pack: authoredsourcesentries may behip(source,entry,build) orhsaco(file,symbol), one kind per map. Entries sharing a translation unit and build, or an hsaco file, share one code object and toc key; each launch keeps its own symbol and signature. Provenance records each launch's authored identity underprovenance.sources.role, consumer agreement andcheck_claimsstay rocke-only;embedded_sourceand authoredkpackentries stay refused.dispatch_symbolof its KDP's UDD, which is how the loader resolves the handler) must name one launch set (a single-launch kernel's set is empty); otherwise the pack fails naming the symbol, each KDP and each kernel's set. KDPs that share a UDD, or name one symbol from two UDDs, are held to one set. The check sees one packaging root, so two roots that register the same symbol are not compared. The probe end-to-end fixture's fused multi-launch KDP gets a UDD of its own, since its single-launch siblings share the original one.Descriptors.hppholds the readers of a kernel's launches.launchNamesOf()builds a kernel's launch-name set, for the state manager andhipdnn_validate_descriptors. A multi-launch kernel leavessourcedefault, which reads as anembedded_sourcekernel, sosingleLaunchSource()returns a single-launch kernel's one source and null for a multi-launch kernel, andallLaunchSources()tests every launch. The state manager's adaptability check,buildForSourceKind, plan capture and the serialization behavior note read through them; the engine-selector digest still hashes both fields as before.source. Saving these plans is tracked in ALMIOPEN-3031.Kind.launch_evidencedecides whether a kind's launches carry rocke effective-spec evidence; hip and hsaco launches get the structural per-launch checks.launcheson hip and hsaco kernel sources, with per-kind override keys;verify_variant_setsidentities for hip and hsaco launches.multi_launchdescriptor set, packed beside the unit and integration roots so no Container loads it; validator fixturemulti_launch_mismatch; the_restamptest helper moved topack_helpers.py.sourceskinds stated; RFC 0009, the provider README and the plugin development guide state that a multi-launch kernel's plan cannot be saved yet; a worked multi-launch handler in the ingestor-enginenative-pack.mdguide; packaging, generator and fixture READMEs updated.sourceskinds share one refusal message.target/roleoverrides are refused.launchKernelDefinitionno longer copies thesourcesmap only to discard it.launches[].Related: #14158 (multi-launch kernel descriptors), #13315 (saving and loading ingestor execution plans), #13839 (RFC 0021, kernel slots), #13972 (kernel-slots implementation), #13897 (RDNA SDPA catalog, a future
sourcesuser).