[BrianHarrisonAMD]

Update All review threads are now addressed. Finalizing, and merging shortly. TY for the feedback!

Summary

RFC proposing a data-driven kernel ingestor for the hipDNN hip-kernel-provider. Today, adding a kernel or fused graph means hand-writing C++ for each one: a plan builder that pattern-matches the graph, an engine and its registration, bespoke launch code, and a selection heuristic, i.e. a code change and recompile per kernel. As kernels multiply per architecture, data type, problem shape, and fusion, this creates four compounding problems: scale, staleness (hand-ported copies fall behind upstream authors), maintainability sprawl, and slow feature velocity.

This RFC replaces that per-kernel C++ with a small family of declarative descriptors that one generic provider loads and runs. The unit an author ships is a KDP (Kernel Descriptor Pack): one file that binds a shared matcher set, one engine, and one dispatch descriptor over a vector of child kernels, so a family of kernels is a few shared descriptors plus one small kernel entry each, not hundreds of near-duplicate files. Every descriptor is referenced by ID:

• UKD (Universal Kernel Descriptor): one launchable kernel, its source details plus concrete metadata values. It carries no logic of its own and inherits its matchers, engine, dispatch, heuristic, and metadata schema from its pack and that pack's engine.
• UMD (Universal Match Descriptor): when a kernel applies, a structural graph pattern plus a declarative criteria expression, and it binds the variables the launch uses. Shared across packs by ID.
• UDD (Universal Dispatch Descriptor): the dispatch ABI (argument binding and ordering, grid, block, shared memory, workspace). It holds one or more Launches; one UDD per pack, shared by every child kernel.
• UED (Universal Engine Descriptor): a stable engine identity, knobs, and behavior/numerical notes. It names the engine's one heuristic (UHD) and one metadata schema (KMD); many KDPs may share one engine.
• UHD (Universal Heuristic Descriptor): the kernel-selection model that ranks the kernels within one engine and picks one.
• KMD (Kernel Metadata Descriptor): the engine's variant fields, each with a type and optional default; it is the feature space the UHD ranks over. Each UKD fills in concrete values, and matchers read them as $kernel.<field>.

A Launch is one dispatch step inside a UDD (grid, block, shared memory, argument signature) with a named source slot, paired at runtime with the UKD source that fills it: a simple kernel is a one-Launch UDD, and a multi-launch kernel such as SDPA backward is a several-Launch UDD run in order.

It generalizes the existing rocKE SDPA launch prototype (PR #9207) into data and covers: graph-criteria matching (UMD) with a declare/bind/use symbol lifecycle; a symbolic launch/workspace expression language and declarative argument binding (UDD); kernel sources plus an adapter model for extending supported source and heuristic formats; observability and diagnostics; AOT packaging and opt-in runtime drop-in; a worked SDPA example; and a phased rollout that starts by publishing a per-format follow-up RFC before implementation. Multi-kernel launch and composition (UCD, Universal Composite Descriptor) are framed as fast-follows, and JIT as a future follow-on that reuses the same descriptors.

Documentation only; no code changes.

JIRA ID: AICK-1668

Risk Assessment

Risk 1 (docs only). No product code, schemas, plugin ABI, or build wiring changes; the blast radius is limited to files under projects/hipdnn/docs/. The RFC proposes an approach and is explicitly requesting feedback before any implementation is scoped, so nothing here alters runtime behavior.

Testing Summary

• Rendered all 8 SVG diagrams and both Markdown documents; verified that in-document and cross-document links and anchors resolve (0 broken across both files), each table of contents matches its section headings, code fences are balanced (20 in the RFC, 5 in the companion), and there are no em-dashes.
• Verified the worked-example split is content-preserving: every substantive line of the old Section 13 is accounted for in the RFC summary or the companion, and the 8 SVGs are unaffected (none were in Section 13).
• Verified example consistency: every descriptor GUID referenced in the examples resolves to a definition, and each engine's KMD fields, UHD features_signature, and UKD metadata agree.
• File hygiene verified on all changed files (no trailing whitespace, CRLF, or tabs; final newlines present).
• pre-commit run on both changed files: all applicable hooks passed (trailing whitespace, end-of-file, large files).

Testing Checklist

• Doc validation (links/anchors, TOC, render, whitespace/EOL, pre-commit) - Status: Passed
• PR CI - GitHub PR checks (ci:docs-only) - Status: Pending

Technical Changes

• Add projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md: the RFC. It defines the descriptor family (UKD kernel, KDP pack, UMD match, UDD dispatch, UED engine, UHD heuristic, KMD metadata), where a KDP binds a matcher set, one engine, and one dispatch over a vector of child UKDs and each engine owns its one heuristic and one metadata schema. Covers matching with a declare/bind/use lifecycle, the symbolic launch/workspace expression language and declarative argument binding, the Launch concept, kernel-source and heuristic adapters/extensibility, observability and diagnostics, AOT packaging and opt-in runtime drop-in, a worked SDPA example, phased delivery (starting with per-format follow-up RFCs), and multi-kernel launch plus composition (UCD) as future work.
• Add projects/hipdnn/docs/rfcs/examples/0017_UniversalKernelDescriptor_WorkedExample.md: the long-form SDPA worked example, split out of the RFC per review. Follows the existing 0012_CuDNN_Shim / 0012_CuDNN_Shim_Reference companion-document precedent. Section 13 of the RFC remains as a summary linking to it, so no sections are renumbered and all existing anchors still resolve. The split also let two prior elisions be filled in: the mask-mode classifier is now spliced into the matcher in full, and both kernel packs carry complete ids.
• Add 8 diagrams under projects/hipdnn/docs/images/ukd_*.svg: before/after, concepts, criteria match, dispatch flow, ingestion flows, adapters, composition, and JIT seam.

[therock-pr-bot]

✅ All Checks Passed — Ready for Review

Check	Status	Details
🌿 Branch Name	✅ Pass	—
📝 PR Title/Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	PR does not contain code files — Unit Test auto-passed
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

[therock-pr-bot]

🎉 All checks passed! This PR is ready for review.

[mousdahl-amd]

High-Level Review

Highest-signal non-blocking items:

• Framing gap — the cost of the status quo is asserted, not quantified; the whole complexity budget rests on kernel-authoring volume that isn't given, and the nearest
  alternative (a build-time codegen path) isn't weighed.
• Observability is unaddressed — "drop in a file and it works" makes "why wasn't my kernel selected/loaded?" the dominant new failure mode, with no diagnostics surface named.
• No fuzz/test strategy for the descriptor parsers and the safe interpreter, which on the drop-in path parse untrusted input.
• Composition (§9) carries ~half the design weight but open question #3 leaves its v1 inclusion undecided — candidate to split into its own follow-up.

Security Review

Threat model (inferred, RFC states none explicitly): in-process GPU provider; the realistic attacker is a malicious/tampered runtime drop-in package — the drop-in path loads
author code objects and JIT-compiles author HIP source, all running as the host process. Secondary: hostile serialized graph/descriptor hitting the matcher + "safe
interpreter."

The load-bearing decision is made correctly — no inline code in descriptors (symbol names + typed args only), drop-in opt-in/off-by-default/built-in-predicates-only. Top
non-blocking gaps:

• Drop-in folder is a code-exec boundary defined by filesystem perms hipDNN doesn't own — needs explicit path resolution, refuse world/group-writable dirs, reject symlinks.
• Signing is deferred but is the actual enforcement — sequence it as a prerequisite before drop-in ships enabled.
• Three new hostile-input parsers (binary descriptor loader, matcher, expression interpreter) — no stated recursion/step/size bounds → stack-exhaustion + truncated-input
  surface.
• Shape/workspace integer-overflow — grid/workspace arithmetic over user-controlled dims → under-alloc + OOB device write; needs checked-width fail-closed.
• Installed AOT bundles are trusted by filesystem location alone — integrity verification, not just compat gating.

[amd-bartgips]

Building on Brian's note in the HIP-macros thread — "the JIT specification will be slightly different from the current UKD's, since you could have 1 definition generate many possible variants that need to be picked between"

I have been playing around with this in ruka and have been implementing the JIT kernel path: porting Triton and FlyDSL kernels onto the KDP format, analogous to the §9.3 path.
I see in the RFC that the JIT variant-space question is explicitly deferred to a follow-up RFC, and therefore my comments are a bit too early.
However, our team mainly develops Triton and FlyDSL kernels, which are all JIT and tend to be quite configurable with multiple (JIT-)compile-time parameters, so I almost immediately ran into this.
Some of it is perhaps obvious, but I thought it wouldn't hurt to already write some stuff down here.

With some thanks to Claude for summarizing it for me:

What we found: with the current schema, a JIT kernel with a curated tuning grid of N configurations requires N separate UKD entries. For a real Triton kernel (MLA decode, with 14 hand-selected grid entries across two head-dim fingerprints), that is roughly 210 lines of JSON for what the kernel author currently expresses in ~20 lines of Python. For larger GEMM or convolution tile sweeps the ratio compounds further.

The underlying asymmetry is that for AOT, one UKD = one independently authored binary, so enumeration is correct and natural. For JIT, one UKD = one compile call: there is one source and N config points, not N independently authored artifacts.

Three directions the JIT RFC might consider — not as requests for this RFC, just as early thoughts from a prototype:

• enum annotations on KMD fields — for when a full Cartesian product is the right search space. A field declared {"name": "block_n", "type": "int", "enum": [4, 8, 16, 32]} expands a template UKD into one concrete UKD per combination of all enumerated fields. This is the right tool when the author genuinely wants to sweep all combinations, e.g. in an initial benchmarking phase before the grid has been curated.

• A grid_variants list on UKD — for when the author has already established that only a specific subspace of parameter combinations is useful. A single UKD carries the shared metadata fields (dtype, head dim, arch) plus a grid_variants list, one dict per tuning point containing only the fields that vary (tile sizes, warp counts, pipeline stages). The loader merges shared + per-variant metadata to produce one concrete UKD per entry. This directly mirrors how JIT kernel authors work in practice: they maintain a curated list of measured config points that is explicitly not a Cartesian product — a GEMM author knows that certain (block_n, num_warps) pairings are not useful combinations and would never include them. The schema change is additive, the expansion is mechanical, and the file stays compact regardless of grid size.

  One follow-on question this raises (which also holds for the enum option): each expanded UKD needs a stable unique identity for the heuristic, for pinning, and eventually for plan serialisation (the §16 id-mutability risk). The JIT RFC should specify whether variant ids are author-supplied per entry or loader-derived (e.g. by hashing the merged metadata tuple).

• Tooling-side expansion. It's also fine if the answer is simply "the schema stays point-per-UKD, and the authoring skills from §11 take care of generating the boilerplate from a compact spec." Although I imagine this was written more with rocKE in mind, and is less easy to generalise/backport to our flyDSL/triton kernels. But this doesn't seem to be the general idea, judging by Brian's comment above.

Otherwise great work so far! I really like this whole idea to standardize the way to describe every kernel's capabilities etc. Looking forward to the actual implementation 👍

[SamuelReeder]

LGTM! Let me know if further fundamental changes are made and I'd be happy to review again.

[BrianHarrisonAMD]

Building on Brian's note in the HIP-macros thread — "the JIT specification will be slightly different from the current UKD's, since you could have 1 definition generate many possible variants that need to be picked between"

I have been playing around with this in ruka and have been implementing the JIT kernel path: porting Triton and FlyDSL kernels onto the KDP format, analogous to the §9.3 path. I see in the RFC that the JIT variant-space question is explicitly deferred to a follow-up RFC, and therefore my comments are a bit too early. However, our team mainly develops Triton and FlyDSL kernels, which are all JIT and tend to be quite configurable with multiple (JIT-)compile-time parameters, so I almost immediately ran into this. Some of it is perhaps obvious, but I thought it wouldn't hurt to already write some stuff down here.

With some thanks to Claude for summarizing it for me:

What we found: with the current schema, a JIT kernel with a curated tuning grid of N configurations requires N separate UKD entries. For a real Triton kernel (MLA decode, with 14 hand-selected grid entries across two head-dim fingerprints), that is roughly 210 lines of JSON for what the kernel author currently expresses in ~20 lines of Python. For larger GEMM or convolution tile sweeps the ratio compounds further.
The underlying asymmetry is that for AOT, one UKD = one independently authored binary, so enumeration is correct and natural. For JIT, one UKD = one compile call: there is one source and N config points, not N independently authored artifacts.

Three directions the JIT RFC might consider — not as requests for this RFC, just as early thoughts from a prototype:

• enum annotations on KMD fields — for when a full Cartesian product is the right search space. A field declared {"name": "block_n", "type": "int", "enum": [4, 8, 16, 32]} expands a template UKD into one concrete UKD per combination of all enumerated fields. This is the right tool when the author genuinely wants to sweep all combinations, e.g. in an initial benchmarking phase before the grid has been curated.
• A grid_variants list on UKD — for when the author has already established that only a specific subspace of parameter combinations is useful. A single UKD carries the shared metadata fields (dtype, head dim, arch) plus a grid_variants list, one dict per tuning point containing only the fields that vary (tile sizes, warp counts, pipeline stages). The loader merges shared + per-variant metadata to produce one concrete UKD per entry. This directly mirrors how JIT kernel authors work in practice: they maintain a curated list of measured config points that is explicitly not a Cartesian product — a GEMM author knows that certain (block_n, num_warps) pairings are not useful combinations and would never include them. The schema change is additive, the expansion is mechanical, and the file stays compact regardless of grid size.
  One follow-on question this raises (which also holds for the enum option): each expanded UKD needs a stable unique identity for the heuristic, for pinning, and eventually for plan serialisation (the §16 id-mutability risk). The JIT RFC should specify whether variant ids are author-supplied per entry or loader-derived (e.g. by hashing the merged metadata tuple).
• Tooling-side expansion. It's also fine if the answer is simply "the schema stays point-per-UKD, and the authoring skills from §11 take care of generating the boilerplate from a compact spec." Although I imagine this was written more with rocKE in mind, and is less easy to generalise/backport to our flyDSL/triton kernels. But this doesn't seem to be the general idea, judging by Brian's comment above.

Otherwise great work so far! I really like this whole idea to standardize the way to describe every kernel's capabilities etc. Looking forward to the actual implementation 👍

Thanks for the review, and feedback @amd-bartgips!
I agree with this 100%.

For JIT, we need capabilities that allow us to express all the different ways variants can be made for populating the catalog that the heuristic ranks. 1 UKD -> Many entries is what I am expecting once we extend to support JIT. As mentioned Cartesian product is something we certainly need, and we can likely take inspiration from some of the work already being done for integration testing in hipDNN that allows us to use a template, and generate many variants. I also like the idea of making the UKD define the template, and then have an explicit list of variants you want to support alongside it.

Decently likely we will also need to expand heuristics to support two-stages. Once JIT exists, the catalog of possible entries explodes, and it becomes unlikely you can run a full heuristic estimation to rank each entry. Instead we would likley need a quick running heuristic that can pick the candidate list, and then run the ranking heuristic to determine the final winner.

---

All that being said, these are just early thoughts on how this can work as we extend to support JIT, and future RFCs will be made that can make this more concrete.