From 491da41b426d71a62ee326f63902d548c7fd4932 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Thu, 15 Oct 2026 23:45:04 +0000 Subject: [PATCH 1/6] docs(hipdnn): RFC 0021 kernel slots, independently selected kernels per engine Adds RFC 0021 and the RFC 0017 edits it needs. An engine declares ordered slots, each with its own KMD, knobs, ranker and catalog. The engine's kernel choice is the per-slot picks run as one program that shares a workspace region. Unslotted engines are unchanged. --- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 267 ++++++++++++++++++ 1 file changed, 267 insertions(+) create mode 100644 projects/hipdnn/docs/rfcs/0021_KernelSlots.md diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md new file mode 100644 index 000000000000..d43acce57725 --- /dev/null +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -0,0 +1,267 @@ +# RFC 0021: Kernel Slots: Several Independently Selected Kernels per Engine + +- Contributors: Brian Harrison +- **Status**: Draft +- **Implementation Version**: TBD +- **Follows**: [RFC 0017](0017_UniversalKernelDescriptor.md) § 14.2, the multi-kernel half of the + "Composition" follow-up. Graph decomposition across engines (the composite engine, or UCD, RFC + 0017 § 15.2) is out of scope. +- **Builds on**: [RFC 0019](0019_UniversalHeuristicDescriptor.md) and + [RFC 0020](0020_UniversalEngineDescriptor.md). + +## Table of Contents + +1. [Summary](#1-summary) +2. [Motivation](#2-motivation) +3. [The Slot Model](#3-the-slot-model) +4. [Descriptor Format](#4-descriptor-format) +5. [Selection and Execution](#5-selection-and-execution) +6. [Shared Workspace and Arguments](#6-shared-workspace-and-arguments) +7. [Measurement and Heuristics](#7-measurement-and-heuristics) +8. [Validation](#8-validation) +9. [Compatibility](#9-compatibility) +10. [Amendments to Other RFCs](#10-amendments-to-other-rfcs) +11. [Deferred Work and Open Questions](#11-deferred-work-and-open-questions) + +Terms follow [RFC 0017 § 2](0017_UniversalKernelDescriptor.md#2-the-descriptors): a **UED** is an +engine, a **KDP** (pack) binds matchers and a dispatch to a set of kernels, a **UKD** is one +kernel, and a **KMD** is the metadata schema its kernels fill in. An engine's **catalog** is the +kernels applicable to a graph; its **ranker** orders the catalog; a **winner record** stores the +order measurement found. + +--- + +## 1. Summary + +An engine may run several kernels in a fixed order to serve one graph. Each position in that order +is a **slot**, and each slot chooses its kernel independently: it has its own KMD, catalog, ranker, +knobs and winner records. The slots share the engine's identity, its graph match, and a workspace +region for the data they hand each other. + +Slots sit between the engine and its packs: **UED → slots → KDPs → UKDs**. Everything below a slot +is unchanged; a UKD is still one kernel. An engine that declares no slots has one implicit slot and +behaves as it does today. To a caller, a slotted engine looks like any other: a configuration is an +engine plus knob settings. + +Independent slots let each stage be tuned on its own: for split SDPA backward, 253 per-kernel +configurations instead of 613 fixed pairs (§ 2). The cost is that a slot has no time of its own, so +measurement times whole programs, varying one slot at a time (§ 7). This RFC proposes the slot model +and a 1.1 revision of the UED and KDP formats; split SDPA backward is the first user. + +## 2. Motivation + +Split SDPA backward runs three kernels on one stream: + +| Slot | Computes | Selected by | +|---|---|---| +| `prep` | Delta (`rowsum(dO ∘ O)`, a per-row term of the softmax gradient) and a sanitized copy of the forward log-sum-exp | the graph alone | +| `dkdv` | dK and dV | its own ranker | +| `dq` | dQ | its own ranker | + +The best `dkdv` and `dq` kernels are only weakly coupled. AOTriton keeps separate gfx950 tuning +databases for dK/dV and dQ +([`bwd_kernel_dk_dv`, `bwd_kernel_dq`](https://github.com/ROCm/aotriton/tree/f65658a69f/modules/flash/database/amd/gfx950)). +Covering 80% of their tuned problem shapes takes 253 distinct configurations chosen per kernel, +against 613 distinct (dK/dV, dQ) pairs. + +RFC 0017 offers two shapes for multi-kernel work, and neither fits: + +- **Multi-launch pack (§ 15.1).** One UKD holds every launch and is selected as a unit. Each + (dK/dV, dQ) pair must be authored as its own UKD, with `prep` duplicated into each. +- **Composite engine (UCD, § 15.2).** Each stage is a separate engine answering a child graph. No + graph node means "dQ only", and Delta is not a graph tensor. + +## 3. The Slot Model + +``` +UED (engine) identity, graph match, shared workspace size, engine estimate + ├── slot "prep" KMD + │ └── KDP(s) matchers, dispatch, UKDs + ├── slot "dkdv" KMD, ranker, knobs + │ └── KDP(s) + └── slot "dq" KMD, ranker, knobs + └── KDP(s) +``` + +A slot holds everything an engine uses to select a kernel. The engine keeps what identifies it and +what reads the graph. Four rules define the model: + +1. **Order is the contract.** Slots launch in declaration order. A slot may read anything an + earlier slot wrote. +2. **Every slot is required.** The engine accepts a graph only if every slot has an applicable + kernel. Optional slots are deferred (§ 11). +3. **Slots are selected independently.** Nothing in one slot constrains the choice in another. +4. **Hand-off conventions belong to the engine.** The format and layout of data passed between + slots, and any convention two slots must agree on (such as the dropout random-number layout), are + fixed for the engine and met by every kernel in every slot. Every KMD field that changes a + kernel's result is pinned to the graph by a matcher. A variant that needs a different convention + is a different engine. This rule is what makes rule 3 safe: every combination of per-slot picks + computes the same result. + +Kernels whose choices depend on each other belong in **one** slot, launched together by that slot's +dispatch handler. Examples are a backward kernel with an optional output-conversion kernel, or a +split-KV kernel with the reduction that consumes its splits. + +## 4. Descriptor Format + +The UED gains two members, and each slot carries the selection members the engine carried before: + +| Member | Meaning | +|---|---| +| `slots` | Ordered list of slots. When present, the engine-level `metadata`, `sort_kernel_catalog`, `knobs` and `predict_applicable_kernels` are absent. | +| `slots[].name` | A lowercase identifier (`^[a-z][a-z0-9_]*$`), unique within the engine. `global` is reserved, because [RFC 0004](0004_EngineConfigKnobs.md) owns `global.` knobs. | +| `slots[].metadata` | The slot's KMD. | +| `slots[].sort_kernel_catalog` | The slot's ranker (optional, metric-less; § 7). | +| `slots[].knobs` | KMD fields the slot exposes as knobs (optional). | +| `shared_workspace` | The bound token holding the shared region's size (optional; § 6). | + +A KDP gains `slot`, naming the slot its kernels serve. It is required for a slotted engine and +forbidden otherwise. A slot may have several packs. A UKD is unchanged; it fills the KMD of its +pack's slot. A kernel serves at most one slot of an engine, so logs and records can name it without +its slot. + +```jsonc +// UED +{ + "version": "1.1", + "id": "5d2f…", + "name": "flydsl:sdpa_bwd_cdna", + "graph_match": {"native": "flydsl.sdpa_bwd_cdna.graph_match"}, + "shared_workspace": "ws.bytes", + "predict_engine": {"gfx950": ["7b1e…"]}, + "slots": [ + {"name": "prep", "metadata": "a10c…"}, + {"name": "dkdv", "metadata": "b27e…", "sort_kernel_catalog": {"gfx950": "c3f1…"}, + "knobs": ["block_m", "block_n", "num_warps"]}, + {"name": "dq", "metadata": "d48a…", "sort_kernel_catalog": {"gfx950": "e59b…"}, + "knobs": ["block_m", "block_n", "num_warps"]} + ] +} + +// KDP: names the slot its kernels serve +{"version": "1.1", "engine": "5d2f…", "slot": "dkdv", "arch": ["gfx950"] /* matchers, dispatch, kernels */} +``` + +## 5. Selection and Execution + +**Matching.** The engine's graph match runs once for all slots. Each slot's packs are then matched +as an engine's packs are today, producing one catalog per slot (rule 2 applies). + +**Knobs.** A knob `f` on slot `s` is advertised as `s.f`, with legal values and a default computed +over that slot's catalog. Setting `s.f` filters slot `s` only; a setting that leaves the slot empty +is rejected, naming the knob. An unslotted engine keeps bare knob ids. + +**Candidate enumeration** lists every combination of per-slot kernels, so its size is the product of +the slot catalog sizes. Entries are computed on demand in a fixed order, independent of rankers and +records, so pages stay stable. The enumeration schema is unchanged, and a candidate's knob settings +cover every slot. + +**Plan build.** Each slot resolves its kernel as an engine does today: order the catalog (winner +record, else ranker, else static order), apply the slot's knob filter, and take the first kernel +that builds. A kernel that fails to build costs only its slot's pick; a slot with no buildable +kernel fails the plan. The plan runs the picks in slot order on the plan stream. + +**Engine selection is unchanged.** Default engine order stays static. A fused single-kernel +alternative is a separate single-slot engine, and engine selection chooses between the two as it +does between any engines. + +## 6. Shared Workspace and Arguments + +**Shared region.** Slots pass data through a region at the start of the plan's workspace. Its layout +depends only on the graph, so the engine's native graph match (provider code, not a declarative +pattern) computes it and publishes it as bound tokens, named values written `$ws.*`. The UED's +`shared_workspace` names the token holding the total size. For split SDPA backward, the region +holds Delta and the sanitized log-sum-exp, sized once for the maximum sequence length. + +**Workspace size** is the shared region plus the largest scratch any one slot needs. Slots run one +at a time, so they reuse the same scratch. Each slot's scratch starts after the shared region; in +v1 every slot dispatches through a native handler, which places it there. Workspace limits and +checks apply to the total. + +**Shared arguments.** Every runtime argument more than one slot uses (scale, dropout parameters and +random-number seed, ragged sequence information, maximum sequence lengths) is computed once in the +graph match. Slots read it rather than recompute it, so they cannot disagree on, for example, the +dropout mask. + +## 7. Measurement and Heuristics + +**Measurement times whole programs, varying one slot at a time,** because slots share one stream and +have no time of their own. With benchmarking enabled, the plan times: + +1. The **reference program**: each slot at its first kernel in plan-build order (§ 5), its + **reference pick**. If a reference pick fails at launch, that slot moves to its next kernel. +2. Each other kernel in each slot, with the remaining slots at their reference picks. + +This takes 1 + Σₛ(Kₛ − 1) runs instead of ∏ₛ Kₛ. The plan then times the **winners program**, each +slot at its measured winner, and serves it. If the winners program fails, the plan serves the +reference program. A slot whose winner record already covers its catalog is not re-measured. + +**Winner records are per slot**, ranking that slot's kernels by the whole-program times above. A +separate record stores the winners program and its time, which is the engine's measured time. + +**Heuristics.** + +- A slot's ranker orders only its own slot. It predicts no time, since a slot has none. +- The engine's estimate comes only from its engine-level model (`predict_engine`), which models the + whole program. +- A configuration-level prediction comes only from measurement. It exists once every slot has a + winner record and the winners program has been timed, and only for knob settings that leave that + program unchanged. + +**Training data.** Rankers are trained on benchmark data. In v1 the benchmark tool times every +enumerated candidate, which for a slotted engine is the full product of slot catalogs. Slot rankers +train on the rows where the other slots hold their reference picks. + +## 8. Validation + +At load, the runtime applies RFC 0020 § 13 and rejects a slotted engine, or a pack, that breaks a +rule in § 4: engine-level selection members beside `slots`, an empty or misnamed slot list, a slot +ranker that declares a metric, `shared_workspace` without a native graph match, a pack with a +missing or unknown `slot`, or one kernel in two slots. The existing per-engine checks (RFC 0020 +§ 13.2, RFC 0019 § 6.3, RFC 0018 § A.5) run per slot. A slot with no loadable pack is a warning: +the engine stays registered and declines every graph. + +## 9. Compatibility + +UED and KDP take a minor bump to 1.1, adding `slots`, `shared_workspace` and `slot`. A 1.0 runtime +rejects 1.1 files; a 1.1 runtime reads 1.0 files unchanged. A slot ranker's provenance names its +slot, and a slotted engine's `predict_engine` model names every slot's KMD; both are minor UHD +format changes (RFC 0019 § 8.1). + +An engine without slots keeps its knob ids, trained models and winner records, so nothing trained or +stored is invalidated. Adding, removing, renaming or reordering slots is a breaking engine revision, +because knob ids and record keys change. + +## 10. Amendments to Other RFCs + +For an unslotted engine, every statement in RFCs 0017-0020 reads as written. For a slotted engine, +statements about an engine's kernel selection apply per slot. The edit list: + +| RFC | Change | +|---|---| +| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 10). | +| 0018 | Kernel-scoped matchers check against the KMD of the pack's slot (§ 8, § A.5). | +| 0019 | KMD, ranker, catalog, winner records, caches and contract checks are per slot; slot rankers are metric-less and train on whole-program times; a slotted engine has no model-based configuration estimate; its benchmark cost is the product of slot catalogs (§ 3, § 4.1, § 5, § 6.3, § 8.1, § 9.2, § 11, § 13.2). | +| 0019.13 | Slot rankers train per slot (§ 3.5, § 7.3, § 8.3). | +| 0020 | UED gains `slots` and `shared_workspace`; `metadata` is required only without `slots` (§ 4.2, § 10). Selection members and knobs move to slots, with knob ids `.` (§ 4.2, § 4.6, § 8). A native graph match may publish `$ws.*` tokens (§ 6.1). Load checks run per slot (§ 13). | + +## 11. Deferred Work and Open Questions + +| Deferred | Path forward | +|---|---| +| Concurrent slots | A `phase` per slot; slots in the same phase run concurrently. Needs helper streams or graph capture. | +| Optional slots | A `required` flag, once a consumer needs one. A slot whose need depends on another slot's pick stays inside that slot. | +| Cross-slot constraints | Constraint fields on the KMD; selection becomes a constrained search. | +| Per-slot timing | Per-kernel timing inside the program would allow calibrated slot rankers and model-based configuration estimates. | +| Cheaper training collection | Collect from benchmarking mode instead of the full product, once winner records carry the benchmark tool's per-run statistics. | +| `predict_applicable_kernels` per slot | `slots[].predict_applicable_kernels`. | +| Shared-region tokens from declarative graph matches | A reserved `ws` token root in RFC 0020 § 6.1. | + +**Open questions.** + +1. **Separability.** Measurement assumes one slot's pick does not change another slot's time; + shared cache state could break this. Comparing the winners program's time with the reference + time detects a large effect. The first implementation (split SDPA backward) should also time the + top 2×2 slot pairs end to end; if pairs reorder, measurement needs a pairwise confirmation step. +2. **Shared kernels across engines.** A standalone `prep` UKD can serve several engines, but each + pack loads its own copy. Is one load per pack acceptable? From c20f8cf9c80d85971b928b3349e67b80fcf43d4a Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Sat, 17 Oct 2026 18:39:26 +0000 Subject: [PATCH 2/6] docs(hipdnn): RFC 0021 workspace contract, program records and sweep rules Specifies the shared-workspace token checks and offsets, how the workspace limit applies to a kernel and to the engine, when a program record is valid and how slot and program records are appended, and the reference-pick rules for benchmarking and sweeps. --- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 79 +++++++++++++------ 1 file changed, 54 insertions(+), 25 deletions(-) diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md index d43acce57725..a5aaa96e44e6 100644 --- a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -89,7 +89,8 @@ what reads the graph. Four rules define the model: 1. **Order is the contract.** Slots launch in declaration order. A slot may read anything an earlier slot wrote. 2. **Every slot is required.** The engine accepts a graph only if every slot has an applicable - kernel. Optional slots are deferred (§ 11). + kernel, the shared-workspace token is valid, and the shared region fits the workspace limit (§ 6). + Optional slots are deferred (§ 11). 3. **Slots are selected independently.** Nothing in one slot constrains the choice in another. 4. **Hand-off conventions belong to the engine.** The format and layout of data passed between slots, and any convention two slots must agree on (such as the dropout random-number layout), are @@ -154,7 +155,8 @@ is rejected, naming the knob. An unslotted engine keeps bare knob ids. **Candidate enumeration** lists every combination of per-slot kernels, so its size is the product of the slot catalog sizes. Entries are computed on demand in a fixed order, independent of rankers and records, so pages stay stable. The enumeration schema is unchanged, and a candidate's knob settings -cover every slot. +cover every slot. A candidate is named by its per-slot kernel ids joined in slot order +(`prep=|dkdv=|dq=`). **Plan build.** Each slot resolves its kernel as an engine does today: order the catalog (winner record, else ranker, else static order), apply the slot's knob filter, and take the first kernel @@ -169,14 +171,23 @@ does between any engines. **Shared region.** Slots pass data through a region at the start of the plan's workspace. Its layout depends only on the graph, so the engine's native graph match (provider code, not a declarative -pattern) computes it and publishes it as bound tokens, named values written `$ws.*`. The UED's -`shared_workspace` names the token holding the total size. For split SDPA backward, the region +pattern) computes it and publishes it as bound tokens, named values written `ws.*`. The UED's +`shared_workspace` names the token holding the total size. The match also publishes +`ws.scratch_offset`, the shared size rounded up to 256 B, and an offset token per shared object +(for example `ws.delta.offset`). The runtime checks the token at catalog build: a missing, non-integer +or negative value makes the engine decline the graph with an error naming the token. For a slotted +engine the `ws.` root is reserved: its tokens are excluded from ranker features, problem features and +training rows. For split SDPA backward, the region holds Delta and the sanitized log-sum-exp, sized once for the maximum sequence length. -**Workspace size** is the shared region plus the largest scratch any one slot needs. Slots run one -at a time, so they reuse the same scratch. Each slot's scratch starts after the shared region; in -v1 every slot dispatches through a native handler, which places it there. Workspace limits and -checks apply to the total. +**Workspace size** is the aligned shared region plus the largest scratch any one slot needs. Slots run +one at a time, so they reuse the same scratch. A slotted kernel's workspace query reports its scratch +only. Every handler receives the unshifted workspace base; in v1 each slot dispatches through a native +handler, which reads the offset tokens in `prepare` and adds them to the base at launch. The plan +rejects a null workspace when shared plus scratch is non-zero. The workspace limit applies to the +total: a kernel is pruned from its slot when the aligned shared region plus its scratch exceeds the +limit, and the engine declines when the shared region alone exceeds it. The shared region does not +depend on any pick, so slots stay independent. **Shared arguments.** Every runtime argument more than one slot uses (scale, dropout parameters and random-number seed, ragged sequence information, maximum sequence lengths) is computed once in the @@ -189,28 +200,38 @@ dropout mask. have no time of their own. With benchmarking enabled, the plan times: 1. The **reference program**: each slot at its first kernel in plan-build order (§ 5), its - **reference pick**. If a reference pick fails at launch, that slot moves to its next kernel. + **reference pick**. If a reference pick fails at launch while benchmarking, that slot moves to + its next kernel. 2. Each other kernel in each slot, with the remaining slots at their reference picks. This takes 1 + Σₛ(Kₛ − 1) runs instead of ∏ₛ Kₛ. The plan then times the **winners program**, each slot at its measured winner, and serves it. If the winners program fails, the plan serves the -reference program. A slot whose winner record already covers its catalog is not re-measured. +reference program and writes no program record, so the next run re-times only the winners +program. A slot whose winner record already covers its catalog is not re-measured. While a knob +filter or workspace limit is active, the plan times the filtered catalogs, serves the winner and +writes no records. **Winner records are per slot**, ranking that slot's kernels by the whole-program times above. A -separate record stores the winners program and its time, which is the engine's measured time. +separate **program record** stores the winners program, as a slot-to-kernel-id map, and its time, +which is the engine's measured time. A program record is valid only if every kernel it names is in +the current catalogs and equals the slot winners; validity is checked on read. Slot records and the +program record are written in one locked append. **Heuristics.** - A slot's ranker orders only its own slot. It predicts no time, since a slot has none. - The engine's estimate comes only from its engine-level model (`predict_engine`), which models the whole program. -- A configuration-level prediction comes only from measurement. It exists once every slot has a - winner record and the winners program has been timed, and only for knob settings that leave that - program unchanged. +- A configuration-level prediction comes only from measurement. It exists while a valid program + record exists, and only for knob settings that leave that program unchanged; otherwise it is + unavailable. **Training data.** Rankers are trained on benchmark data. In v1 the benchmark tool times every -enumerated candidate, which for a slotted engine is the full product of slot catalogs. Slot rankers -train on the rows where the other slots hold their reference picks. +enumerated candidate, which for a slotted engine is the full product of slot catalogs; a reference +kernel that fails to launch fails the sweep. Training reference picks are each slot's first kernel in +kernel-id order, marked by a `slot_reference` column; slot rankers train on the rows where the other +slots hold their training reference picks. These can differ from the runtime reference picks, which +is acceptable under the separability assumption (§ 11). ## 8. Validation @@ -219,18 +240,26 @@ rule in § 4: engine-level selection members beside `slots`, an empty or misname ranker that declares a metric, `shared_workspace` without a native graph match, a pack with a missing or unknown `slot`, or one kernel in two slots. The existing per-engine checks (RFC 0020 § 13.2, RFC 0019 § 6.3, RFC 0018 § A.5) run per slot. A slot with no loadable pack is a warning: -the engine stays registered and declines every graph. +the engine stays registered and declines every graph. An unslotted engine with no loadable pack is +still dropped, as today. Within a slotted engine, a duplicate metadata tuple in a slot or a duplicate +kernel id in a slot's arch group drops the offending pack; unslotted engines keep today's behavior. ## 9. Compatibility -UED and KDP take a minor bump to 1.1, adding `slots`, `shared_workspace` and `slot`. A 1.0 runtime -rejects 1.1 files; a 1.1 runtime reads 1.0 files unchanged. A slot ranker's provenance names its -slot, and a slotted engine's `predict_engine` model names every slot's KMD; both are minor UHD -format changes (RFC 0019 § 8.1). +UED and KDP take a minor bump to 1.1, adding `slots`, `shared_workspace` and `slot`; UHD takes 1.1 +for a slot ranker, whose `trained_against` adds `slot`, and a slotted `predict_engine` model, whose +`trained_against` carries `kmds: [{slot, id, revision}]` instead of `kmd` (RFC 0019 § 8.1). Only +files that use slots are written as 1.1. A 1.0 runtime skips 1.1 files with a warning, and a 1.1 KDP +beside a skipped UED is reported as orphaned; a 1.1 runtime reads 1.0 files unchanged. A 1.0-labelled +file that contains slot members is rejected. An engine without slots keeps its knob ids, trained models and winner records, so nothing trained or stored is invalidated. Adding, removing, renaming or reordering slots is a breaking engine revision, -because knob ids and record keys change. +because knob ids and record keys change. For a slotted engine, winner records carry the slot name, +the cache shard path uses the UED id and the hash of the slot rankers' model hashes in slot order, +and a slot ranker with a model but no hash disables persistence, as for an unslotted engine +(RFC 0019 § 4.1, § 9.2). Winner-record lines move to format v3 for slotted engines only; v2 lines +remain readable. ## 10. Amendments to Other RFCs @@ -239,11 +268,11 @@ statements about an engine's kernel selection apply per slot. The edit list: | RFC | Change | |---|---| -| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 10). | +| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers that read offset tokens (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 10). | | 0018 | Kernel-scoped matchers check against the KMD of the pack's slot (§ 8, § A.5). | | 0019 | KMD, ranker, catalog, winner records, caches and contract checks are per slot; slot rankers are metric-less and train on whole-program times; a slotted engine has no model-based configuration estimate; its benchmark cost is the product of slot catalogs (§ 3, § 4.1, § 5, § 6.3, § 8.1, § 9.2, § 11, § 13.2). | | 0019.13 | Slot rankers train per slot (§ 3.5, § 7.3, § 8.3). | -| 0020 | UED gains `slots` and `shared_workspace`; `metadata` is required only without `slots` (§ 4.2, § 10). Selection members and knobs move to slots, with knob ids `.` (§ 4.2, § 4.6, § 8). A native graph match may publish `$ws.*` tokens (§ 6.1). Load checks run per slot (§ 13). | +| 0020 | UED gains `slots` and `shared_workspace`; `metadata` is required only without `slots` (§ 4.2, § 10). Selection members and knobs move to slots, with knob ids `.` (§ 4.2, § 4.6, § 8). A native graph match may publish `ws.*` tokens (§ 6.1). Load checks run per slot (§ 13). | ## 11. Deferred Work and Open Questions @@ -255,7 +284,7 @@ statements about an engine's kernel selection apply per slot. The edit list: | Per-slot timing | Per-kernel timing inside the program would allow calibrated slot rankers and model-based configuration estimates. | | Cheaper training collection | Collect from benchmarking mode instead of the full product, once winner records carry the benchmark tool's per-run statistics. | | `predict_applicable_kernels` per slot | `slots[].predict_applicable_kernels`. | -| Shared-region tokens from declarative graph matches | A reserved `ws` token root in RFC 0020 § 6.1. | +| Shared-region tokens from declarative graph matches | Declarative publication of the `ws` token root, which this RFC reserves for slotted engines, in RFC 0020 § 6.1. | **Open questions.** From 440caa28ff1208d79467182ca0def9c79764b6d8 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Sun, 18 Oct 2026 15:15:52 +0000 Subject: [PATCH 3/6] docs(hipdnn): RFC 0021 clarifications: slot ranker workspace tokens, shard path wording A slot ranker that reads a ws.* token is disabled at load and refused by the validator, because the root is reserved. The runtime also checks the offset tokens, not only the size token. The cache shard path uses one digest over every slot's name and rankers. --- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md index a5aaa96e44e6..169b410a9249 100644 --- a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -114,7 +114,7 @@ The UED gains two members, and each slot carries the selection members the engin | `slots[].metadata` | The slot's KMD. | | `slots[].sort_kernel_catalog` | The slot's ranker (optional, metric-less; § 7). | | `slots[].knobs` | KMD fields the slot exposes as knobs (optional). | -| `shared_workspace` | The bound token holding the shared region's size (optional; § 6). | +| `shared_workspace` | The bound token holding the shared region's size (optional, only with `slots`; § 6). | A KDP gains `slot`, naming the slot its kernels serve. It is required for a slotted engine and forbidden otherwise. A slot may have several packs. A UKD is unchanged; it fills the KMD of its @@ -174,10 +174,11 @@ depends only on the graph, so the engine's native graph match (provider code, no pattern) computes it and publishes it as bound tokens, named values written `ws.*`. The UED's `shared_workspace` names the token holding the total size. The match also publishes `ws.scratch_offset`, the shared size rounded up to 256 B, and an offset token per shared object -(for example `ws.delta.offset`). The runtime checks the token at catalog build: a missing, non-integer -or negative value makes the engine decline the graph with an error naming the token. For a slotted -engine the `ws.` root is reserved: its tokens are excluded from ranker features, problem features and -training rows. For split SDPA backward, the region +(for example `ws.delta.offset`). The runtime checks the tokens at catalog build: a missing, +non-integer or negative size token, a `ws.scratch_offset` that is not that size rounded up to 256 B, +or an offset token that is not a non-negative integer makes the engine decline the graph with an +error naming the token. For a slotted engine the `ws.` root is reserved: its tokens are excluded +from ranker features, problem features and training rows. For split SDPA backward, the region holds Delta and the sanitized log-sum-exp, sized once for the maximum sequence length. **Workspace size** is the aligned shared region plus the largest scratch any one slot needs. Slots run @@ -244,6 +245,10 @@ the engine stays registered and declines every graph. An unslotted engine with n still dropped, as today. Within a slotted engine, a duplicate metadata tuple in a slot or a duplicate kernel id in a slot's arch group drops the offending pack; unslotted engines keep today's behavior. +A slot ranker whose feature signature reads a `ws.*` token is disabled at load (its engine stays and +the slot orders its catalog by priority, then id) and is refused by the validator: the root is +reserved (§ 6), so the feature could never be bound. + ## 9. Compatibility UED and KDP take a minor bump to 1.1, adding `slots`, `shared_workspace` and `slot`; UHD takes 1.1 @@ -256,8 +261,8 @@ file that contains slot members is rejected. An engine without slots keeps its knob ids, trained models and winner records, so nothing trained or stored is invalidated. Adding, removing, renaming or reordering slots is a breaking engine revision, because knob ids and record keys change. For a slotted engine, winner records carry the slot name, -the cache shard path uses the UED id and the hash of the slot rankers' model hashes in slot order, -and a slot ranker with a model but no hash disables persistence, as for an unslotted engine +the cache shard path uses the UED id and one digest over every slot's name and rankers, in slot +order, and a slot ranker with a model but no hash disables persistence, as for an unslotted engine (RFC 0019 § 4.1, § 9.2). Winner-record lines move to format v3 for slotted engines only; v2 lines remain readable. From 2a16823e6b054a0ac460e5de02a984516674e773 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Sun, 18 Oct 2026 22:38:29 +0000 Subject: [PATCH 4/6] docs(hipdnn): RFC 0021 producer packing policy and public surface changes States what the packer checks for a slotted engine: it mirrors the shape and resolvable rules, and the loader stays authoritative for runtime binding, native registrations, per-arch collisions and empty slots. Adds a table of the public surface changes with how to adapt. --- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 64 +++++++++++++++++-- 1 file changed, 58 insertions(+), 6 deletions(-) diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md index 169b410a9249..f077005acedc 100644 --- a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -119,7 +119,8 @@ The UED gains two members, and each slot carries the selection members the engin A KDP gains `slot`, naming the slot its kernels serve. It is required for a slotted engine and forbidden otherwise. A slot may have several packs. A UKD is unchanged; it fills the KMD of its pack's slot. A kernel serves at most one slot of an engine, so logs and records can name it without -its slot. +its slot. A producer also reads `slot` to pick the KMD that completes a pack's metadata and the +ranker that orders it; no wire format other than the member itself changes. ```jsonc // UED @@ -236,6 +237,29 @@ is acceptable under the separability assumption (§ 11). ## 8. Validation +**Producer.** A producer packer (`hkp_pack`) rejects the statically checkable conditions below +before it writes a bundle, with the loader's rules and messages where the two overlap: + +- the version gates: `slots`, `shared_workspace` and KDP `slot` need format 1.1, and a KDP of an + unslotted engine has no `slot`; +- the `slots` shape: nonempty, names matching § 4, no `global`, no duplicates, no engine-level + selection members beside `slots`, no top-level `metadata` on a slotted engine; +- `shared_workspace` only beside `slots`, with a native graph match and a nonempty token; +- a KDP `slot` that names a declared slot, and a kernel id that serves at most one slot where the + identities are static; +- each slot's KMD, which completes a pack's metadata for compilation and desk check, and each + slot's knobs, checked against that slot's KMD; +- slot rankers without a metric and without `ws.*` reads in their feature signatures; +- UHD 1.1 provenance: a slot ranker's `slot` and `kmd`, and a slotted `predict_engine` model's + `kmds`, which name every declared slot once with that slot's KMD id. + +Every slot's KMD and ranker is reachable from its engine, so a pruned shard keeps them. A slotted +engine with an empty slot is not a packer error, because the loader permits it. + +**Loader.** The SDK load stays authoritative for what a packer cannot decide: model binding and +provenance revisions, native registrations, per-arch collisions and preflight, and a slot with no +loadable pack. + At load, the runtime applies RFC 0020 § 13 and rejects a slotted engine, or a pack, that breaks a rule in § 4: engine-level selection members beside `slots`, an empty or misnamed slot list, a slot ranker that declares a metric, `shared_workspace` without a native graph match, a pack with a @@ -244,10 +268,12 @@ missing or unknown `slot`, or one kernel in two slots. The existing per-engine c the engine stays registered and declines every graph. An unslotted engine with no loadable pack is still dropped, as today. Within a slotted engine, a duplicate metadata tuple in a slot or a duplicate kernel id in a slot's arch group drops the offending pack; unslotted engines keep today's behavior. +The outcomes stay distinct: a broken engine is dropped, a broken pack is dropped, a model with bad +provenance is disabled while its engine stays, and an empty slot warns. A slot ranker whose feature signature reads a `ws.*` token is disabled at load (its engine stays and -the slot orders its catalog by priority, then id) and is refused by the validator: the root is -reserved (§ 6), so the feature could never be bound. +the slot orders its catalog by priority, then id) and is refused by the validator and the producer +packer: the root is reserved (§ 6), so the feature could never be bound. ## 9. Compatibility @@ -258,6 +284,10 @@ files that use slots are written as 1.1. A 1.0 runtime skips 1.1 files with a wa beside a skipped UED is reported as orphaned; a 1.1 runtime reads 1.0 files unchanged. A 1.0-labelled file that contains slot members is rejected. +Producer packers accept the 1.1 forms of all three formats, including the UHD `slot` and `kmds` +provenance, and refuse them under a 1.0 label. A bundle without slots is written as 1.0 with the +same bytes it has without slot support. + An engine without slots keeps its knob ids, trained models and winner records, so nothing trained or stored is invalidated. Adding, removing, renaming or reordering slots is a breaking engine revision, because knob ids and record keys change. For a slotted engine, winner records carry the slot name, @@ -273,11 +303,33 @@ statements about an engine's kernel selection apply per slot. The edit list: | RFC | Change | |---|---| -| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers that read offset tokens (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 10). | +| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers that read offset tokens (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 4, § 10). Packaging validation applies the shape rules of RFC 0021 § 4, and a pack's KMD is the KMD of its slot for the packer's compilation completion and desk check (RFC 0021 § 8). | | 0018 | Kernel-scoped matchers check against the KMD of the pack's slot (§ 8, § A.5). | -| 0019 | KMD, ranker, catalog, winner records, caches and contract checks are per slot; slot rankers are metric-less and train on whole-program times; a slotted engine has no model-based configuration estimate; its benchmark cost is the product of slot catalogs (§ 3, § 4.1, § 5, § 6.3, § 8.1, § 9.2, § 11, § 13.2). | +| 0019 | KMD, ranker, catalog, winner records, caches and contract checks are per slot; slot rankers are metric-less and train on whole-program times; a slotted engine has no model-based configuration estimate; its benchmark cost is the product of slot catalogs (§ 3, § 4.1, § 5, § 6.3, § 8.1, § 9.2, § 11, § 13.2). Packaging reaches and checks every slot's ranker, and the `trained_against` provenance of a slot ranker or the engine estimate names declared slots and their KMDs (§ 8). | | 0019.13 | Slot rankers train per slot (§ 3.5, § 7.3, § 8.3). | -| 0020 | UED gains `slots` and `shared_workspace`; `metadata` is required only without `slots` (§ 4.2, § 10). Selection members and knobs move to slots, with knob ids `.` (§ 4.2, § 4.6, § 8). A native graph match may publish `ws.*` tokens (§ 6.1). Load checks run per slot (§ 13). | +| 0020 | UED gains `slots` and `shared_workspace`; `metadata` is required only without `slots` (§ 4.2, § 10). Selection members and knobs move to slots, with knob ids `.` (§ 4.2, § 4.6, § 8). A native graph match may publish `ws.*` tokens (§ 6.1). Load checks run per slot (§ 13). Packaging reaches every slot's KMD and checks each slot's knobs against that slot's KMD (§ 8). | + +### Public surface changes + +Unslotted engines behave as before. "Source" marks changes that affect code built against the SDK +headers. + +| Item | Change | Affected | How to adapt | +|---|---|---|---| +| `GenericEngine` constructor | Gains a trailing defaulted `BenchmarkPlan::Timer timer = {}` parameter. | None: existing calls compile unchanged. | Pass a timer only to inject benchmark timing, as tests do. | +| `HeuristicProvenance` | Gains `slot` (string) and `kmds` (`std::vector`) members between `kmd` and `umd`. | Source: positional aggregate initialization. | Initialize by member name, or supply `slot` and `kmds` before `umd`. | +| `KernelIngestorStateManager::metadataSchema()` | Throws `std::logic_error` on a slotted manager, like the other unslotted-only accessors. | Source: code that reads the schema of an engine that may be slotted. | Test `isSlotted()`, and call `slotSchema(slot)` with the slot's index for a slotted engine. | +| `SlotProgramBenchmark.hpp`, `BenchmarkDetail.hpp` | New SDK headers: the slotted benchmark plan (§ 7) and the sampling machinery it shares with `BenchmarkPlan`. | None; both are SDK-internal. | Include them only to use those types directly. | +| `hipdnn_bench enumerate`, `--json` | Enumerated pages carry `slot_kernels` (a `{slot, kernel}` list in slot order) on slotted candidates. `--json` result rows of a slotted sweep carry `slot_reference`, the names of the slots whose training rows the candidate is, joined by `\|` (empty when it is none). Unslotted output is unchanged. | Consumers of the bench JSON. | Ignore the keys, or read them to train per-slot rankers (§ 7). | +| `hipdnn_bench` sweep CSV | A slotted sweep adds the columns `kernel` (the composite candidate id) and `slot_reference` (as in `--json`) between the `kernel.` columns and `engine`. | Consumers of the sweep CSV. | Read columns by header name, not position. | +| Candidate ids | A candidate id that contains `=` is a slotted composite (`prep=\|dkdv=\|dq=`, § 5); a malformed composite is an error. | Code that parses candidate ids. | Split on `\|`, then `=`. A bare kernel id (a UUID) contains no `=`. | +| Descriptor formats | UED and KDP gain `slots`, `shared_workspace` and `slot`; a UHD's `trained_against` gains `slot` beside `kmd`, or `kmds` in place of `kmd`. All require version 1.1 (`uhd.schema.json` admits `"1.1"` and states the rule), and a slot member under a 1.0 label is refused by the loader, the validator and the packer. | Descriptor producers and consumers. | Label slotted documents 1.1; unslotted documents stay as they are. | +| `makeStateManager`, `makeKernelHeuristic`, `UhdKernelHeuristic::makeResolver`, `UhdKernelHeuristic::create` | `makeStateManager` gains trailing defaulted `slotKnobs` and `sharedWorkspaceToken` parameters; the other three gain a trailing `bool reserveWorkspaceTokens = false`. | None: existing calls compile unchanged. | Pass the new arguments only to build a slotted engine. | +| `winners.jsonl` | Slotted engines write format v3: per-slot lines carry `slot`, and `program` lines record the selected slot combination. Unslotted lines stay v2. | Readers of the winner cache. | Decode v2 and v3, and tell per-slot records from program records. | +| `uhd_gen --slot` | `dataset export`, `generate` and `knobs` select one slot's reference rows and rename its `kernel..` columns to `kernel.`; `generate --slot` writes a version 1.1 UHD. `--slot` refuses a sweep that is not slotted, and `generate --slot` needs `--dataset`. | Training pipelines. | Sweep with `hipdnn_bench`, then run `generate --dataset ... --slot ` once per slot. | +| `uhd_gen promote` | Installs a slot ranker under `heuristics///sort_kernel_catalog//`, reading the slot from the model's `trained_against.slot`. It refuses a `sort_kernel_catalog` model with no slot, an unknown slot, a slot ranker that declares `score.metric`, and any `predict_applicable_kernels` model on a slotted engine. `generate` without `--dataset` refuses a slotted engine. | Training and install pipelines. | Train each slot with `generate --slot`, then `promote` each model. | +| Descriptor and cache types | `EngineDescriptor` gains `slots` and `sharedWorkspaceToken`, `DescriptorSet` gains `slots`, `KernelDescriptorPack`, `CatalogKey` and `WinnerKey` gain `slot`; all default to empty. `HeuristicProvenance` is the one mid-struct insertion (above). | None: existing initializations compile unchanged. | Read each slot's schema, packs and ranker for a slotted set instead of the singular selection fields. | +| `hkp_pack` | Accepts slotted UED, KDP and UHD bundles and resolves each pack against its slot's KMD. A pack that names no declared slot, a kernel reused across slots, a slot ranker that declares a metric or reads `ws.*`, and slot provenance that disagrees with the slot fail packaging. | Bundle producers. | Declare a slot for every pack, keep each kernel id in one slot, and give slot rankers matching `slot` and `kmd` provenance. | ## 11. Deferred Work and Open Questions From c9e131783c303178828a1cbf26fea9474ae8afda Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Fri, 23 Oct 2026 19:30:59 +0000 Subject: [PATCH 5/6] docs(hipdnn): RFC 0021 composes with multi-launch sources RFC 0017 now ships the kernel-descriptor half of multi-launch (UKD 1.1 `sources`, #14158) and states that kernel slots are orthogonal to it. RFC 0021 no longer claims to replace RFC 0017 section 15.1: a slot selects one UKD, and that UKD may carry `sources`. The launch-set check runs per slot pack, so packs with different launch sets name different dispatch descriptors. A multi-launch slot kernel reports the scratch of all its launches. Descriptor versions are per file type (UKD 1.1 means `sources`; UED, KDP and UHD 1.1 mean slots). Dependent kernels in one slot can also be one UKD whose code object holds them, launched by symbol from the slot's native handler. That form serves `hsaco` code objects, which a producer does not pack into `sources`. RFC 0017 gains a roadmap row and a composition note pointing at RFC 0021, and the slotted workspace sentence in section 8.4. --- .../rfcs/0017_UniversalKernelDescriptor.md | 7 +- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 75 +++++++++++-------- 2 files changed, 50 insertions(+), 32 deletions(-) diff --git a/projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md b/projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md index be2d82ca8a26..44e81160e820 100644 --- a/projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md +++ b/projects/hipdnn/docs/rfcs/0017_UniversalKernelDescriptor.md @@ -130,6 +130,7 @@ adapter yet. | Heuristic sources | `NATIVE` (a registered C++ score symbol). Not yet built: LightGBM model; custom C-API library | other model formats, static tables | | Runtime drop-in | None | prebuilt code objects (opt-in, off by default); JIT-compiled sources. Prebuilt code objects ship through build-time packing (`kind: hsaco`) | | Multi-kernel launch program (e.g. SDPA backward) | Kernel descriptor: a `sources` map of several launches ([§15.1](#151-several-kernels-for-one-operation)), authored as `rocke` entries and launched in an order the pack's native handler owns | the dispatch-descriptor `launches[]` program, intermediates and cross-step checks; `hip` and `hsaco` entries; composition | +| Independently selected kernels per engine (e.g. split SDPA backward) | None | kernel slots, [RFC 0021](0021_KernelSlots.md) (draft); orthogonal to `sources` ([§15.4](#154-execution-and-selection)) | | Selection composition: UCD (Universal Composite Descriptor) decomposition | None | composition | | JIT compilation; normalized providers | None | JIT | | Authoring workflow | agentic skills that draft, validate, and inspect descriptors | further skills for retraining, packaging, and migration | @@ -1543,7 +1544,9 @@ arrives with no execution context, so in general no kernel has been chosen yet. filter the config carries, load the dispatch descriptor of each surviving kernel's pack, and evaluate its workspace requirement over the cached bound token state: `workspace_bytes`, or for a multi-launch program the sum of its intermediates and each Launch's own scratch -([Section 15.3](#153-intermediate-buffers)). Report the **maximum** across survivors. +([Section 15.3](#153-intermediate-buffers)). Report the **maximum** across survivors. (For an engine +with kernel slots, the shared region plus the largest slot scratch; +[RFC 0021 § 6](0021_KernelSlots.md#6-shared-workspace-and-arguments).) *Why the dispatch descriptor loads here:* workspace is the first question whose answer is a dispatch property. *Stored:* the parsed dispatch descriptors, in the descriptor cache; the plan build reuses them. @@ -2107,7 +2110,7 @@ a descriptor format with the subsystem it drives. Together they form the planned | KMD + metadata schema | The metadata format plus the field/type/default declaration and the feature contract the heuristic and matchers consume | | Runtime drop-in | Loading custom bundles, compatibility gating, and source-trust rules. Five open questions land here: the enablement and location mechanism; the minimum trust requirement for drop-in source, including whether JIT source is allowed at all; whether a pack may carry its own engine/heuristic pair or must bind an installed one; how an extension provider (a DSL wheel) is bundled and released; and the three removal/deprecation cases | | Adapters | Registering kernel-source and heuristic adapters | -| Composition | Multi-kernel launch, intermediate buffers, and composite (UCD) graph decomposition | +| Composition | Multi-kernel launch, intermediate buffers, and composite (UCD) graph decomposition. Independently selected kernels per engine are kernel slots ([RFC 0021](0021_KernelSlots.md), draft) | | JIT and normalized providers | JIT sources, general pattern matching, and normalizing existing providers onto the descriptor system | --- diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md index f077005acedc..a1b9fe9e9843 100644 --- a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -25,7 +25,8 @@ Terms follow [RFC 0017 § 2](0017_UniversalKernelDescriptor.md#2-the-descriptors): a **UED** is an engine, a **KDP** (pack) binds matchers and a dispatch to a set of kernels, a **UKD** is one -kernel, and a **KMD** is the metadata schema its kernels fill in. An engine's **catalog** is the +kernel, selected as a unit even when its `sources` launch several kernels (RFC 0017 § 15.1), and a +**KMD** is the metadata schema its kernels fill in. An engine's **catalog** is the kernels applicable to a graph; its **ranker** orders the catalog; a **winner record** stores the order measurement found. @@ -39,9 +40,9 @@ knobs and winner records. The slots share the engine's identity, its graph match region for the data they hand each other. Slots sit between the engine and its packs: **UED → slots → KDPs → UKDs**. Everything below a slot -is unchanged; a UKD is still one kernel. An engine that declares no slots has one implicit slot and -behaves as it does today. To a caller, a slotted engine looks like any other: a configuration is an -engine plus knob settings. +is unchanged; a slot selects one UKD, which may itself launch several kernels through `sources`. An +engine that declares no slots has one implicit slot and behaves as it does today. To a caller, a +slotted engine looks like any other: a configuration is an engine plus knob settings. Independent slots let each stage be tuned on its own: for split SDPA backward, 253 per-kernel configurations instead of 613 fixed pairs (§ 2). The cost is that a slot has no time of its own, so @@ -64,13 +65,16 @@ databases for dK/dV and dQ Covering 80% of their tuned problem shapes takes 253 distinct configurations chosen per kernel, against 613 distinct (dK/dV, dQ) pairs. -RFC 0017 offers two shapes for multi-kernel work, and neither fits: +RFC 0017 offers two shapes for multi-kernel work, and neither selects the stages independently: -- **Multi-launch pack (§ 15.1).** One UKD holds every launch and is selected as a unit. Each - (dK/dV, dQ) pair must be authored as its own UKD, with `prep` duplicated into each. +- **Multi-launch UKD (§ 15.1, `sources`).** One UKD holds every launch and is selected as a unit. + Choosing dK/dV and dQ separately means one UKD per (dK/dV, dQ) pair, with `prep` repeated in each. - **Composite engine (UCD, § 15.2).** Each stage is a separate engine answering a child graph. No graph node means "dQ only", and Delta is not a graph tensor. +Slots and `sources` are orthogonal (RFC 0017 § 15.4) and compose: a slot selects among UKDs, and the +UKD it selects may carry `sources`. + ## 3. The Slot Model ``` @@ -100,8 +104,11 @@ what reads the graph. Four rules define the model: computes the same result. Kernels whose choices depend on each other belong in **one** slot, launched together by that slot's -dispatch handler. Examples are a backward kernel with an optional output-conversion kernel, or a -split-KV kernel with the reduction that consumes its splits. +dispatch handler: either one multi-launch UKD (RFC 0017 § 15.1), or one UKD whose code object holds +the dependent kernels, which the slot's native handler launches by symbol names its KMD records. +The second form needs no `sources`, so it also serves code objects authored as `hsaco`, which a +producer does not pack into `sources`. Examples are a backward kernel with an optional reduction or +output-conversion kernel, or a split-KV kernel with the reduction that consumes its splits. ## 4. Descriptor Format @@ -118,9 +125,13 @@ The UED gains two members, and each slot carries the selection members the engin A KDP gains `slot`, naming the slot its kernels serve. It is required for a slotted engine and forbidden otherwise. A slot may have several packs. A UKD is unchanged; it fills the KMD of its -pack's slot. A kernel serves at most one slot of an engine, so logs and records can name it without -its slot. A producer also reads `slot` to pick the KMD that completes a pack's metadata and the -ranker that orders it; no wire format other than the member itself changes. +pack's slot, with `kernel_source` or `sources`. The launch-set rule of RFC 0017 § 15.1 holds per +pack, so slots may differ: a single-launch `prep` can sit beside a multi-launch `dkdv`. A pack's +launch set is its dispatch handler's, so packs with different launch sets name different dispatch +descriptors; in practice each slot has its own dispatch descriptor and native handler. A kernel +serves at most one slot of an engine, so logs and records can name it without its slot. A producer +also reads `slot` to pick the KMD that completes a pack's metadata and the ranker that orders it; no +wire format other than the member itself changes. ```jsonc // UED @@ -184,12 +195,13 @@ holds Delta and the sanitized log-sum-exp, sized once for the maximum sequence l **Workspace size** is the aligned shared region plus the largest scratch any one slot needs. Slots run one at a time, so they reuse the same scratch. A slotted kernel's workspace query reports its scratch -only. Every handler receives the unshifted workspace base; in v1 each slot dispatches through a native -handler, which reads the offset tokens in `prepare` and adds them to the base at launch. The plan -rejects a null workspace when shared plus scratch is non-zero. The workspace limit applies to the -total: a kernel is pruned from its slot when the aligned shared region plus its scratch exceeds the -limit, and the engine declines when the shared region alone exceeds it. The shared region does not -depend on any pick, so slots stay independent. +only; for a multi-launch kernel that is the scratch of all its launches, which its handler owns +(RFC 0017 § 15.1). Every handler receives the unshifted workspace base; in v1 each slot dispatches +through a native handler, which reads the offset tokens in `prepare` and adds them to the base at +launch. The plan rejects a null workspace when shared plus scratch is non-zero. The workspace limit +applies to the total: a kernel is pruned from its slot when the aligned shared region plus its +scratch exceeds the limit, and the engine declines when the shared region alone exceeds it. The +shared region does not depend on any pick, so slots stay independent. **Shared arguments.** Every runtime argument more than one slot uses (scale, dropout parameters and random-number seed, ragged sequence information, maximum sequence lengths) is computed once in the @@ -264,12 +276,13 @@ At load, the runtime applies RFC 0020 § 13 and rejects a slotted engine, or a p rule in § 4: engine-level selection members beside `slots`, an empty or misnamed slot list, a slot ranker that declares a metric, `shared_workspace` without a native graph match, a pack with a missing or unknown `slot`, or one kernel in two slots. The existing per-engine checks (RFC 0020 -§ 13.2, RFC 0019 § 6.3, RFC 0018 § A.5) run per slot. A slot with no loadable pack is a warning: -the engine stays registered and declines every graph. An unslotted engine with no loadable pack is -still dropped, as today. Within a slotted engine, a duplicate metadata tuple in a slot or a duplicate -kernel id in a slot's arch group drops the offending pack; unslotted engines keep today's behavior. -The outcomes stay distinct: a broken engine is dropped, a broken pack is dropped, a model with bad -provenance is disabled while its engine stays, and an empty slot warns. +§ 13.2, RFC 0019 § 6.3, RFC 0018 § A.5) and the launch-set check (RFC 0017 § 15.1) run per slot. A +slot with no loadable pack is a warning: the engine stays registered and declines every graph. An +unslotted engine with no loadable pack is still dropped, as today. Within a slotted engine, a +duplicate metadata tuple in a slot or a duplicate kernel id in a slot's arch group drops the +offending pack; unslotted engines keep today's behavior. The outcomes stay distinct: a broken +engine is dropped, a broken pack is dropped, a model with bad provenance is disabled while its +engine stays, and an empty slot warns. A slot ranker whose feature signature reads a `ws.*` token is disabled at load (its engine stays and the slot orders its catalog by priority, then id) and is refused by the validator and the producer @@ -280,13 +293,15 @@ packer: the root is reserved (§ 6), so the feature could never be bound. UED and KDP take a minor bump to 1.1, adding `slots`, `shared_workspace` and `slot`; UHD takes 1.1 for a slot ranker, whose `trained_against` adds `slot`, and a slotted `predict_engine` model, whose `trained_against` carries `kmds: [{slot, id, revision}]` instead of `kmd` (RFC 0019 § 8.1). Only -files that use slots are written as 1.1. A 1.0 runtime skips 1.1 files with a warning, and a 1.1 KDP -beside a skipped UED is reported as orphaned; a 1.1 runtime reads 1.0 files unchanged. A 1.0-labelled -file that contains slot members is rejected. +files that use slots are written as 1.1. Versions are per file type: a UKD's 1.1 means `sources` +(RFC 0017 § 15.1), independent of slots, so a slotted engine's UKDs are 1.0 or 1.1 by their own +content. A 1.0 runtime skips 1.1 files with a warning, and a 1.1 KDP beside a skipped UED is +reported as orphaned; a 1.1 runtime reads 1.0 files unchanged. A 1.0-labelled file that contains +slot members is rejected. Producer packers accept the 1.1 forms of all three formats, including the UHD `slot` and `kmds` -provenance, and refuse them under a 1.0 label. A bundle without slots is written as 1.0 with the -same bytes it has without slot support. +provenance, and refuse them under a 1.0 label. A bundle without slots writes its UED, KDP and UHD as +1.0, with the same bytes it has without slot support. An engine without slots keeps its knob ids, trained models and winner records, so nothing trained or stored is invalidated. Adding, removing, renaming or reordering slots is a breaking engine revision, @@ -303,7 +318,7 @@ statements about an engine's kernel selection apply per slot. The edit list: | RFC | Change | |---|---| -| 0017 | Kernel slots replace the multi-launch pack (§ 15.1) and its intermediate buffers (§ 15.3) for multi-kernel engines; the UCD (§ 15.2) is unaffected. KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers that read offset tokens (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 4, § 10). Packaging validation applies the shape rules of RFC 0021 § 4, and a pack's KMD is the KMD of its slot for the packer's compilation completion and desk check (RFC 0021 § 8). | +| 0017 | Slots are orthogonal to multi-launch `sources` (§ 15.1, § 15.4): a slot's UKD may carry `sources`, and the launch-set check runs per slot pack. Slots need neither the deferred `launches[]` program nor intermediate buffers (§ 15.3): the native graph match publishes the shared region (§ 6). KDPs gain `slot` (§ 4). Applicability requires a kernel in every slot, and workspace includes the shared region (§ 8.1, § 8.4, § 8.6). Declarative workspace arguments are not offset past the shared region, so v1 slots use native handlers that read offset tokens (§ 6). Metadata-tuple uniqueness is per slot; a kernel id belongs to one slot (§ 4, § 10). Packaging validation applies the shape rules of RFC 0021 § 4, and a pack's KMD is the KMD of its slot for the packer's compilation completion and desk check (RFC 0021 § 8). | | 0018 | Kernel-scoped matchers check against the KMD of the pack's slot (§ 8, § A.5). | | 0019 | KMD, ranker, catalog, winner records, caches and contract checks are per slot; slot rankers are metric-less and train on whole-program times; a slotted engine has no model-based configuration estimate; its benchmark cost is the product of slot catalogs (§ 3, § 4.1, § 5, § 6.3, § 8.1, § 9.2, § 11, § 13.2). Packaging reaches and checks every slot's ranker, and the `trained_against` provenance of a slot ranker or the engine estimate names declared slots and their KMDs (§ 8). | | 0019.13 | Slot rankers train per slot (§ 3.5, § 7.3, § 8.3). | From bcd1c8387a54ef226bb31d3b0e29cdb479e40459 Mon Sep 17 00:00:00 2001 From: Brian Harrison Date: Sat, 24 Oct 2026 19:05:11 +0000 Subject: [PATCH 6/6] docs(hipdnn): RFC 0021 public surface and plan serialization CatalogGraphStage carries one first-pack position per slot and the slotted decline reason and shared workspace size, because staged applicability covers slotted engines. KernelIngestorStateManager::cacheKey takes an optional slot. Saving a slot program under RFC 0009 is deferred: a slotted engine does not advertise plan serialization and its plans are not saved. --- projects/hipdnn/docs/rfcs/0021_KernelSlots.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md index a1b9fe9e9843..8168d01e14a4 100644 --- a/projects/hipdnn/docs/rfcs/0021_KernelSlots.md +++ b/projects/hipdnn/docs/rfcs/0021_KernelSlots.md @@ -334,7 +334,7 @@ headers. | `GenericEngine` constructor | Gains a trailing defaulted `BenchmarkPlan::Timer timer = {}` parameter. | None: existing calls compile unchanged. | Pass a timer only to inject benchmark timing, as tests do. | | `HeuristicProvenance` | Gains `slot` (string) and `kmds` (`std::vector`) members between `kmd` and `umd`. | Source: positional aggregate initialization. | Initialize by member name, or supply `slot` and `kmds` before `umd`. | | `KernelIngestorStateManager::metadataSchema()` | Throws `std::logic_error` on a slotted manager, like the other unslotted-only accessors. | Source: code that reads the schema of an engine that may be slotted. | Test `isSlotted()`, and call `slotSchema(slot)` with the slot's index for a slotted engine. | -| `SlotProgramBenchmark.hpp`, `BenchmarkDetail.hpp` | New SDK headers: the slotted benchmark plan (§ 7) and the sampling machinery it shares with `BenchmarkPlan`. | None; both are SDK-internal. | Include them only to use those types directly. | +| `SlotProgramBenchmark.hpp`, `BenchmarkDetail.hpp`, slotted types | Two new installed SDK headers: the slotted benchmark plan (§ 7) and the sampling machinery it shares with `BenchmarkPlan`. `GenericPlan.hpp` gains `SlotStage` and `SlotProgramPlan`, `KernelIngestorStateManager` gains slot overloads and accessors, and `MetadataSchema` gains `fieldNames()`. | None: all additions. | Use them only to build or inspect a slotted plan. | | `hipdnn_bench enumerate`, `--json` | Enumerated pages carry `slot_kernels` (a `{slot, kernel}` list in slot order) on slotted candidates. `--json` result rows of a slotted sweep carry `slot_reference`, the names of the slots whose training rows the candidate is, joined by `\|` (empty when it is none). Unslotted output is unchanged. | Consumers of the bench JSON. | Ignore the keys, or read them to train per-slot rankers (§ 7). | | `hipdnn_bench` sweep CSV | A slotted sweep adds the columns `kernel` (the composite candidate id) and `slot_reference` (as in `--json`) between the `kernel.` columns and `engine`. | Consumers of the sweep CSV. | Read columns by header name, not position. | | Candidate ids | A candidate id that contains `=` is a slotted composite (`prep=\|dkdv=\|dq=`, § 5); a malformed composite is an error. | Code that parses candidate ids. | Split on `\|`, then `=`. A bare kernel id (a UUID) contains no `=`. | @@ -344,6 +344,8 @@ headers. | `uhd_gen --slot` | `dataset export`, `generate` and `knobs` select one slot's reference rows and rename its `kernel..` columns to `kernel.`; `generate --slot` writes a version 1.1 UHD. `--slot` refuses a sweep that is not slotted, and `generate --slot` needs `--dataset`. | Training pipelines. | Sweep with `hipdnn_bench`, then run `generate --dataset ... --slot ` once per slot. | | `uhd_gen promote` | Installs a slot ranker under `heuristics///sort_kernel_catalog//`, reading the slot from the model's `trained_against.slot`. It refuses a `sort_kernel_catalog` model with no slot, an unknown slot, a slot ranker that declares `score.metric`, and any `predict_applicable_kernels` model on a slotted engine. `generate` without `--dataset` refuses a slotted engine. | Training and install pipelines. | Train each slot with `generate --slot`, then `promote` each model. | | Descriptor and cache types | `EngineDescriptor` gains `slots` and `sharedWorkspaceToken`, `DescriptorSet` gains `slots`, `KernelDescriptorPack`, `CatalogKey` and `WinnerKey` gain `slot`; all default to empty. `HeuristicProvenance` is the one mid-struct insertion (above). | None: existing initializations compile unchanged. | Read each slot's schema, packs and ranker for a slotted set instead of the singular selection fields. | +| `CatalogGraphStage` | `firstPack` becomes a `std::vector` with one entry per slot (one entry for an unslotted engine). The struct gains `declineReason` and `sharedWorkspaceBytes`. | Source: code that builds or reads the stage passed from `beginCatalog` to `finishCatalog`. | Use `firstPack.front()` for an unslotted engine, and leave the new members at their defaults. | +| `KernelIngestorStateManager::cacheKey` | Gains a trailing defaulted `std::string_view slot` parameter. | None: existing calls compile unchanged. | Pass a slot only to key one slot's catalog in a slotted engine. | | `hkp_pack` | Accepts slotted UED, KDP and UHD bundles and resolves each pack against its slot's KMD. A pack that names no declared slot, a kernel reused across slots, a slot ranker that declares a metric or reads `ws.*`, and slot provenance that disagrees with the slot fail packaging. | Bundle producers. | Declare a slot for every pack, keep each kernel id in one slot, and give slot rankers matching `slot` and `kmd` provenance. | ## 11. Deferred Work and Open Questions @@ -357,6 +359,7 @@ headers. | Cheaper training collection | Collect from benchmarking mode instead of the full product, once winner records carry the benchmark tool's per-run statistics. | | `predict_applicable_kernels` per slot | `slots[].predict_applicable_kernels`. | | Shared-region tokens from declarative graph matches | Declarative publication of the `ws` token root, which this RFC reserves for slotted engines, in RFC 0020 § 6.1. | +| Saving a slot program (RFC 0009) | A saved ingestor plan holds one kernel's code object and launch inputs. A slotted engine does not advertise `SUPPORTS_EXECUTION_PLAN_SERIALIZATION`, and a save of its plan is refused. Saving needs a payload that carries every slot's pick. | **Open questions.**