[BradPepersAMD]

JIRA ID: ALMIOPEN-2953, ALMIOPEN-2954, ALMIOPEN-2955

Supersedes #13713 and #13714 (folded in here), the WIP draft #13157, and #13888.

Summary

Adds FlyDSL kernels to the hip-kernel-provider as a second kernel source for the kernel ingestor, beside rocKE: the pipeline that ships prebuilt FlyDSL code objects, and nine engines on top of it.

Area	Engines	Targets
RMSNorm	hipkernel:flydsl_rmsnorm (forward, inference and training), hipkernel:flydsl_rmsnorm_bwd (dx, dgamma)	gfx11-generic, gfx12-generic, gfx90a, gfx942, gfx950
SDPA forward, RDNA	hipkernel:flydsl_sdpa_rdna, hipkernel:flydsl_sdpa_rdna_decode, hipkernel:flydsl_sdpa_aiw (gfx12 only)	gfx11-generic, gfx12-generic
SDPA forward, CDNA	hipkernel:flydsl_sdpa_cdna, hipkernel:flydsl_sdpa_cdna_decode, hipkernel:flydsl_sdpa_cdna_dualwave (gfx950 only)	gfx90a, gfx942, gfx950

This was a stack of three PRs (#13712 → #13713 → #13714). It is now one PR, reviewed as one change; the three commits keep their own messages.

How it works

FlyDSL kernels are compiled ahead of time into code objects (.hsaco), with descriptors generated from them, as a FlyDSL/ bundle of the provider's production descriptor root. The provider's shared packer packs them into each architecture's kpack shard, as it does for every other bundle. Neither the build nor the runtime needs FlyDSL or a kernel compiler, and the runtime needs no Python: by the time hipDNN sees them, they are ordinary code objects in an archive. The build runs Python 3 (already required by the provider's packer) for gen_descriptors.py --check and tools/check_shards.py. Each engine registers four native symbols (graph match, kernel match, score, dispatch) through the existing ingestor pack table.

The objects are stored with DVC, not in git. Each FlyDSL/<op>/<target>/ directory (the objects, their manifest.json and the target's KDP: 1,453 objects, ~29 MB in all) is a DVC output on the ingestor remote (s3://therock-dvc/rocm-libraries/hipdnn/ingestor, anonymous reads), the one develop's rocKE descriptors use (#13153). Git holds one <op>/<target>.dvc sidecar per directory (22), which records the directory's content hash, plus the human-facing files: each target's SOURCE.md, the op-level descriptors (KMD/UDD/UED/UHD/UMD), the generators and the docs.

• Fetch before configuring: dvc pull -r ingestor dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/FlyDSL/*/*.dvc (TheRock's fetch_sources pulls every pointer).
• Same rule as rocKE's DVC files. The provider's existing configure-time check ([hipDNN] ALMIOPEN-2832: ingestor DVC remote and flag-gated rocKE descriptors #13153) now covers FlyDSL's pointers too: a directory that is missing or incomplete (file count vs its pointer) gets a warning with its dvc pull command, and that target ships no FlyDSL kernels; builds and tests still run. -DHIPKERNELPROVIDER_REQUIRE_DVC_DESCRIPTORS=ON, which the superbuild CI and release builds set, makes it an error.
• Integrity is unchanged: each manifest.json still records every object's SHA256, which gen_descriptors.py --check verifies before every pack, and the sidecar's hash pins the directory as a whole.

Objects per target:

Target	Objects
gfx11-generic	309 (RMSNorm 90 + 63, SDPA 84 + 72)
gfx12-generic	381 (RMSNorm 90 + 63, SDPA 84 + 72 + 72)
gfx950	253 (RMSNorm 90 + 63, SDPA 56 + 20 + 24)
gfx942	255 (RMSNorm 90 + 63, SDPA 78 + 24)
gfx90a	255 (RMSNorm 90 + 63, SDPA 78 + 24)

RMSNorm

153 objects per target (90 forward, 63 backward).

• Data types: bf16, f16 and f32, plus bf16/f16 with an f32 scale (the mixed form cuDNN and PyTorch commonly send).
• Training forward: every forward object has a twin that also writes inv_rms (one f32 per row). TRAINING graphs with a per-row f32 inv_rms are served; TRAINING without it, and inv_rms outside training, decline.
• Two tiers: baked widths for the 16-bit types and a generic tier that reads N at runtime. The baked widths were chosen by measurement: each candidate was timed against the generic tier (profiler kernel time, M = 4096/16384) on gfx90a, gfx942, gfx950, gfx1201 and gfx1100, and kept where it gained at least 15% on two archs. Forward bakes 2560–16384 (ten widths, 1.2–4.2× over generic on all five).
• Epsilon is a kernel argument (vendored modification 4), so any finite, non-negative epsilon is served: 1e-5, 1e-6, PyTorch's eps=None, fixed in the graph or supplied at execute. The engine declares graph schema 1.2.0 so runtime pass-by-value epsilons reach it.
• Shapes: the normalized axes follow the frontend's rule (the trailing axes where scale matches x), so cuDNN's x {B*S, D, 1, 1} / scale {1, D, 1, 1} layout and multi-axis normalized shapes are served. Strides on size-1 axes are ignored, so hipdnn_torch's stride-0 weight is served. Other strided layouts are declined.

Backward (dx and dgamma) comes from upstream FlyDSL's deterministic two-stage kernel, vendored with two recorded modifications (a bounded runtime-N variant; a runtime program count). A partial kernel writes dx and one f32 dgamma row per program into a workspace; a reduce kernel in the same object sums them into dgamma, so dgamma is bitwise reproducible. Same dtypes and gamma forms as the forward; baked widths 1536–16384 (twelve, 1.2–8.4× over generic on all five archs) and generic tiers bounded at 1024/8192/16384 (the partial kernel keeps dgamma columns in registers; no shipped object spills VGPRs or uses scratch, though the CDNA n_max 16384 partial kernels spill a few SGPRs; wider graphs decline). dbias declines. An atomic dgamma variant was measured and not shipped: 2.6–90× slower at large M, and its small-M edge sat inside launch overhead.

SDPA forward, RDNA

Built for the LLVM generic targets gfx11-generic (RDNA3/3.5) and gfx12-generic (RDNA4); each ships to every member arch.

Engine	Objects	What it is
hipkernel:flydsl_sdpa_rdna	84 per family	FlyDSL's RDNA flash-attention forward (from AITER), ported to gfx11 WMMA
hipkernel:flydsl_sdpa_rdna_decode	72 per family	Split-KV decode, with the GQA group packed into one tile, plus its merge kernel
hipkernel:flydsl_sdpa_aiw	72, gfx12-generic only	AOTriton's FlyDSL gfx1201 forward, vendored under attention_rdna4/

• Coverage: bf16/f16 × head_dim 64/96/128/256 × causal; additive bias (f32 or the query's dtype, broadcast over any axis); generic tiers for every other head_dim that is a multiple of 8, up to 256; MLA 192/128; wide single-KV-head decode (576/512 and 512/512); attention sinks (gpt-oss).
• Runtime arguments, so one object serves every shape of its class: batch, both sequence lengths, head counts (MHA/GQA/MQA), every stride, the scale, the mask bounds (causal at either corner, bands, sliding windows) and the LSE output. Everything else is declined so another engine gets the plan. COVERAGE.md §5 lists every rule and the remaining gaps (e.g. padding/varlen).

SDPA forward, CDNA

Built per arch from FlyDSL's own MFMA kernels.

Engine	Arches	Objects
hipkernel:flydsl_sdpa_cdna (generic MFMA forward)	gfx90a, gfx942, gfx950	78 / 78 / 56
hipkernel:flydsl_sdpa_cdna_dualwave (dual-wave MFMA forward)	gfx950	20
hipkernel:flydsl_sdpa_cdna_decode (split-KV decode + merge)	gfx90a, gfx942, gfx950	24 each

• Coverage: bf16/f16 at head_dim 64, 128 and 256, causal and not; bucket objects serve the head dims in between; MLA 192/128; gfx950 adds a bias in the query's dtype at 64 and 128.
• Runtime arguments: both sequence lengths, head counts, every operand's strides (BSHD, BHSD, packed QKV), the scale, the causal diagonal and the LSE output.
• Tile choice. Each class ships a default object plus measured companion tiles, chosen by timing every candidate on the dnn-benchmarking attention graphs (COVERAGE.md §7.6).

None of the SDPA engines needs HIPDNN_ENABLE_SDPA to build; that flag only exposes SDPA in the frontend.

Feature flag

• HIPKERNELPROVIDER_ENABLE_FLYDSL is ON by default, like the ASM SDPA (ENABLE_ASM_SDPA_ENGINE) and ML-ops (ENABLE_HIP_MLOPS_ENGINE) engines, and switched off the same way (-DHIPKERNELPROVIDER_ENABLE_FLYDSL=OFF).
• It takes effect only with HIPDNN_ENABLE_KERNEL_INGESTOR=ON, since only the ingestor loads FlyDSL's shard. The ingestor is OFF by default in develop (and in TheRock), so a default build is unchanged: configure reports flydsl: not built, builds nothing FlyDSL and needs no dvc pull.
• It does not depend on HIPDNN_ENABLE_SDPA, which only gates the frontend's graph->sdpa(): the full stack builds and passes with it OFF.
• HIPKERNELPROVIDER_FLYDSL_ACTIVE (internal) is raised only when the product pack ships FlyDSL objects for a requested architecture. The engine sources, tests and census key on it, so building for an architecture with no FlyDSL objects reports "dormant" and compiles nothing FlyDSL.
• CI. The hipDNN superbuild CI already enables the ingestor (the separate CI-flag PR [hipDNN][FlyDSL] ci: build the hipDNN superbuild with FlyDSL enabled #13888 is no longer needed). The Windows gfx1151 lane packs the production root, so it now pulls them with its existing recursive descriptors pull (no workflow change) and runs the host side: FlyDSL engines, gen_descriptors --check, pack, check_shards.py, census, host tests, and the packaging probes, which include FlyDSL's non-target arches. That lane's runner has no GPU, so no CI lane launches a FlyDSL kernel; device validation is manual (below) until the ingestor flag flips (ALMIOPEN-2835). The Linux gfx942 lane packs the descriptor-packaging fixture tree, so FlyDSL is dormant there by design and it needs no pull. TheRock's Linux CI already pulls the whole repository's DVC content.

What changes outside flydsl/

• Top-level CMakeLists.txt: the option, and add_subdirectory(flydsl) under the ingestor gate.
• HkpPackaging.cmake: FlyDSL added to HKP_DESCRIPTOR_FAMILIES; the packer excludes the folder when the option is OFF.
• Ingestor: one pack-table entry and one module cache per engine, behind HIPDNN_ENGINE_FLYDSL.
• Engine natives under kernel_ingestor_engine/packs/: FlydslRmsNorm{,Bwd}Native.cpp with FlydslRmsNormCommon.hpp; FlydslSdpa{Rdna,RdnaDecode,Aiw}Native.cpp with FlydslSdpaRdnaCommon.hpp; FlydslSdpaCdna{,Dualwave,Decode}Native.cpp with FlydslSdpaCdnaCommon.hpp.
• KernelCompileOptions: an arch-only constructor for packs with no input tensor to derive dtype or layout from.
• descriptors/FlyDSL/: the op-level descriptors, each target's SOURCE.md, the 22 .dvc sidecars (remote: ingestor) and one .gitignore per op listing its DVC-managed directories.
• The provider's DVC pointer check (top-level CMakeLists.txt) loops over rocKE and FlyDSL instead of rocKE alone; HIPKERNELPROVIDER_REQUIRE_DVC_DESCRIPTORS covers both.
• Tests: main.cpp adds the product shard as an additive descriptor root (only when FlyDSL is active); IngestorPackTestSupport gains findLoadedSet(); one engine suite and one census suite per engine. TestContainer: MAX_EXPECTED_ENGINES raised from 8 to 32 (a buffer bound: FlyDSL adds one engine per kernel family), and its SDPA-graph test expects the FlyDSL CDNA engines beside the ASM engine on gfx90a/942/950.

Inside flydsl/

• kernels_src/: vendored kernel sources at pinned commits. FlyDSL at 89ad52fbbb9e (recorded in generators/_flydsl_env.py, each SOURCE.md and THIRD_PARTY_LICENSES.md); AITER (RDNA attention) and AOTriton (attention_rdna4/). Every file keeps its upstream SPDX and copyright lines. Modified files carry a header recording upstream, commit, path and every local modification: rmsnorm_kernel.py (4), rmsnorm_bwd_kernel.py (2); the attention modifications are listed in each file's header, attention_rdna4/README.md and attention_cdna/README.md (1–27).
• THIRD_PARTY_LICENSES.md: FlyDSL's notice and the Apache-2.0 text, and an AOTriton (MIT) section, installed under share/doc/hip-kernel-provider/flydsl/.
• generators/: the instance table (_instances.py, one row per shipped object), the per-op generators and the shared machinery.
• gen_descriptors.py: derives every descriptor from the objects and their manifest: one KDP per op × target, holding one inline hsaco UKD per object. --check runs before every pack and fails on any SHA256, target or input-digest mismatch. It reads the generic-target table CMake passes (HIPDNN_PLUGIN_SDK_GPU_GENERIC_TARGETS_JSON), so provider-only checkouts work.
• tools/check_shards.py: runs after the pack and confirms every member arch's shard ships exactly the shipped objects.
• tools/diff_upstream.py: a review aid that diffs the vendored sources against FlyDSL, AITER and AOTriton (optional --aotriton). Nothing in the build runs it.
• Docs: README.md (what ships, how it reaches hipDNN, fetching the objects), REGEN.md (byte-reproducible regeneration, verification, and the dvc add/dvc push step), COVERAGE.md (every accept and decline rule, with its reason, and measured performance).

Risk and mitigation

Risk	Mitigation
Behaviour change for existing builds	A default develop build (ingestor OFF) is unchanged and needs no DVC. A build with the ingestor ON now gets FlyDSL unless it passes -DHIPKERNELPROVIDER_ENABLE_FLYDSL=OFF; with it OFF nothing FlyDSL compiles and the FlyDSL/ folder is not packed.
Objects not fetched	Configure warns with the exact dvc pull command and that target ships no FlyDSL kernels; with REQUIRE_DVC_DESCRIPTORS=ON (CI, release) it fails. Both verified by configuring against a tree with one directory removed.
Prebuilt binaries drift from their sources	Each manifest records SHA256 per object plus an input digest over the op's sources, generators and instance rows. gen_descriptors.py --check runs before every pack, so a source edit without regeneration fails the build. The CTest hip-kernel-provider-flydsl-drift-guards proves the checks refuse tampered trees.
Binaries can't be reproduced	Regeneration is byte-reproducible with the pinned toolchain (flydsl==0.3.4, ROCm 7.13). REGEN.md §2 gives the procedure.
Engine runs a graph it cannot compute	The matchers gate every graph field they read and decline everything else, so another engine gets the plan. Each rule is listed in COVERAGE.md with a decline test.
Fast-math folding the by-design −inf (keyless rows) into poison	The attention kernels test −inf by bit pattern and select the result explicitly; the split-KV merge was fixed for this (CDNA modification 27). Checked idle and under concurrent GPU load.
One object set for a whole generic family	Objects are stamped and verified as gfx11-generic / gfx12-generic (ELF machine and amdhsa.target, checked by gen_descriptors --check). The packer copies the same bytes into each member's shard; check_shards.py verifies every shard.
Census green against an empty shard	The shard censuses (with negative controls) fail when a suite runs against an empty or absent shard.
Selection between overlapping engines	Engines rank by per-object priority. Autotune compares every applicable engine; UHD heuristics are a follow-up.
Decode engine claiming prefill-sized graphs	Correct (13/13 DeepSeek-V4 head-dim-512 causal graphs at Sq 2048/8192 pass on gfx950) but slow there (~14 TF/s vs PyTorch 210–447), since no prefill object covers d512 yet. Autotune picks another engine where one applies.
Coupling with other integrations	Gated on FlyDSL's own variable, not the ingestor's; no FlyDSL suite references another engine.
Third-party code	Vendored sources are Apache-2.0 (FlyDSL, AITER) and MIT (AOTriton); THIRD_PARTY_LICENSES.md carries the notices and license texts and is installed with the provider.

Testing

Strategy.

• Host tests cover matchers, scoring and decline rules: one decline test per gated field.
• GPU tests run real launches against a host reference.
• The shard censuses prove the staged shards are really there.
• Two build-level checks: gen_descriptors --check before the pack and check_shards after it.
• Byte-identical regeneration ties the binaries to their sources.

This head (one commit on develop 3440eec, objects in DVC; ROCm 7.13 SDK, clang-tidy on, gfx1151 host):

Check	Result
Build with every target staged (GPU_TARGETS=gfx1151;gfx1200;gfx1201;gfx90a;gfx942;gfx950), objects pulled	OK, 0 warnings; shards verified for all six
Shard census and FlyDSL CTests (incl. drift guards)	166/166; *Flydsl* gtests 569 passed on gfx1151 (the 48 skipped are the gfx12 aiw and CDNA SDPA device cases)
Configure with REQUIRE_DVC_DESCRIPTORS=ON, everything pulled	OK
Configure with one <op>/<target>/ directory missing	default: warning with the dvc pull -r ingestor <pointer> command, other targets still ship; REQUIRE_DVC_DESCRIPTORS=ON: configure error
DVC round trip	after dvc push -r ingestor, an anonymous pull into a fresh repository with an empty cache and no credentials returned all 22 directories (1,497 files) byte-identical

The move to DVC changed no object byte: every directory's content is the same as at the previous head, and the manifests' SHA256s still verify.

RMSNorm (at c0443e8 and the later test revision):

Check	Result
FlyDSL gtests, gfx1151	167/167 at the test revision; GPU launches: forward bf16/f16/f32, f32 scale, training (y and inv_rms), epsilon from the graph and at execute, cuDNN layout, stride-0 scale, baked widths; backward dx and dgamma at baked and generic widths, f32 gamma, many rows per program, bitwise determinism
Remote hardware: gfx90a (MI210), gfx942 (MI300X), gfx950 (MI355X), gfx1201	at c0443e8: FlyDSL gtests 134/134 with all 20 GPU cases launched, none skipped; census 15/15; every shipped object (153 per target) launched by name and passed. The full unit binary's only failures there are 11 TestGfx950AttentionDense* path-error-message tests (develop code, environment-sensitive, unrelated)
Vendored backward kernel, JIT on gfx1151, gfx90a, gfx942, gfx950	11/11 each
Through the public hipDNN API vs PyTorch 2.10 F.rms_norm and autograd	matches on gfx1151, gfx1201, gfx90a, gfx942, gfx950 for every dtype and gamma form; inv_rms within 1e-7; dgamma bitwise repeatable per plan
Baked-width measurement	5 archs × 14 widths × 4 row counts × forward and backward; the rule and the per-width gains are in COVERAGE.md §1 and §2a

SDPA RDNA:

Check	Result
FlyDSL gtests on gfx1151 / working branch on gfx1201	297 passed, 8 skipped (aiw GPU cases need gfx12) / 465/0
diff_upstream.py (FlyDSL, AITER, AOTriton at their pins)	every difference explained by a recorded modification
Through hipDNN	autotuned dnn-benchmarking sweeps on gfx1151, gfx1100 and gfx1201 against the PyTorch reference

SDPA CDNA:

Check	Result
FlyDSL GPU suites, gfx950 (MI355X), gfx942 (MI300X), gfx90a (MI210)	438/0 on the validated working branch, whose code and objects are identical; census; full unit (only the 11 unrelated Gfx950AttentionDense failures)
Autotuned dnn-benchmarking sweeps, every served graph vs PyTorch	0 wrong: gfx950 2345 graphs, gfx942 2299, gfx90a 2173. Speed vs PyTorch (MHA): 1.12× on gfx950, 1.05× on gfx942
Hardware health	every remote run gated on GPU health; one faulty gfx942 GPU identified and excluded

Whole stack:

Check	Result
hip_kernel_provider_tests (full)	1592 passed, 0 failed (rebased on develop c7943b1)
gen_descriptors.py --check	gfx11-generic 309, gfx12-generic 381, gfx950 253, gfx942 255, gfx90a 255
Regeneration of every op on every target	all 1453 objects byte-identical
Configure with HIPKERNELPROVIDER_ENABLE_PACKAGING_PROBES=ON, GPU_TARGETS=gfx942 (as the Linux superbuild CI)	OK; 12 automatic probes (the 10 gfx11/gfx12 member arches, gfx90a and gfx950); probe tests 13/13
Configure: develop default / ingestor ON, no flag / ingestor ON, ENABLE_FLYDSL=OFF	flydsl: not built / FlyDSL active / off, no FlyDSL build rules
Full stack with HIPDNN_ENABLE_SDPA=OFF	builds, 0 warnings; FlyDSL gtests pass

Test tiers. Every tier, pre-merge included, runs all cases but the Full/ ones, among them the drift guards and TestGpuFlydslRmsNormObjects/OnePerForm; the nightly full tier adds Full/...Every (every RMSNorm object per target) and Full/TestGpuFlydslRmsNormTiers. GTest names prefixed Full/ are excluded from the quick, standard and comprehensive tiers by HIP_MLOPS_ENGINE_test_categories.yaml.

ASIC Coverage

Run on gfx1151, gfx1201, gfx90a, gfx942 and gfx950 (above). gfx1200 and gfx1100–1103/1150/1152/1153 were not run as unit suites; they load the same generic objects (gfx12-generic and gfx11-generic), and gfx1100 ran in the RDNA SDPA dnn-benchmarking sweep.

Known caveat

hipDNN's CPU reference and the dnn-benchmarking PyTorch reference drop the sliding window when a graph sets the deprecated causal-mask boolean together with bounds. FlyDSL keeps it, as cuDNN does. Graphs with that combination (e.g. DeepSeek-V4's 128-key window) can show as mismatches against those references. Tracked in #13065.

Not in this PR / follow-ups

• Wiring the FlyDSL engines into the hipDNN integration suite and support claims; UHD heuristics (selection relies on per-engine priority and autotune today).
• RMSNorm toward cuDNN parity: bias and dbias, strided rows, fusions (residual add, FP8/MXFP8 output quantize, zero-centered gamma); forward widths ≤ 2048 baked (upstream's small-N kernel measured 1.5–5× over generic at 128 on four of five archs); backward widths above 16384.
• SDPA: padding/varlen (blocked on hipDNN reference support); dedicated head-dim 96/160/192/224 buckets on CDNA; a d512 prefill object; MLA 192/128 performance vs ASM; AGPR use with the FlyDSL team; upstreaming the kernel modifications to FlyDSL, AITER and AOTriton.
• Known limitation: TheRock's hipkernelprovider artifact has no doc component, so THIRD_PARTY_LICENSES.md would not be packaged by TheRock until one is added. That is a prerequisite of flipping the ingestor flag (ALMIOPEN-2835).

🤖 Generated with Claude Code

[therock-pr-bot]

❌ PR Check — Action Required

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🔎 pre-commit	❌ Fail	Error: Check concluded with cancelled.
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

⚠️ 1 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

• ❌ pre-commit

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[therock-pr-bot]

⏳ Waiting for required checks to start or finish:

• ⏳ pre-commit

[BradPepersAMD]

Superseded by #14111, which carries the whole FlyDSL change (pipeline, RMSNorm forward/backward, RDNA and CDNA SDPA forward) as one commit on a branch named for it, with the kernel objects in DVC rather than in git history. Reviews restart there. Closing in favour of #14111.