Skip to content

Add gfx1250/MI450 target reference and AMD kernel-compiler survey - #476

Merged
gstoner merged 3 commits into
mainfrom
claude/amd-kernel-compiler-survey
Jul 29, 2026
Merged

gstoner merged 3 commits into
mainfrom
claude/amd-kernel-compiler-survey

Conversation

@gstoner

@gstoner gstoner commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Depends on #475 — four links in this PR point at docs/audit/compiler/TILESIGHT_ASSESSMENT.md, which lands there. Merge #475 first, or expect those four to dangle until it does. The dependency is substantive, not accidental: two of this PR's findings are direct answers to the mock-cost-model gap #475 records.

Motivation

Two questions drove this: what is gfx1250 (MI450) actually, and what have AMD's own kernel compilers already solved that we are about to re-derive?

The tree's gfx1250 entries were explicitly marked PROVISIONAL — placeholders copied from RDNA because no ISA had been consulted. And we had never read AMD's kernel compilers (as opposed to kernel libraries) at source level, despite building the same kind of thing.

Documentation only. No code changes.

Technical Details

backend/rocm/GFX1250_MI450_COMPILER_REFERENCE.md

Target facts derived from LLVM tablegen, ROCm library source, and the on-machine assembler rather than vendor prose. Every claim carries a provenance marker ([V] source / [A] locally assembled / [S] AMD prose / [I] inference), §0 gives the resolution order when sources disagree — they do — and §10 gives commands to re-derive everything without AMD hardware.

Findings that correct or extend current assumptions:

Finding Current tree
FeatureClusters is on gfx1250/1251, not gfx950 rocm_target.py asserts the exact opposite. Latent — supports_cluster_mode() has no codegen consumer — but inverted.
LDS 327680 B, 1024 addressable VGPRs 65536 / 256, both marked PROVISIONAL. The VGPR figure matters: rocm_tiling.py treats the register budget as the dominant tiling lever.
gfx1250 vs gfx1251 are complementary SKUs — same WMMA instruction 4x apart, FP64 6x apart the other way _GFX1250_CLASS_ARCHES treats them as one class. Correct for ABI, wrong for any cost model.
Async/tensor waits are never auto-inserted by LLVM — they come from llvm.amdgcn.{asyncmark,wait.asyncmark}, and a too-large index yields no wait, not a conservative one ROCM_WaitTokenOp has no immediate operand, so it can only express a full drain.
AMD clusters are multicast-into-own-LDS + a barrier, shader-engine scoped, with no distributed shared address space An NVIDIA CGA kernel that reads a peer CTA's shared memory will not lower.

Also covers the WMMA co-execution hazard (LLVM's exact wait-state table — going through LLVM handles it, hand-emitted asm does not), TDM's descriptor contract, device-initiated SDMA, and the UALink/IFoE fabric.

compiler/AMD_KERNEL_COMPILER_SURVEY.md

StinkyTofu, rocRoller, Composable Kernel and hipBLASLt read for transferable architecture. §6 ranks 21 items take/skip. The two most useful are structural:

  • rocRoller's observer scheduling (peek/modify/observe) gives a cost query before commitment — the primitive a measured arbiter needs and that a pass pipeline cannot express.
  • CK derives vector width, access count and traversal order from a distribution encoding, rather than having a kernel author write them. Of four concerns, only two are authored.

It also records two static, device-free quality metrics found in production AMD code: a step-distance locality histogram, and a bank-conflict analyzer that computes N-way conflict from a descriptor alone. Both bear directly on the mock-cost-model finding in #475 — and the second is unit-testable ("this descriptor is conflict-free for a warp-wide access on N banks") with no device.

Convergence worth noting: StinkyTofu's Logical-IR → Asm-IR split is Decision #19 arrived at independently, and its "new arch = a directory of .def files, no C++ edits" is Decision #28's plugin seam done as data — which also gives us a falsifiable acceptance test for our own seam.

APPLE_AUDIT.md — backlog item 8

The follow-on the survey implies. Apple is the only backend that executes broadly enough to say whether such a metric predicts anything, so the action is to calibrate one against recorded latency, not to add one. A metric that cannot rank Apple kernels should not be trusted to rank ROCm or NVIDIA kernels we cannot measure.

Test Plan

  • tests/unit/test_audit_docs.py — frontmatter / lifecycle gate for authored audit docs
  • scripts/check_generated_docs.sh — generated-doc drift gate
  • Manual: all relative links resolve; all internal §N.M cross-references resolve; all referenced in-tree source paths exist

Test Result

Both documents are audit_role: reference and state explicitly that they are not status surfaces. No counts are copied into prose (Decision #26).

Risk level

None. Documentation only — no code, no C++, no build files, no generated docs regenerated.

🤖 Generated with Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 84076eb11f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/audit/backend/apple/APPLE_AUDIT.md Outdated
Comment thread docs/audit/compiler/AMD_KERNEL_COMPILER_SURVEY.md
Comment thread docs/audit/compiler/AMD_KERNEL_COMPILER_SURVEY.md
gstoner and others added 3 commits July 29, 2026 08:55
Two primary-source reference documents from a source-level read of AMD's
ROCm repositories, plus the Apple follow-on they imply. Documentation only;
no code changes.

backend/rocm/GFX1250_MI450_COMPILER_REFERENCE.md — target facts for gfx1250
(MI450), derived from LLVM tablegen, ROCm library source, and the on-machine
assembler rather than from vendor prose. Every claim carries a provenance
marker and section 10 gives commands to re-derive it without AMD hardware.

Findings that correct or extend what the tree currently assumes:

  - Our cluster_mode flag is inverted. FeatureClusters is on gfx1250/1251
    and not on gfx950; rocm_target.py asserts the opposite. Latent today —
    supports_cluster_mode() has no codegen consumer — but wrong.
  - gfx1250 LDS is 327680 bytes and it has 1024 addressable VGPRs; our
    entries carry 65536 and 256, marked PROVISIONAL. The VGPR figure matters
    because rocm_tiling.py treats the register budget as the dominant lever.
  - gfx1250 and gfx1251 are complementary SKUs, not revisions: the same WMMA
    instruction runs 4x apart, and FP64 runs 6x apart the other way.
    _GFX1250_CLASS_ARCHES is right for ABI and wrong for any cost model.
  - Async and tensor waits are never inserted automatically by LLVM. They
    must come from llvm.amdgcn.{asyncmark,wait.asyncmark}, and a too-large
    index yields no wait at all rather than a conservative one. Our
    ROCM_WaitTokenOp has no immediate operand, so it can only express a full
    drain.
  - AMD clusters are multicast-into-own-LDS plus a barrier, scoped to a
    shader engine. There is no distributed shared address space, so an
    NVIDIA CGA kernel that reads a peer CTA's shared memory will not lower.

compiler/AMD_KERNEL_COMPILER_SURVEY.md — StinkyTofu, rocRoller, Composable
Kernel and hipBLASLt read for transferable architecture. Section 6 ranks 21
items take/skip. The two most useful are structural: rocRoller's observer
scheduling gives a cost query before commitment, and CK derives vector width,
access count and traversal order from a distribution encoding rather than
having a kernel author write them.

It also records two static, device-free quality metrics found in production
AMD code — a step-distance locality histogram and a bank-conflict analyzer
that computes N-way conflict from a descriptor alone. Both bear directly on
the mock-cost-model finding in TILESIGHT_ASSESSMENT.md.

APPLE_AUDIT.md gains backlog item 8 as the follow-on: Apple is the only
backend that executes broadly enough to say whether such a metric predicts
anything, so the action is to calibrate one against recorded latency rather
than to add one. A metric that cannot rank Apple kernels should not be
trusted to rank kernels we cannot measure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Extends the two survey documents with six more ROCm projects. Documentation
only.

Compiler survey gains section 4.7 on rocisa, TensileLite's nanobind assembly
generator — the same Python-driving-C++ shape we have. Three findings worth
copying: IR nodes carry a mandatory clone() deep-copy contract; exporting a
vector to Python is a copy, so elements are mutable through their shared_ptr
but cannot be assigned or replaced; and import raises if any C++ source is
newer than the built extension. That last one is added to the take list — we
lost time this session to a tessera-opt binary that silently did not match
its sources.

Patterns doc gains four project briefs and a rocWMMA re-read:

rocFFT has the best cache design in the ecosystem. The kernel name is the
cache key, with every differentiating parameter encoded into it, so profiler
output and cache identity are the same string and the cache needs no schema
update when a new parameter appears. Three further key fields guard
staleness — architecture, HIP version, and generator version. A read-only
system cache ships with the library alongside a read-write user cache, the
shipped one populated at build time by a helper that shares the generator but
is not installed. AOT and JIT are one path with a policy knob rather than two
lanes. Also records that hipRTC holds process-wide locks, so parallel
compilation needs a helper process.

rocPRIM turns tuning output into generated headers, and its fallback_config
is a typed fallback ladder: an untuned type inherits the config of a
representative matched on size range and floating-pointness. That is dtype
bucketing, the same move Decision #28 makes for shapes.

rocRAND is the one with a direct bearing on us. Under dynamic ordering it
picks launch geometry per device, and AMD states plainly that the number of
generators and the sequence of generated numbers can vary as a result. So
reproducibility versus performance is a named opt-in mode, not an emergent
property. Worth confirming the same holds for Decision #18: if a tuned launch
configuration ever fed an RNG offset scheme, autotuning would silently change
numerical output.

rocALUTION is included as a contrast, not a pattern. It selects execution
location at run time via RTTI and silently migrates an object back to the
host when a routine is unavailable on the accelerator. That is the opposite
of Decision #21, which requires a diagnostic naming the op and target. Both
are defensible for their audience; the contrast is worth recording because
silent host migration is how a performance cliff hides.

rocWMMA re-read adds that collaborative fragments are a movement concept and
are explicitly unsupported in MMA functions, that partial and oversized tiles
became the library's problem in 2.0.0, and that the wavefront-centric
contract is undefined behaviour rather than a hint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n correctly

P2 — the ds_read_b128 phase table listed only 3 of 4 phases, covering 48 of 64
lanes. An analyzer built from it would leave lanes 36-43, 48-51 and 60-63
unmodelled and could accept a layout that conflicts on exactly those lanes.
Phase 3 is T36-39, T40-43, T48-51, T60-63; the structure is phase 1 = phase 0 +
32 and phase 3 = phase 2 + 32, which doubles as the completeness check (four
disjoint sets covering 0-63 exactly once).

P1 — APPLE_AUDIT.md scoped the hardware-free cost-model calibration to Apple
alone, asserting that ROCm and NVIDIA kernels "cannot be measured". Both claims
were false, written from the pre-bring-up framing CLAUDE.md itself retired:
NVIDIA has a committed, consumed, device-keyed sm_120 autotune corpus and ROCm
has measured gfx1151 retune/ratchet evidence. Excluding them would also have
weakened the calibration - a score fitted on one architecture reproduces the
single-arch overfit the TileSight assessment records for NeuSight.

All four architecture queues now record a state under sync key
COSTMODEL-CALIB-2026-07-29: NVIDIA-CALIB-1 (shape depth, cheapest to land -
analysis over already-recorded data), ROCM-CALIB-1 (the metric's home ground,
extracted from production AMD code; wave32 constants must be re-derived),
APPLE-CALIB-1 (op breadth, with the conflict metric possibly not applicable to
Metal threadgroup memory), X86-CALIB-1 (split verdict: locality applies, the
bank-conflict analyzer does not - no software-managed scratchpad or wave
phases).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gstoner
gstoner force-pushed the claude/amd-kernel-compiler-survey branch from 2f2296b to bec0c0d Compare July 29, 2026 14:58
@gstoner
gstoner merged commit 82a905e into main Jul 29, 2026
16 checks passed
@gstoner
gstoner deleted the claude/amd-kernel-compiler-survey branch July 29, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant