Phase 8.5 — apple_gpu story doc + CLAUDE.md update - #16
Merged
Merged
Conversation
Adds two more 2-op fusion patterns to the apple_gpu MSL envelope, both mirroring the Phase 8.4.3 matmul -> softmax structurally. Together with the existing fusions, the runtime now collapses every common 2-op chain in a transformer block: matmul -> softmax (Phase 8.4.3, attention scores -> probs) matmul -> gelu (Phase 8.4.7, MLP block activation) matmul -> rmsnorm[_safe](Phase 8.4.7, transformer pre/post normalization) matmul -> softmax -> matmul (Phase 8.4.5, full attention block) MLIR / runtime - Two new MSL kernels (matmul_gelu_f32, matmul_rmsnorm_f32). matmul_gelu uses tanh-approximation gelu pointwise; matmul_rmsnorm computes RMSNorm with a runtime-passed eps so a single kernel handles both tessera.rmsnorm (eps=1e-5) and tessera.rmsnorm_safe (eps=1e-6). - Two new lowering passes mirroring the matmul_softmax fusion shape. RMSNorm pass uses two concrete subclasses (one per rmsnorm variant) since pre-C++20 doesn't allow float NTTPs. - apple_gpu_runtime_stub.cpp gets matching reference fallbacks. Python - driver.py: _apple_gpu_chain_kind extended to recognize matmul_gelu and matmul_rmsnorm patterns. _backend_artifact_for picks the right symbol. - target_ir.py: two new fp32 MSL source constants + sha256 cache_keys. _apple_gpu_kernel_msl_for_dtype helper extended with matmul_gelu and matmul_rmsnorm keys (f32 only this phase). _apple_gpu_module_fusion_kind detects the new chain shapes by source-set. - target_ir.py: _apple_gpu_module_is_mps_runtime now checks the chain ORDER (first compute op must be matmul). Without this, a softmax -> matmul (suffix-only) chain would incorrectly flip to metal_runtime even though no fusion fires. - runtime.py: generic _apple_gpu_metadata_is_matmul_postlude_chain detector replaces the per-postlude variants. Two new dispatchers (_apple_gpu_dispatch_matmul_gelu, _apple_gpu_dispatch_matmul_rmsnorm) with ctypes wrappers. Loader gate updated. Tests - New lit fixture apple_gpu_mlp_fusions.mlir — verifies both fusions. Uses tessera.rmsnorm_safe (registered op); end-to-end Python test covers the unregistered tessera.rmsnorm spelling. - 5 new unit tests: artifact contract for both kernels, end-to-end numerical correctness for both, ABI shim exposure. - Updated negative-case tests in test_apple_backend_roadmap.py and test_target_ir_contract.py — matmul -> gelu is now fused, so the "unrecognized chain" negative cases switched to softmax -> matmul (suffix-only chain, currently still unrecognized). Verified on Apple Silicon (LLVM/MLIR 21, Metal active): 2020 unit tests passing (2015 + 5 net new MLP-fusion tests); 16/16 Phase 8 lit fixtures passing against the in-tree tessera-opt. Both kernels match numpy reference at rtol=1e-4. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pure documentation phase wrapping up the Phase 8 apple_gpu work. No code
changes — the runtime envelope is exactly what landed in 8.3 → 8.4.7.
New documentation
- docs/apple_gpu_overview.md (~6 pages) — architecture story:
* Big picture diagram (Graph IR -> Target IR -> Metal device)
* Three-layer dispatch decision (compile-time gate, chain detection,
runtime dtype dispatch)
* Pipeline ordering (longest fusion first)
* Anatomy of a kernel emission (MSL source, C symbol, lowering pass,
Python dispatcher)
* Step-by-step "how to add a new kernel" + "how to add a new fusion"
* Compile-time vs runtime dispatch nuance (@jit is type-polymorphic)
* MetalDeviceContext + kernel cache mechanics
* Constraints summary
* Files-at-a-glance table
- docs/apple_gpu_kernel_inventory.md — symbol-level reference:
* Full table of all 26 runtime C ABI symbols across single-op,
fused 2-op, and fused 3-op kernels
* Phase landed, backend (MPS/MSL/conversion), constraints per symbol
* ABI summary (i64 ptrs, i32 dims, f32 scalars, uint16_t for f16/bf16)
* 3x9 dtype coverage matrix
* Test surface tally (16 lit fixtures, ~80 unit tests, benchmark harness)
CLAUDE.md updates
- Header status line: now reflects Apple operational + summary of the 26
symbols + pointer to the new overview doc
- Phase Completion Status table: Phase 8 marked 🟢 Apple operational
- Phase 8 section in body: every sub-phase 8.3 -> 8.4.7 has a one-paragraph
"landed" entry naming the kernel(s), the design choice, and the gate
- Pipelines table: adds tessera-lower-to-apple_gpu-runtime with its full
pass composition order
- Apple backend entry in Key Source Locations: now points at the new
overview + inventory docs
- Key Reference Files table: adds apple_gpu_overview.md, kernel_inventory,
and benchmark harness entries
- Last-updated footer: refreshed test counts (2020 unit + 16/16 lit) and
full kernel inventory summary
Verified: 93/93 focused unit tests still passing (pure docs change).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pure documentation phase wrapping up the Phase 8 apple_gpu work. No source-code changes beyond
docs/andCLAUDE.md— the runtime envelope is exactly what landed in 8.3 → 8.4.7. This PR makes the architecture and the kernel inventory legible to anyone joining the project.What changed
docs/apple_gpu_overview.md— architecture story (big picture, three-layer dispatch decision, pipeline ordering, anatomy of a kernel emission, step-by-step recipes for adding new kernels/fusions, MetalDeviceContext + cache mechanics, files-at-a-glance reference)docs/apple_gpu_kernel_inventory.md— symbol-level reference: 26 runtime C ABI symbols across single-op + fused 2-op + fused 3-op kernels, with per-symbol backend / phase / constraints, a 9×3 dtype coverage matrix, and ABI summarytessera-lower-to-apple_gpu-runtimewith full pass composition; Key Reference Files adds the new docs + benchmark harness; footer test counts refreshed (2,020 unit + 16/16 lit)Test plan
docs/andCLAUDE.mdPR stack — Phase 8 Apple work, complete
Once this stack lands,
@jit(target="apple_gpu")covers the standard transformer building blocks end-to-end on Apple Silicon, with documentation that lets the next contributor pick up the thread.🤖 Generated with Claude Code