J: roofline attainment — % of peak as the hot-path bar (W7) - #313
Merged
Merged
Conversation
The E2 latency ratchet is a relative bar (did it get slower?). J adds the absolute bar the plan asks for: % of peak. - benchmarks/roofline.py: a grounded per-device peak table (rocm:gfx1151 = 29.7 TF fp32 / 59.4 TF fp16 / 256 GB/s, each with a `source` string deriving it from rocminfo CU/SIMD/clock + documented RDNA3 rates — Decision #27, auditable), FLOP/byte models (matmul 2*MNK, flash_attn 4*B*H*S^2*D), and achieved_tflops / pct_peak / evaluate_attainment. - The committed gfx1151 ratchet rows gain pct_peak + achieved_tflops + an attainment_floor (= pct_peak / margin, symmetric with the latency cap), computed from the EXISTING medians (no re-timing — ratchet caps unchanged). - perf_gate --attainment: gates a row that regresses below its floor (the absolute analog of the latency ratchet). record_hot_path_baseline annotates future baselines automatically. Honest scope: the ratchet median is end-to-end wall-clock (H2D/launch/D2H + tessera-opt shell-out), so pct_peak is an END-TO-END attainment — a lower bound on kernel efficiency. The current gfx1151 lanes sit at ~0.3-2.9%, so the metric's immediate value is making the headroom visible and giving it a ratchet floor. Proof: test_roofline_attainment.py (12) — FLOP/peak/attainment model, gate pass/fail/coverage, and the committed baseline self-passes `perf_gate --attainment`. ruff clean; drift gate green. Still open (J): kernel-isolated attainment (strip host overhead); NV sm_120 + Apple peak rows; floors that ratchet upward as lanes optimize. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3c37aabaf9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…fline import - P1: measured ratchet-report rows carry latency_ms (as evaluate_ratchet reads), not median_ms — evaluate_attainment fell back to 0.0, so pct_peak returned None and every measured row false-failed on coverage. Read latency_ms with a median_ms fallback (baseline/self-check rows use median_ms). - P2: `from roofline import` broke under `python -m benchmarks.perf_gate` and `from benchmarks import perf_gate` (package context — roofline not on sys.path). Try `from benchmarks.roofline import` first, fall back to the script-dir import. Tests: a measured-row latency_ms gate case (pass at baseline latency, fail 10x slower) and a package-import case (from benchmarks import perf_gate + main --attainment on a latency_ms report). 14 pass; ruff clean; both script and `-m benchmarks.perf_gate --attainment` invocations verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workstream J / W7 — absolute roofline attainment. The E2 latency ratchet is a relative bar (did a hot path get slower?). J adds the absolute bar the plan asks for: % of peak.
benchmarks/roofline.pyrocm:gfx1151= 29.7 TF fp32 / 59.4 TF fp16 / 256 GB/s, each with asourcestring deriving it fromrocminfo(40 CU × 2 SIMD32 × 32 lanes @ 2.9 GHz × 2 FMA × 2 dual-issue; fp16 WMMA 2×; LPDDR5X-8000 256-bit) — auditable/correctable in one place (Decision Apple GPU MLA: multi-sequence block-paged cache (vLLM-style paged attention) #27), not a guessed marketing number.2·M·N·K, flash_attn4·B·H·S²·D.achieved_tflops/pct_peak/evaluate_attainment.Gate + baseline
pct_peak+achieved_tflops+ anattainment_floor(=pct_peak / margin, symmetric with the latency cap), computed from the existing medians — no re-timing, so the latency caps are unchanged.perf_gate --attainmentgates a row that regresses below its floor (the absolute analog of the latency ratchet).record_hot_path_baselineannotates future baselines automatically.Honest scope
The ratchet median is end-to-end wall-clock (H2D/launch/D2H +
tessera-optshell-out), sopct_peakis an end-to-end attainment — a lower bound on kernel efficiency. The current gfx1151 lanes sit at ~0.3–2.9% of peak, so the metric's immediate value is making that headroom visible and ratchet-able (2048³ matmul is the best at 2.9% — compute dominates overhead at scale).Proof
test_roofline_attainment.py(12) — FLOP/peak/attainment model, gate pass/fail/coverage, and the committed baseline self-passesperf_gate --attainment. ruff clean; generated-doc drift gate green.Still open (J)
Kernel-isolated attainment (strip host overhead);
[NV]sm_120 + Apple peak rows; floors that ratchet upward as the lanes optimize.🤖 Generated with Claude Code