docs(audit): CAKE assessment + Phase 1/2 scoping for the Tile sync surface

[gstoner]

Assesses CAKE: Compiler–Agent Co-Design for Frontier Kernel Evolution (arXiv:2608.12629, NVIDIA/CMU) and its FlashInfer PR #4262 artifact against the tree at f96695f. Adds docs/audit/compiler/compiler_enhancement.md in the house assessment style (alongside TILESIGHT/TILERT/SPARDA), reference role — not a status surface (Decision #26).

Docs only. No code paths touched.

Why it matters

CAKE is the first controlled A/B on IR design as agent ergonomics: same model, scaffold, oracle, token budget, GPU, and task — the only variable is the representation the agent edits. That is a question Tessera has answered by assertion since Decision #28.

Statistical audit (§2)

Because n=3 and the paper reports median [min,max], the reported triple is the complete sample, so exact permutation tests apply.

Metric	CAKE IR	Direct CUDA/PTX	Exact one-sided p
Best speedup @ 80M	{1.041, 1.144, 1.205}	{0.852, 0.928, 1.151}	0.200
Active evolve time (h)	{1.02, 1.89, 2.33}	{3.59, 3.73, 4.34}	0.050
Plateau by 80M	3/3	0/3	0.050 (Fisher)

The headline 1.144× vs 0.928× is not significant — the arms overlap; the control's best run (1.151) beats the treatment median (1.144). The design's Bonferroni family-wise floor is 0.15, so no effect size could have made it significant. What is separated is convergence: half the wall-clock, complete separation, 3/3 vs 0/3.

This reshapes the plan. The paper's own curve crosses baseline only at 55M tokens — an order of magnitude above a realistic per-kernel budget here — so an authoring surface must be justified on time-to-plateau and convergence, never on the speedup number.

In-tree findings (§3, measured against f96695f)

• F1 — the typing hole is on the sync/memory surface. 55 of 74 Tile op defs carry AnyType in their arguments; only 8 use a declared Tile_* type. Sharpest defect: tile.mbarrier.wait cannot consume the !tile.mbarrier_token that tile.mbarrier.arrive_expect_tx produces, so the arrive→wait edge is not expressible in the type system — while tile.try_wait, four lines away in the same file, already has the correct form. Also: tile.pipeline_advance need not consume a pipeline_state, though Tile_PipelineStateType's own summary claims SSA def-use ownership (Decision Apple GPU MLA: native f16/bf16 for the absorbed decode kernel #29 stated and violated in one file).
• F2 — WarpSpecLegalityPass is type-blind by construction. It matches on name substrings and attributes and never reads an operand type (Decision Apple GPU MLA: native f16/bf16 for the two secondary decode kernels #30, one level below the EffectLattice instance CLAUDE.md already indicts). Tightening ODS buys the verifier nothing on its own — which is why Phase 1 is one phase, not two. Separately, the legality fixtures run under --allow-unregistered-dialect while the fixtures exercising the registered vocabulary don't run the legality passes: the registered path and the verified path are disjoint sets.
• F3 — no author can state a schedule. tessera.kernel is a shard decorator; tile_ir.py is a lowering; barriers and TMEM are synthesized per-arch by target_ir.py from a tile.matmul.

Relationship to W1.1 (§3.5)

Scoping changed once W1_1_TYPING_DESIGN.md was read — steps 1, 2, 4 landed, 3 partial. Nothing here duplicates it. W1.1 owns !tile.fragment / tile.mma / producer migration / Target IR dialects; this doc owns tile.mbarrier.*, tile.tma.*, tile.tmem.*, tile.pipeline_*. tile.tcgen05.mma is the seam and is deferred until after W1.1 step 5 so it doesn't fork the fragment contract.

Phase 1 also inherits W1.1's four method lessons rather than rediscovering them — notably §4.2 (both backends materialized a zero constant for the MMA's C operand, so a lowering fixture passed a silently wrong GEMM). Hence the exit gates require numerical execution on gfx1151, not a lowering fixture.

What's scoped

• Phase 1 — one typed, verified sync surface: hardest-case-first experiments, 8 ordered ODS rows, verifier rewrite to derive from types and def-use, fixture migration off --allow-unregistered-dialect, 6 exit gates.
• Phase 2 — roles and producer/consumer sets on the barrier declaration (the smallest load-bearing idea in the paper), consuming wave_specialization.py instead of leaving it a declaration with no consumer.
• Phases 3–6 sketched behind the gates Phase 1/2 must produce.
• §8 — the capability layer beyond CAKE, led by schedule-level autodiff (CAKE hand-authors FA4 BWD as 514 lines).

Also corrects an earlier arbiter proposal: emit/candidate.py:365 does min(cands, key=measure), so measure is the terminal selector — wiring a predictor into it leaves regret unbounded. The two-stage form is gated on first instrumenting the rejection fraction p, which CAKE never reports.

Deliberately unresolved

§5.1's experiments haven't been run (they're Phase 1's first work item because their outcome could invalidate ODS rows 6–7), and Phase 2 gate 2 — whether one role model verifies both Hopper producer/consumer and CDNA ping-pong without a target branch — is flagged as a genuine open question. §9 records the assessment's own limits, including that the AnyType count is ODS-derived and therefore an upper bound on the hole.

Verification

• bash scripts/check_generated_docs.sh → ok: 25 generated doc(s) in sync
• Index rows added to docs/audit/compiler/README.md and docs/audit/README.md per Decision Apple GPU MLA: paged-cache decoder (production-serving wiring) #26

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d56d5ba832

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".