Repository navigation
feat(cake_gdn_det): sm_107a (R200) libraries with grad - #7974
Merged
Merged
Conversation
…ries Adds sm_107a to ARCHES, the (10, 7) capability and every record, the provisional sm_107a prep_tma window (3 CTAs per SM) and dhu_tc2 selection constants, and the suite rows at 216 SMs. Co-Authored-By: Claude <noreply@anthropic.com>
Exported render: 87 libraries (29 programs on sm_100a, sm_103a and sm_107a; sm_107a needs CUDA 13.4). The program names follow the moved kernel sources; the sm_100a and sm_103a machine code is unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
The sm_107a dhu_tc2 selection constants and prep_tma window, measured on a 216-SM R200: t_lat 1.708, t_full 2.357, t_unit 1.329 and c_dvl 1.696 us; 3 CTAs per SM and window 1.0 (no plain-prep band: prep_tma was faster in every swept cell up to 10,240 CTAs). The suite's sm_107a r01 T_tc follows (113.3), and the prep-rule tests accept a window without a band. The sm_100a and sm_103a entries are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
The sm_100a and sm_103a prep-rule assertions are again those of the SM107 registry commit (window > 1, a plain-prep band); the no-band branch runs for sm_107a only and asserts its window of 1.0. The sm_107a window note is corrected: on the R200 prep sweep (128-16,384 CTAs, BF16 and FP32 beta) the window rule found no plain-prep band in 1.0-6.0 x slots, so 1.0 (no band) was chosen. prep_tma was faster in 78 of 80 cells; with BF16 beta it tied at 10,240 CTAs and lost 1.6 % at 12,288 CTAs (19 x slots, beyond every candidate band). Co-Authored-By: Claude <noreply@anthropic.com>
The generator now folds additive-zero terms in index arithmetic (for example `(x + 0)` becomes `x`). Five kernel sources, each built for sm_100a, sm_103a and sm_107a, lose those terms; build.json records their new hashes and the generator revision. The SASS and resource usage of all 87 libraries are unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
yyihuang
requested review from
Anerudhan,
Jingfan-Sun,
YangXu1990uiuc,
aleozlx,
bkryu,
flashinfer-bot and
yzh119
as code owners
October 30, 2026 05:26
flashinfer-bot
added
the
op: linear attention
Contributor
|
No actionable comments were generated in the recent review.
 |
| Layer / File(s) | Summary |
|---|---|
Register SM107 and configure dispatch flashinfer/experimental/cake_gdn_det_train/cake_jit.py, flashinfer/experimental/cake_gdn_det_train/cake_backend.py, flashinfer/experimental/cake_gdn_det_train/__init__.py, flashinfer/experimental/cake_gdn_det_train/README.md, tests/experimental/test_cake_gdn_det_train.py |
The architecture registry and package descriptions add SM107. The prep_tma selector uses 3 CTAs per SM and a 1.0 window for SM107; the README states that all tested SM107 grids use TMA prep and that CUDA 13.4 or newer is required. Tests add SM107 device properties, segmented adjoint-plan expectations, and prep-dispatch checks. |
Update generated kernels for SM107 flashinfer/experimental/cake_gdn_det_train/csrc/*_kernel.cu |
Generated kernel identifiers are updated. Several kernels use 0x7FFF shared-memory descriptor masks for architecture 1070 while retaining 0x3FFF on other architectures. Some generated input staging and TMEM load offsets are also updated; many other edits rename local variables without changing the summarized calculations or data flow. |
Retarget generated bindings flashinfer/experimental/cake_gdn_det_train/csrc/*_binding.cu |
Generated bindings use the updated kernel identifiers for declarations, shared-memory attributes, launches, and run registrations. The summarized binding validation and launch configurations remain unchanged. |
Priority:
Estimated code review effort: 3 (Moderate) | ~30 minutes
Change: Feature
Suggested reviewers: yzh119
Merge Risk: 
f6bfe
-
SM107 support adds no identified merge-blocking risk; existing architecture builds remain isolated.
Pre-merge checks: 
-
Check name Status Explanation Resolution Description check The description gives a detailed and relevant summary, implementation scope, compatibility requirement, and benchmark results. However, it omits required template sections and declarations, including … Add the missing template sections. Provide the related issue, complete the pre-commit and test checklist, state whether full-scope CI is required and why, and complete the Experimental Track section with the tracking issue, owner, reason, g… Docstring Coverage Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold. -
Check name Status Explanation Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Title check The title "feat(cake_gdn_det): sm_107a (R200) libraries with grad" names the main change: adding sm_107a (R200) kernel libraries to the cake_gdn_det_train backend. The changeset adds sm_107a to ARCHES… Full details: Title check
-
Explanation
The title "feat(cake_gdn_det): sm_107a (R200) libraries with grad" names the main change: adding sm_107a (R200) kernel libraries to the cake_gdn_det_train backend. The changeset adds sm_107a to ARCHES and ARCH_BY_CAPABILITY, renames regenerated kernel binaries, adds sm_107a prep and tests, and updates the README. The title is specific and concise. The term "with grad" refers to the backward (gradient) training path and is a valid aspect of the change, though it is less precise than the architecture addition.
-
Full details: Description check
-
Explanation
The description gives a detailed and relevant summary, implementation scope, compatibility requirement, and benchmark results. However, it omits required template sections and declarations, including related issues, checklist status, test status, the experimental tracking issue and graduation details, and the experimental test targets.
Resolution
Add the missing template sections. Provide the related issue, complete the pre-commit and test checklist, state whether full-scope CI is required and why, and complete the Experimental Track section with the tracking issue, owner, reason, graduation plan, intended-hardware validation, and an experimental-tests block listing existing targets under tests/experimental/. Include a runnable example if required by the experimental policy.
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 47 functions across 50 files. (9 skipped: 1 unsupported, 8 over the file limit.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
-
Finishing Touches: 💡 1
-
🛠️ Fix failing CI checks: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
yzh119
approved these changes
Oct 30, 2026
Collaborator
Author
|
@flashinfer-bot run tests/experimental/test_cake_gdn_det_train.py |
yyihuang
changed the title
yyihuang
enabled auto-merge (squash)
October 30, 2026 05:44
Collaborator
Author
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds an sm_107a (R200) build of
flashinfer.experimental.cake_gdn_det_train. That package, added in #7592, is the deterministic Cake backend for chunked Gated DeltaNet (GDN, scalar gate) training. It has these variants:All 29 kernel libraries in the package now also ship for sm_107a, so the package has 87 libraries across sm_100a, sm_103a and sm_107a.
(10, 7)devices select the sm_107a libraries, which need CUDA 13.4 or newer. The sm_100a and sm_103a libraries are unchanged.No earlier Cake version exists for sm_107a, so the baseline is FLA 0.5.2 (
chunk_gated_delta_rule, Triton). For each column, the FLA value is the faster of two FLA forms: q/k repeated to the value heads, or native grouped value heads.NVIDIA R200 (sm_107a, 216 SMs), CUDA 13.4, BF16 inputs. Median latency in ms with L2 flushed, both backends measured on the same GPU. Speedup = FLA / Cake, with a 95 % paired bootstrap CI.
The other Cake GDN backend in FlashInfer is
cake_gdn_train, a separate training backend with context parallelism;cake_gdn_det_trainnow ships for sm_100a (B200 / GB200), sm_103a (B300 / GB300) and sm_107a (R200).Summary by CodeRabbit