Skip to content

perf(cake_rmsnorm_silu_gate_train): L2-prefetching extra-long forward on SM107 - #8701

Merged
yyihuang merged 1 commit into
flashinfer-ai:mainfrom
yyihuang:cake-rmsnorm-silu-gate-sm107-xlong-l2-prefetch
Nov 4, 2026
Merged

perf(cake_rmsnorm_silu_gate_train): L2-prefetching extra-long forward on SM107#8701
yyihuang merged 1 commit into
flashinfer-ai:mainfrom
yyihuang:cake-rmsnorm-silu-gate-sm107-xlong-l2-prefetch

Conversation

@yyihuang

@yyihuang yyihuang commented Nov 4, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Collaborator

This PR changes which module runs the sm_107a forward in the non-deterministic package of Cake's SiLU-gated RMSNorm training kernel (flashinfer.cake_rmsnorm_silu_gate_train with deterministic=False). From 1740 rows per SM, the sm_107a forward now runs cake_rmsnorm_silu_gate_train_7f014401d5cc855ab883. This is the L2-prefetching 256-thread forward with the packed gate, which PR #8673 added for sm_103a from 3000 rows per SM. On sm_107a, the new route takes over the whole range the long forward used to serve, so the long forward no longer runs on that architecture. All eight changed shapes in Table 1 are at or above 1740 rows per SM on the 212-SM R200.

The previous Cake baseline below is the long forward, cake_rmsnorm_silu_gate_train_7b7fac905aff1ddaa258, which ran these inputs on sm_107a before this PR. This PR adds no kernel source. It builds the existing module for sm_107a and drops the long forward's sm_107a build, which no sm_107a input reaches any more. The long forward still ships on sm_100a and sm_103a. Every other module and route runs the same code as before, including the backward, every sm_107a forward below 1740 rows per SM, and every sm_100a and sm_103a route.

The PR is one commit on FlashInfer main after PR #8673. The A/B compared two FlashInfer trees. In the first, the long forward serves sm_107a from 1740 rows per SM. The second is the same tree plus this PR's sm_107a route. On sm_107a, this PR's routes, library entries and kernel sources are identical to those of the measured second tree.

A speedup is baseline us / new us, so a value above 1 means this PR is faster. The table gives ranges (min-max) over the eight changed shapes of Table 1 on R200 (VR200 node; nvidia-smi name NVIDIA Graphics Device), sm_107a, 212 SMs, CUDA 13.4.

op det Cake / new strongest non-Cake / new previous Cake / new (A/B)
fwd 1.096-1.201 1.054-1.116 1.019-1.059
fwdg 1.100-1.201 1.055-1.114 1.016-1.060
bwdg 1.028-1.109 1.038-1.107 0.9998-1.0003
ckptg 1.074-1.119 1.040-1.086 1.015-1.028
stepg_f32 1.077-1.125 (A/B) - 1.016-1.028
fwd + bwd 1.060-1.114 (A/B) - 1.006-1.020

In the pre-registered A/B, the code changes in 32 cells (fwd, fwdg, ckptg and stepg_f32 at the eight shapes), and new is faster than the previous Cake route in all 32. bwdg and bwdg_f32 run the same code in both trees. The strongest non-Cake kernel is not faster than new on any row of Table 1. The deterministic Cake kernel is faster than new in one cell: 1031x48 fwdg in Table 2 (det / new 0.996). That shape is below 1740 rows per SM, and this PR does not change its forward. Every cell below 1x against the previous Cake route is in a cell that runs the same code in both trees. "Cells below 1x" lists each one.

The kernel has two other Cake variants: the deterministic package flashinfer.cake_rmsnorm_silu_gate_det_train (the same API with deterministic=True) and the FP32 weight-gradient output of the non-deterministic package; both packages ship on sm_100a, sm_103a and sm_107a, and this PR changes only the non-deterministic package's forward on sm_107a from 1740 rows per SM.

How the tables were measured

  • Latency is CUPTI kernel time with a cold L2, taken with FlashInfer's GPU timer in flashinfer.testing. Inputs are BF16, and all times are in us.
  • Table 1: the deterministic Cake kernel, the strongest non-Cake kernel and new were measured in the same process. Each value is the median of the per-round medians over 10 rounds.
  • A/B values come from the pre-registered interleaved A/B, which ran 16 sequences and 160 processes. They are the previous Cake and new (A/B) columns of Table 1 and all of Table 2, and each is a median.
    • stepg_f32 has no same-process measurement, so its deterministic Cake values also come from the A/B.
    • The two new columns of Table 1 (new us and new us (A/B)) are two separate measurements of the same code.
  • Strongest non-Cake kernel per row: the fastest of the following, with torch 2.14.1+cu130 and Triton 3.8.0. On every measured row of Table 1, the fastest was FLA 0.6.0 rms_norm_gated.
    • FLA 0.6.0 rms_norm_gated (fla.modules.fused_norm_gate).
    • mamba_ssm v2.3.2 rmsnorm_fn (layernorm_gated).
    • torch.compile in default and max-autotune-no-cudagraphs modes, best of 5 fresh compiles each.
    • The inductor-generated persistent-reduction Triton kernel of the FP32 formula, launched directly (fwd and fwdg rows only).
  • Deterministic Cake: flashinfer.cake_rmsnorm_silu_gate_det_train, unchanged by this PR.
  • Operations:
    • fwd is one eager forward call, and fwdg is the forward replayed from a CUDA graph.
    • bwdg is the backward replayed from a CUDA graph, and bwdg_f32 is the same backward with an FP32 dw.
    • ckptg is the checkpointed step (2 x forward + backward) in one CUDA graph, and stepg_f32 is the same step with an FP32 dw.
    • fwd + bwd is fwdg + bwdg, derived.
  • In Table 2, the forward at the first eight shapes runs the new module. 1031x48, 3534x48, 6000x48 and 7684x48 are below 1740 rows per SM, so they run the same code in both trees.

Cells below 1x

  • Against the strongest non-Cake kernel (Table 1): none.
  • Against the deterministic Cake kernel (Tables 1 and 2): one cell. Table 2, 1031x48 fwdg: 0.996 (det 7.584 us, new 7.616 us). This shape runs the same code in both trees.
  • Against the previous Cake route (A/B): only in cells that run the same code in both trees.
    • Table 1, bwdg: 7685x48 0.9999, 12296x48 0.9998
    • Table 2, changed shapes (backward cells): 7685x48 bwdg 0.9999, 9222x48 bwdg_f32 0.9999, 16384x32 bwdg_f32 0.9999, 12296x48 bwdg 0.9998
    • Table 2, same-code shapes: 1031x48 ckptg 0.9997, 3534x48 bwdg 0.99998, 3534x48 ckptg 0.999996, 3534x48 bwdg_f32 0.999, 3534x48 fwd + bwd 0.99999, 6000x48 fwdg 0.9997, 6000x48 bwdg 0.99999, 6000x48 ckptg 0.99996, 6000x48 bwdg_f32 0.99999, 6000x48 stepg_f32 0.99996, 6000x48 fwd + bwd 0.9999, 7684x48 fwdg 0.999, 7684x48 bwdg 0.9999, 7684x48 ckptg 0.9999, 7684x48 bwdg_f32 0.999996, 7684x48 fwd + bwd 0.9997
  • The same-process run behind Table 1 also timed the previous Cake route, but those times are not in the tables. In that run, bwdg previous / new is below 1 at 8070x48 0.9997 (62.400 / 62.417 us), 10600x48 0.9998 (80.353 / 80.368 us) and 32737x48 0.998 (217.426 / 217.890 us). Both routes run the same backward module there.

Results on R200 (sm_107a, 212 SMs)

  • GPU: R200 (VR200 node; nvidia-smi name NVIDIA Graphics Device), sm_107a, compute capability 10.7, 212 SMs, driver 620.43.
  • CUDA: 13.4 (nvcc Build cuda_13.4.r13.4/compiler.38855100_0). nvidia-smi on this node does not report a CUDA version.
  • PyTorch 2.14.1+cu130 (CUDA 13.0); Triton 3.8.0; Python 3.14.8.

Table 1 (R200, VR200 node, sm_107a, CUDA 13.4): new vs the deterministic Cake kernel, the strongest non-Cake kernel and the previous Cake route

T x H op det Cake us strongest non-Cake us strongest non-Cake kernel new us previous Cake us (A/B) new us (A/B) det / new non-Cake / new previous / new (A/B)
7685x48 fwd 34.240 33.073 FLA rms_norm_gated 31.232 32.288 30.784 1.096 1.059 1.049
7685x48 fwdg 34.208 32.864 FLA rms_norm_gated 31.088 32.128 30.624 1.100 1.057 1.049
7685x48 bwdg 66.017 65.921 FLA rms_norm_gated 59.552 59.808 59.816 1.109 1.107 0.9999
7685x48 ckptg 131.969 127.841 FLA rms_norm_gated 119.025 120.888 118.801 1.109 1.074 1.018
7685x48 stepg_f32 132.664 (A/B) not measured - - 121.033 118.729 1.117 (A/B) n/a 1.019
8070x48 fwd 36.064 34.400 FLA rms_norm_gated 32.224 33.536 32.032 1.119 1.068 1.047
8070x48 fwdg 35.920 34.192 FLA rms_norm_gated 32.065 33.600 32.000 1.120 1.066 1.050
8070x48 bwdg 68.609 67.200 FLA rms_norm_gated 62.417 62.577 62.569 1.099 1.077 1.0001
8070x48 ckptg 137.633 131.777 FLA rms_norm_gated 123.800 126.113 123.680 1.112 1.064 1.020
8070x48 stepg_f32 138.048 (A/B) not measured - - 126.272 123.801 1.115 (A/B) n/a 1.020
9222x48 fwd 40.560 38.400 FLA rms_norm_gated 36.096 37.056 36.352 1.124 1.064 1.019
9222x48 fwdg 40.368 38.368 FLA rms_norm_gated 35.904 37.048 36.448 1.124 1.069 1.016
9222x48 bwdg 76.033 74.673 FLA rms_norm_gated 71.921 72.792 72.788 1.057 1.038 1.00005
9222x48 ckptg 154.049 147.345 FLA rms_norm_gated 139.921 142.809 140.664 1.101 1.053 1.015
9222x48 stepg_f32 154.368 (A/B) not measured - - 142.613 140.416 1.099 (A/B) n/a 1.016
10600x48 fwd 46.336 43.457 FLA rms_norm_gated 40.481 43.072 41.088 1.145 1.074 1.048
10600x48 fwdg 46.400 43.488 FLA rms_norm_gated 40.480 42.985 40.945 1.146 1.074 1.050
10600x48 bwdg 84.672 84.657 FLA rms_norm_gated 80.368 80.192 80.168 1.054 1.053 1.0003
10600x48 ckptg 174.337 167.185 FLA rms_norm_gated 159.280 163.128 159.945 1.095 1.050 1.020
10600x48 stepg_f32 173.825 (A/B) not measured - - 163.265 159.576 1.089 (A/B) n/a 1.023
12296x48 fwd 52.769 49.472 FLA rms_norm_gated 45.408 48.000 45.632 1.162 1.089 1.052
12296x48 fwdg 52.736 49.297 FLA rms_norm_gated 45.217 47.937 45.440 1.166 1.090 1.055
12296x48 bwdg 96.032 96.737 FLA rms_norm_gated 91.056 91.265 91.281 1.055 1.062 0.9998
12296x48 ckptg 198.306 191.121 FLA rms_norm_gated 180.242 184.335 180.161 1.100 1.060 1.023
12296x48 stepg_f32 198.080 (A/B) not measured - - 184.945 180.760 1.096 (A/B) n/a 1.023
16384x32 fwd 47.040 44.704 FLA rms_norm_gated 42.401 44.192 42.144 1.109 1.054 1.049
16384x32 fwdg 46.977 44.561 FLA rms_norm_gated 42.257 44.168 42.128 1.112 1.055 1.048
16384x32 bwdg 84.913 85.792 FLA rms_norm_gated 82.593 82.696 82.677 1.028 1.039 1.0002
16384x32 ckptg 176.129 170.473 FLA rms_norm_gated 163.969 167.824 163.889 1.074 1.040 1.024
16384x32 stepg_f32 176.940 (A/B) not measured - - 167.832 164.329 1.077 (A/B) n/a 1.021
16384x48 fwd 68.704 64.161 FLA rms_norm_gated 59.040 62.144 58.849 1.164 1.087 1.056
16384x48 fwdg 68.673 64.065 FLA rms_norm_gated 59.040 62.104 58.848 1.163 1.085 1.055
16384x48 bwdg 119.969 121.521 FLA rms_norm_gated 115.489 115.920 115.913 1.039 1.052 1.0001
16384x48 ckptg 254.178 244.465 FLA rms_norm_gated 230.194 236.080 229.649 1.104 1.062 1.028
16384x48 stepg_f32 254.521 (A/B) not measured - - 236.593 230.560 1.104 (A/B) n/a 1.026
32737x48 fwd 132.353 122.977 FLA rms_norm_gated 110.177 116.704 110.176 1.201 1.116 1.059
32737x48 fwdg 132.289 122.657 FLA rms_norm_gated 110.129 116.544 109.985 1.201 1.114 1.060
32737x48 bwdg 225.921 229.345 FLA rms_norm_gated 217.890 218.561 218.554 1.037 1.053 1.00003
32737x48 ckptg 487.251 472.547 FLA rms_norm_gated 435.267 448.137 436.026 1.119 1.086 1.028
32737x48 stepg_f32 489.885 (A/B) not measured - - 447.854 435.632 1.125 (A/B) n/a 1.028

Rows where the strongest non-Cake kernel beats new: none.

Table 2 (R200, VR200 node, sm_107a, CUDA 13.4): full step, previous Cake route vs new (A/B medians)

T x H rows op previous Cake us new us previous / new det Cake us det / new
7685x48 368880 fwd 32.288 30.784 1.049 34.848 1.132
7685x48 368880 fwdg 32.128 30.624 1.049 34.752 1.135
7685x48 368880 bwdg 59.808 59.816 0.9999 66.017 1.104
7685x48 368880 ckptg 120.888 118.801 1.018 132.664 1.117
7685x48 368880 bwdg_f32 59.936 59.928 1.0001 66.097 1.103
7685x48 368880 stepg_f32 121.033 118.729 1.019 132.664 1.117
7685x48 368880 fwd + bwd 91.936 90.440 1.017 100.769 1.114
8070x48 387360 fwd 33.536 32.032 1.047 36.032 1.125
8070x48 387360 fwdg 33.600 32.000 1.050 35.952 1.124
8070x48 387360 bwdg 62.577 62.569 1.0001 68.952 1.102
8070x48 387360 ckptg 126.113 123.680 1.020 138.104 1.117
8070x48 387360 bwdg_f32 62.433 62.432 1.00001 68.784 1.102
8070x48 387360 stepg_f32 126.272 123.801 1.020 138.048 1.115
8070x48 387360 fwd + bwd 96.177 94.569 1.017 104.904 1.109
9222x48 442656 fwd 37.056 36.352 1.019 40.681 1.119
9222x48 442656 fwdg 37.048 36.448 1.016 40.480 1.111
9222x48 442656 bwdg 72.792 72.788 1.00005 76.064 1.045
9222x48 442656 ckptg 142.809 140.664 1.015 154.225 1.096
9222x48 442656 bwdg_f32 72.221 72.224 0.9999 76.016 1.052
9222x48 442656 stepg_f32 142.613 140.416 1.016 154.368 1.099
9222x48 442656 fwd + bwd 109.840 109.236 1.006 116.544 1.067
10600x48 508800 fwd 43.072 41.088 1.048 45.984 1.119
10600x48 508800 fwdg 42.985 40.945 1.050 46.080 1.125
10600x48 508800 bwdg 80.192 80.168 1.0003 84.696 1.056
10600x48 508800 ckptg 163.128 159.945 1.020 173.849 1.087
10600x48 508800 bwdg_f32 80.064 80.060 1.0001 84.768 1.059
10600x48 508800 stepg_f32 163.265 159.576 1.023 173.825 1.089
10600x48 508800 fwd + bwd 123.177 121.113 1.017 130.776 1.080
16384x32 524288 fwd 44.192 42.144 1.049 47.296 1.122
16384x32 524288 fwdg 44.168 42.128 1.048 47.256 1.122
16384x32 524288 bwdg 82.696 82.677 1.0002 85.088 1.029
16384x32 524288 ckptg 167.824 163.889 1.024 176.730 1.078
16384x32 524288 bwdg_f32 82.912 82.921 0.9999 85.216 1.028
16384x32 524288 stepg_f32 167.832 164.329 1.021 176.940 1.077
16384x32 524288 fwd + bwd 126.864 124.805 1.017 132.344 1.060
12296x48 590208 fwd 48.000 45.632 1.052 52.768 1.156
12296x48 590208 fwdg 47.937 45.440 1.055 52.632 1.158
12296x48 590208 bwdg 91.265 91.281 0.9998 95.968 1.051
12296x48 590208 ckptg 184.335 180.161 1.023 198.137 1.100
12296x48 590208 bwdg_f32 91.264 91.264 1.000 96.032 1.052
12296x48 590208 stepg_f32 184.945 180.760 1.023 198.080 1.096
12296x48 590208 fwd + bwd 139.202 136.721 1.018 148.600 1.087
16384x48 786432 fwd 62.144 58.849 1.056 68.896 1.171
16384x48 786432 fwdg 62.104 58.848 1.055 68.720 1.168
16384x48 786432 bwdg 115.920 115.913 1.0001 120.096 1.036
16384x48 786432 ckptg 236.080 229.649 1.028 254.689 1.109
16384x48 786432 bwdg_f32 115.361 115.360 1.00001 120.033 1.041
16384x48 786432 stepg_f32 236.593 230.560 1.026 254.521 1.104
16384x48 786432 fwd + bwd 178.025 174.761 1.019 188.817 1.080
32737x48 1571376 fwd 116.704 110.176 1.059 132.977 1.207
32737x48 1571376 fwdg 116.544 109.985 1.060 132.984 1.209
32737x48 1571376 bwdg 218.561 218.554 1.00003 226.529 1.036
32737x48 1571376 ckptg 448.137 436.026 1.028 489.689 1.123
32737x48 1571376 bwdg_f32 218.432 218.432 1.000001 226.573 1.037
32737x48 1571376 stepg_f32 447.854 435.632 1.028 489.885 1.125
32737x48 1571376 fwd + bwd 335.105 328.538 1.020 359.513 1.094
1031x48 49488 fwd 7.520 7.520 1.000 7.520 1.000
1031x48 49488 fwdg 7.616 7.616 1.000 7.584 0.996
1031x48 49488 bwdg 14.928 14.928 1.000 19.592 1.312
1031x48 49488 ckptg 27.120 27.128 0.9997 30.848 1.137
1031x48 49488 bwdg_f32 14.880 14.880 1.000 19.648 1.320
1031x48 49488 stepg_f32 26.936 26.932 1.0001 30.264 1.124
1031x48 49488 fwd + bwd 22.544 22.544 1.000 27.176 1.205
3534x48 169632 fwd 16.384 16.384 1.000 17.984 1.098
3534x48 169632 fwdg 16.256 16.256 1.000 17.792 1.094
3534x48 169632 bwdg 33.056 33.056 0.99998 38.496 1.165
3534x48 169632 ckptg 63.320 63.320 0.999996 72.216 1.140
3534x48 169632 bwdg_f32 33.008 33.032 0.999 38.329 1.160
3534x48 169632 stepg_f32 63.576 63.576 1.000002 71.976 1.132
3534x48 169632 fwd + bwd 49.312 49.312 0.99999 56.288 1.141
6000x48 288000 fwd 24.544 24.544 1.000 28.320 1.154
6000x48 288000 fwdg 24.513 24.520 0.9997 28.384 1.158
6000x48 288000 bwdg 49.504 49.504 0.99999 55.041 1.112
6000x48 288000 ckptg 95.757 95.760 0.99996 108.737 1.136
6000x48 288000 bwdg_f32 49.504 49.504 0.99999 54.912 1.109
6000x48 288000 stepg_f32 96.121 96.125 0.99996 108.584 1.130
6000x48 288000 fwd + bwd 74.017 74.024 0.9999 83.425 1.127
7684x48 368832 fwd 30.680 30.672 1.0003 34.641 1.129
7684x48 368832 fwdg 30.528 30.553 0.999 34.496 1.129
7684x48 368832 bwdg 59.864 59.872 0.9999 65.896 1.101
7684x48 368832 ckptg 118.441 118.449 0.9999 132.328 1.117
7684x48 368832 bwdg_f32 59.744 59.744 0.999996 66.017 1.105
7684x48 368832 stepg_f32 118.544 118.537 1.0001 132.416 1.117
7684x48 368832 fwd + bwd 90.393 90.424 0.9997 100.392 1.110

Summary by CodeRabbit

  • Performance Improvements

    • Updated forward-processing paths on supported hardware to better handle larger workloads. The extra-long path is now available on additional hardware, with its activation point aligned to the existing long-path threshold.

  • Documentation

    • Clarified when forward processing prefetches the next block on supported hardware.

  • Tests

    • Expanded coverage for routing behavior and inputs near the updated processing threshold.

@averyhNV
perf(cake_rmsnorm_silu_gate_train): warm the next row block in L2 in …
10c6017 
…the long forward on SM107

With deterministic=False, on sm_107a from 1740 rows per SM (the long
forward's start there) the forward runs the long forward's program whose
thread 0 also warms in L2, with one bulk prefetch per tensor where x and
z hold their rows packed, the CTA's next row block: the extra-long tier
(role fwd_xlong) that sm_103a runs from 3000 rows per SM, in place of
the long forward on sm_107a.  Every other route, grid, threshold and
architecture is unchanged, and y stays bitwise equal to the
deterministic forward (a prefetch writes no register and no memory).

cake_jit: the extra-long forward record on sm_103a and sm_107a (from
3000 and 1740 rows per SM), its sm_107a route in place of the long
forward's, and the long forward record without sm_107a.  The tests cover
the extra-long tier's start, routes, grid and stall behavior on both
architectures; the docs name it.
@yyihuang

yyihuang commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

@flashinfer-bot run tests/norm/test_cake_rmsnorm_silu_gate_det_train.py tests/norm/test_cake_rmsnorm_silu_gate_det_train_aliases.py tests/norm/test_cake_rmsnorm_silu_gate_det_train_api.py tests/norm/test_cake_rmsnorm_silu_gate_det_train_stall.py tests/norm/test_cake_rmsnorm_silu_gate_train.py tests/norm/test_cake_rmsnorm_silu_gate_train_api.py tests/norm/test_cake_rmsnorm_silu_gate_train_stall.py

@yyihuang
yyihuang enabled auto-merge (squash) November 4, 2026 18:38
@flashinfer-bot flashinfer-bot added the op: misc norm, activation, sampling, RoPE, quantization, etc. label Nov 4, 2026
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The sm_107a forward route now selects the extra-long module from 1740 rows per SM. The extra-long grid rule, build metadata, documentation, and routing, API, grid, and stall-edge tests are updated.

Changes: Forward routing and validation

Layer / File(s) Summary
Extra-long route and thresholds
flashinfer/cake_rmsnorm_silu_gate_train_backend/cake_jit.py, flashinfer/cake_rmsnorm_silu_gate_train_backend/__init__.py, flashinfer/cake_rmsnorm_silu_gate_train_backend/build.json		 The extra-long forward record now includes sm_107a at 1740 rows per SM, and the sm_107a route selects that record. The long-forward record no longer includes sm_107a. Documentation and build metadata are updated.
Routing and tier-edge validation
tests/norm/test_cake_rmsnorm_silu_gate_train.py, tests/norm/test_cake_rmsnorm_silu_gate_train_api.py, tests/norm/test_cake_rmsnorm_silu_gate_train_stall.py		 Tests cover sm_107a routing and extra-long grid behavior at the updated threshold. The stall-edge lookup now checks subsequent routed forward tiers.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Possibly related issues

  • Issue 6598 tracks work on the non-deterministic RMSNorm SiLU gate kernel family, including SM107 forward tiers. This change continues that scope by routing sm_107a to the extra-long forward module.

Suggested reviewers: yzh119

Merge Risk: Minimal · up to 10c60

The routing and build metadata are consistent, with no concrete merge-blocking issue identified. Merge after normal checks pass.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title names the perf change (L2-prefetching extra-long forward on SM107), which is the main change in the changeset: the sm_107a forward route moves to the 7f014401 extra-long module from 1740 row…
Description check Passed The description follows the repository template's main sections: it explains what changes and why, gives measurement tables and test context, and describes reviewer-relevant scope. It lacks the templa…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the perf change (L2-prefetching extra-long forward on SM107), which is the main change in the changeset: the sm_107a forward route moves to the 7f014401 extra-long module from 1740 rows per SM. The title is short and specific. The scope prefix 'perf(cake_rmsnorm_silu_gate_train)' matches the affected module.

Full details: Description check

Explanation

The description follows the repository template's main sections: it explains what changes and why, gives measurement tables and test context, and describes reviewer-relevant scope. It lacks the template's 'Related Issues' link (none is given), the pre-commit and test checklist items are not ticked, and the experimental-tests fence and the required full-scope CI reason are absent. The description is long and dense, but it is substantively complete for the code change.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 4 functions across 5 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2b6ac527-f9f2-43f4-b334-03a0d77c270a
Commits

Reviewing files that changed from the base of the PR and between a2f011c and 10c6017.

Files selected for processing: 6
  • flashinfer/cake_rmsnorm_silu_gate_train_backend/__init__.py
  • flashinfer/cake_rmsnorm_silu_gate_train_backend/build.json
  • flashinfer/cake_rmsnorm_silu_gate_train_backend/cake_jit.py
  • tests/norm/test_cake_rmsnorm_silu_gate_train.py
  • tests/norm/test_cake_rmsnorm_silu_gate_train_api.py
  • tests/norm/test_cake_rmsnorm_silu_gate_train_stall.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 1 remain after this review.

yzh119
yzh119 approved these changes Nov 4, 2026
View reviewed changes

@yzh119 yzh119 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one nit: LONG_FORWARD_ROWS_PER_SM in test_cake_rmsnorm_silu_gate_train_api.py still lists sm_107a: 1740 though that arch has no long forward now. Check that nothing still reads it for sm_107a.

@yyihuang yyihuang mentioned this pull request Nov 4, 2026
Open
1 task done
flashinfer-bot
flashinfer-bot approved these changes Nov 4, 2026
View reviewed changes

@flashinfer-bot flashinfer-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CODEOWNERS gap attestation (not a code review).

A code owner of a changed path has approved this PR:

  • @yzh119 -> tests/norm/test_cake_rmsnorm_silu_gate_train_api.py
  • @yzh119 -> tests/norm/test_cake_rmsnorm_silu_gate_train.py
  • @yzh119 -> tests/norm/test_cake_rmsnorm_silu_gate_train_stall.py

The following files match only the catch-all rule; this approval covers that technical gap under the one-approval policy. GitHub still requires each owned path's own owner.

  • flashinfer/cake_rmsnorm_silu_gate_train_backend/build.json
  • flashinfer/cake_rmsnorm_silu_gate_train_backend/cake_jit.py
  • flashinfer/cake_rmsnorm_silu_gate_train_backend/init.py

Head: 10c6017

@yyihuang
yyihuang merged commit dfb2170 into flashinfer-ai:main Nov 4, 2026
34 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yzh119 yzh119 yzh119 approved these changes
@flashinfer-bot flashinfer-bot flashinfer-bot approved these changes
@kahyunnam kahyunnam Awaiting requested review from kahyunnam kahyunnam is a code owner
@bkryu bkryu Awaiting requested review from bkryu bkryu is a code owner
@yongwww yongwww Awaiting requested review from yongwww yongwww is a code owner
@aleozlx aleozlx Awaiting requested review from aleozlx aleozlx is a code owner
@Jingfan-Sun Jingfan-Sun Awaiting requested review from Jingfan-Sun Jingfan-Sun is a code owner
@YangXu1990uiuc YangXu1990uiuc Awaiting requested review from YangXu1990uiuc YangXu1990uiuc is a code owner
@Anerudhan Anerudhan Awaiting requested review from Anerudhan Anerudhan is a code owner

Assignees

No one assigned

Labels

op: misc norm, activation, sampling, RoPE, quantization, etc. run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yyihuang @yzh119 @flashinfer-bot @averyhNV