FA4 consumer Blackwell (sm_120) integration — squashed history - #3

Open

thad0ctor wants to merge 100 commits into

mainfrom

Commits on May 24, 2026

[Cute,Fwd,Sm120] Disable use_tma_O in SM80 base class

authored and
thad0ctor
committed
ead67d3
[Cute,Fwd,Sm120] Use universal smem-store atom for SM80 MMA layout

authored and
thad0ctor
committed
bc67a9c
[Cute,Fwd,Sm120] Default pack_gqa=False on consumer Blackwell

authored and
thad0ctor
committed
c0250ca
Add SM120 TMA forward kernel with warp specialization

authored and
thad0ctor
committed
e595c95
Fix arch override and add SMEM fallback in SM120 TMA forward

authored and
thad0ctor
committed
a6b9896
Move `stream` to end of FlashAttentionForwardSm120Tma.__call__

blake-snc
authored and
thad0ctor
committed
bf10b6d
Add SM80/SM120 block-sparse forward attention support

authored and
thad0ctor
committed
b7be538
Use get_total_block_count utility for block-sparse early exit check

authored and
thad0ctor
committed
3cd8cb6

Commits on May 26, 2026

Commits on May 27, 2026

Bump AITER submodule to commit 3b2e6f4 (#2540)

sstamenk
and
Copilot
authored
59f01d6

Commits on May 28, 2026

Commits on May 29, 2026

replace deprecated apis (#2602)
brandon-yujie-sun
authored
eaf806d

Commits on May 30, 2026

Bump nvidia-cutlass-dsl to >=4.5.2 and quack-kernels to >=0.5.0 (#2605)
Johnsonms
authored
f82d0dc

Commits on Jun 1, 2026

[CuTe,Fwd,Sm100] refactor mla sm100 forward and add page table (#2558)
jayhshah
authored
6dba037

Commits on Jun 2, 2026

Commits on Jun 3, 2026

Commits on Jun 4, 2026

[CuTe] Fix lint failures (#2625)
drisspg
authored
22a7223
[CuTe] Fix lint failure in flash_bwd_sm100.py (#2627)
Johnsonms
authored
766ed20

Commits on Jun 5, 2026

Commits on Jun 6, 2026

Add aux-scalars to interface to enable dynamic ints and floats in expressions (#2616)
drisspg
authored
bc58abc

Commits on Jun 8, 2026

Commits on Jun 9, 2026

Commits on Jun 10, 2026

Commits on Jun 15, 2026

Commits on Jun 16, 2026

Commits on Jun 17, 2026

Make q_subtile_factor default to identity (#2660)
drisspg
authored
d16e381

Commits on Jun 19, 2026

fix(hd256/sm100): make q/k/v contiguous before dedicated hd256 kernel (#2666)
yunweili3
authored
bdda748

Commits on Jun 20, 2026

[Cute,Bwd,Sm100] add sparse MLA (Deepseek v4) backward kernels (#2621)
jayhshah
authored
940cd96

Commits on Jun 24, 2026

fix: sync callers with new _flash_attn_fwd 4-tuple return signature (#2674)
hhy3
authored
6a94f8b

Commits on Jun 25, 2026

Fix compatibility issues with CuTe DSL 4.6.0+ (#2648)
anakinxc
authored
82d6441

Commits on Jun 26, 2026

Commits on Jun 27, 2026

ci: rebake cu130 image for cutlass-dsl 4.6.0.dev0 floor (#2684)
Johnsonms
authored
6f644f9
Update FA4 cute quack compatibility (#2676)

Luosuu
and
Johnsonms
authored
0046957

Commits on Jun 28, 2026

ci: install cutlass-dsl/quack at runtime to decouple from the baked image (#2685)
Johnsonms
authored
5ed0361

Commits on Jun 29, 2026

Commits on Jun 30, 2026

Parallelize splitkv alignment templated kernels, remove flag (#2683)
janeyx99
authored
73c992c

Commits on Jul 3, 2026

[FA3] uv installation support (#2458)

bbuschkaemper
and
Johnsonms
authored
002cce0

Commits on Jul 6, 2026

[AMD ROCm] Enable RDNA backward and adopt CK unified workspace (#2675)

authored
1f7ce2f

Commits on Jul 7, 2026

Commits on Jul 11, 2026

Commits on Jul 12, 2026

hopper/setup.py: harden tarfile extraction against path traversal and symlink escape (#2702)
aryanputta
authored
0816ef1

Commits on Jul 14, 2026

Enable 2CTA for SM100 block-sparse backward (#2661)
drisspg
authored
2402cb0

Commits on Jul 16, 2026

Add paged-KV block_table bounds check in mha_fwd_kvcache (#2711)
yunweili3
authored
77aacb6

Commits on Jul 17, 2026

Commits on Jul 19, 2026

[CuTe, SM100] Fix FP8 e4m3 accuracy: make max_offset dtype-aware to avoid P saturation (#2717)
yunweili3
authored
2409214

Commits on Jul 23, 2026

[CuTe, Flex] Allow score mod use in varlen backward (#2547)
reubenconducts
authored
b54df16

Commits on Jul 24, 2026

Expand FLASHATTENTION_DISABLE_DROPOUT to not bring in unneeded headers (#2669)
janeyx99
authored
00756db

Commits on Jul 27, 2026

add linearize scheduling to combine kernel for full cudagraph (#2692)
liangel-02
authored
14c3779

Commits on Jul 28, 2026

Numeric tweaks to fp8 (#2731)
drisspg
authored
849f660

Commits on Jul 29, 2026

Remove SM100 Functions from Hopper (#2746)
ankutalev
authored
c75d019

Commits on Aug 2, 2026

[CuTe,Sm100] Varlen Dynamic Persistent scheduler and metadata (#2559)
reubenconducts
authored
c46b814

Commits on Aug 3, 2026

[AI] Add doc on debug methodology (#2753)
jayhshah
authored
df61ab6

Commits on Aug 4, 2026

Commits on Aug 5, 2026

[CuTe,Bwd,Sm90] Fix: wait for bwd_preprocess on the block-sparse path, matching the dense path (#2756)

Fugoes
and
qaf
authored
4460ebb
[Cute, bwd, sm90/100/110] Support learnable sink in backward (#2706)
Henry Tsang
authored
5579b12

Commits on Aug 6, 2026

Commits on Aug 7, 2026

Commits on Aug 9, 2026

Fix CLC fuzz scheduler expectations (#2766)
JiaxuanBai
authored
a369df7

Commits on Aug 10, 2026