Upstream Sync - up to: 27f501dbe011f4371bff938fe7e09311ab3002fa#56

Merged

LucasWilkinson merged 82 commits intomainfrom

lwilkinson/upstream-sync-3-attempt-2

Mar 20, 2025

Commits on Jan 18, 2025

Support ROCM builds from source distribution, and improve error handling (#1446)
mgorny
authored

Commits on Jan 30, 2025

Commits on Feb 3, 2025

Implement HeadDim_V != HeadDim_QK, support hdimQK=192, hdimV=128
tridao
committed

Commits on Feb 7, 2025

Rename Mma0->MmaQK, Mma1->MmaPV, use Cluster only if hdimV >= 192
tridao
committed

Commits on Feb 10, 2025

Adjust ninja build file
tridao
committed

Commits on Feb 11, 2025

Rename collective_mainloop -> mainloop, move tile_scheduler variable
tridao
committed

Commits on Feb 12, 2025

Commits on Feb 16, 2025

make seqused optional on top level interface (#1497)
vasqu
authored

Commits on Feb 18, 2025

Commits on Feb 19, 2025

Commits on Feb 22, 2025

Make rotary test optional in FA3
tridao
committed

Commits on Feb 23, 2025

Enable MLA flag in FA3 (rope=64, latent=512) (#1504)
tzadouri
authored

Commits on Feb 25, 2025

Fix loop in prepare_scheduler.cu (h/t Jay Shah)
tridao
committed

Commits on Feb 28, 2025

Commits on Mar 1, 2025

Fix cuda 12.1 build (#1511)
LucasWilkinson
authored

Commits on Mar 2, 2025

Commits on Mar 4, 2025

Commits on Mar 5, 2025

Commits on Mar 7, 2025

Always split fwd_combine_kernel on batch
tridao
committed

Commits on Mar 8, 2025

For each batch, if num_splits=1, write to O instead of O_partial
tridao
committed

Commits on Mar 11, 2025

Commits on Mar 13, 2025

Commits on Mar 14, 2025

Fix: num_splits_dynamic_ptr needs to be set before get_num_splits
tridao
committed

Commits on Mar 20, 2025