[ROCM] Add support with Infinity Cache (LLC) awareness for performance improvement - [PR#2147 rebased on PR#2178]#2217

Open

tianwyan wants to merge 12 commits into

Dao-AILab:mainfrom

ROCm:tianwyan/triton_navi_2147_2178

Commits on Jan 29, 2026

[Triton]Implemented LLC-aware head grouping algorithm for long seqlen
Tianwei Yang
committed
[Navi]improved with default triton config
Tianwei Yang
committed
[Navi]Add LLC-aware head grouping to varlen_fwd
Tianwei Yang
committed
[Triton]Add one more PRE_LOAD_V_OPTIONS for autotune
Tianwei Yang
committed
[Navi]enable LLC aware heads grouping for more archs
Tianwei Yang
committed
[Navi]Adjust the default LLC untilization and triton config parameters
Tianwei Yang
committed
[Navi]Correct some configs for navi4x
Tianwei Yang
committed

Commits on Jan 30, 2026

[Navi] Moving the heads grouping to fwd_prefill.py from interface level
Tianwei Yang
committed
[Navi] code clean up, reuse get_arch and remove CUs num
Tianwei Yang
committed
[Navi]Limits the heads grouping for known RDNA; changed the DEBUG env
Tianwei Yang
committed
[ROCM] revert the fallback triton config
Tianwei Yang
committed
[Navi]use pre-allocate to reuse memory, avoids allocator overhead/fragmentation
Tianwei Yang
committed