Skip to content

Enable CUDA graphs for MTP inference - #4260

Merged
santhnm2 merged 148 commits into
NVIDIA:mainfrom
santhnm2:mtp_perf
Apr 26, 2026
Merged

Enable CUDA graphs for MTP inference#4260
santhnm2 merged 148 commits into
NVIDIA:mainfrom
santhnm2:mtp_perf

Commits

Commits on Apr 10, 2026

Commits on Apr 11, 2026

Commits on Apr 13, 2026

  • more perf optimizations

    committed
  • Fix dtype

    committed
  • Fix dtype

    committed
  • Fix cuda graph

    committed
  • Clean up CUDA graph

    committed
  • Clean up graph

    committed
  • Fix graph

    committed
  • Debugging

    committed
  • Fix graphs

    committed

Commits on Apr 14, 2026

Commits on Apr 20, 2026

Commits on Apr 21, 2026

Commits on Apr 22, 2026

Commits on Apr 23, 2026

Commits on Apr 24, 2026

Commits on Apr 26, 2026