Repository navigation

perf(cuda-ep): optimize MatMulNBits int4 dequant GEMV on native decode (40→47 tok/s) - #867

Merged

justinchuby merged 1 commit into

mainfrom

squad/cuda-matmulnbits-gemv

Aug 13, 2026

Merged

perf(cuda-ep): optimize MatMulNBits int4 dequant GEMV on native decode (40→47 tok/s)#867
justinchuby merged 1 commit into
mainfrom
squad/cuda-matmulnbits-gemv

Commits on Aug 13, 2026
- perf(cuda-ep): cache Float16-staged constant scales in MatMulNBits bf16 decode (40→47 tok/s)
  
  justinchuby
  and
  Copilot
  committed
  abc61d8