Resident decode: attention page arrays in the graph, sampler scalars in one copy, a row per KV pool - #1143

Resident decode: attention page arrays in the graph, sampler scalars in one copy, a row per KV pool#1143

NSagan271 wants to merge 17 commits into

mainmstar-project/mstar:mainfrom

host-overhead/resident-sampler-attnmstar-project/mstar:host-overhead/resident-sampler-attnCopy head branch name to clipboard