fix(inference): preserve valid rows, cached KV, and greedy semantics - #7061

Draft

wdykas wants to merge 7 commits into

NVIDIA:mainfrom

wdykas:fix/dynamic-inference-correctness-edge-cases

Commits on Sep 11, 2026

fix(inference): use real token count for NVLS EP=1
wdykas
committed
d584aec
fix(inference): protect inherited cached KV writes
wdykas
committed
1efe414
fix(inference): make FlashInfer greedy sampling deterministic
wdykas
committed
ab6c185
fix(inference): preserve greedy semantics with joint filters
wdykas
committed
c1725d9
fix(inference): rebind NVLS metadata after resume
wdykas
committed
2965d99
fix(inference): align Torch greedy log probabilities
wdykas
committed
fdd7419
fix(inference): refresh NVLS metadata for EP1 MTP
wdykas
committed
d0abc65