Backend sampling multi output - #25532

Merged

ggerganov merged 15 commits into

ggml-org:masterfrom

gaugarg-nv:backend-sampling-multi-output

Aug 10, 2026

Commits on Aug 10, 2026

Enable backend sampling with token speculation
gaugarg-nv
committed
fc4fc9b
Clamp the mask sum before converting it into the sampled index
gaugarg-nv
committed
c186a3f
Add a numeric context parameter declaring the maximum outputs one sequence
gaugarg-nv
committed
62dedf4
More fixes
gaugarg-nv
committed
dbcec66
Don't reuse memory for output views.
gaugarg-nv
committed
aaa886f
Match dist between CPU and GPU
gaugarg-nv
committed
37d606e
Fix CPU and backend sampling mismatches
gaugarg-nv
committed
4dbed2e
Simpify some of the changes
gaugarg-nv
committed
eecc657
Fix tests on Vulkan
gaugarg-nv
committed
3b11240
More test fixes
gaugarg-nv
committed
a5663e1
Rebase changes
gaugarg-nv
committed
9e87478
Rebase and address review comments
gaugarg-nv
committed
1826901
Address review comments
gaugarg-nv
committed
d4c699c
Address review comments
gaugarg-nv
committed
9f5d059
Update src/llama-sampler.cpp

gaugarg-nv
and
ggerganov
authored
0ec96eb