Skip to content

[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU - #54901

Merged
ywang96 merged 10 commits into
vllm-project:mainfrom
aoshen02:perf/sampling-mask-gpu-compact
Sep 4, 2026
Merged

[Perf][Model Runner V2] Compact sampling masks on GPU instead of unpacking the full-vocab bitmask on CPU#54901
ywang96 merged 10 commits into
vllm-project:mainfrom
aoshen02:perf/sampling-mask-gpu-compact

Fix sampling mask test for top-k ties

7d5a97c
Select commit
Loading
Failed to load commit list.
Meta CodeSync / Meta Internal-Only Changes Check succeeded Sep 3, 2026 in 0s

There is no internal Diff connected, this can be merged now

View more details on Meta CodeSync