Skip to content

[Perf] Skip blocking GPU->CPU sync of num_accepted_tokens in hybrid+a…#42574

Open
mamingyuan-nv wants to merge 1 commit into
vllm-project:mainfrom
mamingyuan-nv:skip-mamba-postprocess-blocking-sync
Open

[Perf] Skip blocking GPU->CPU sync of num_accepted_tokens in hybrid+a…#42574
mamingyuan-nv wants to merge 1 commit into
vllm-project:mainfrom
mamingyuan-nv:skip-mamba-postprocess-blocking-sync