Skip to content

[Frontend] Speed up online server preprocess by using sync tokenizer. - #27407

Closed
noooop wants to merge 7 commits into
vllm-project:mainfrom
noooop:server_overhead
Closed

[Frontend] Speed up online server preprocess by using sync tokenizer.#27407
noooop wants to merge 7 commits into
vllm-project:mainfrom
noooop:server_overhead

set --api-server-count=4 as default

d7a7de6
Select commit
Loading
Failed to load commit list.
Meta CodeSync / Meta Internal-Only Changes Check succeeded Feb 16, 2026 in 0s

There is no internal Diff connected, this can be merged now