Disaggregate prefill for kv cache register style#950

Merged

ganyi1996ppo merged 96 commits into

vllm-project:mainfrom

ganyi1996ppo:ganyi/disaggregate_prefill

Jul 26, 2025

Commits on Jul 22, 2025

draft of a3 llmdatadist connector
ganyi1996ppo
committed
add some fix for llmdatadist
ganyi1996ppo
committed
fix dp issues in new vllm
ganyi1996ppo
committed
fix v1 disaggregate prefill example ranktable generation issue

machenglong2025
authored and
ganyi1996ppo
committed
upadata gen_ranktable.sh

liziyu179
authored and
ganyi1996ppo
committed
some bugfix

liziyu179
authored and
ganyi1996ppo
committed
pd e2e & llmdatadist connector ut

liziyu179
authored and
ganyi1996ppo
committed
undate connector ut

liziyu179
authored and
ganyi1996ppo
committed
pd lifecycle ut update

underfituu
authored and
ganyi1996ppo
committed
update PD README

liziyu179
authored and
ganyi1996ppo
committed
update ut

zouyida2002
authored and
ganyi1996ppo
committed
update ut

zouyida2002
authored and
ganyi1996ppo
committed
lint fix
ganyi1996ppo
committed
refactor the connector's name to LLMDataDistCMgrConnector
ganyi1996ppo
committed
refactor the connector's name to LLMDataDistCMgrConnector
ganyi1996ppo
committed
spell issue fix
ganyi1996ppo
committed
fix transfer_param issue
ganyi1996ppo
committed
remove all additional config usage
ganyi1996ppo
committed
update e2e edge test

liziyu179
authored and
ganyi1996ppo
committed
update kv_connector ut

underfituu
authored and
ganyi1996ppo
committed
fix yapf
ganyi1996ppo
committed
fix the refactor mistake
ganyi1996ppo
committed
fix lint issue
ganyi1996ppo
committed
update edge e2e test

liziyu179
authored and
ganyi1996ppo
committed
fix mypy
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix kv_connector ut

underfituu
authored and
ganyi1996ppo
committed
fix isort
ganyi1996ppo
committed
fix isort
ganyi1996ppo
committed
fix kv_connector ut

liziyu179
authored and
ganyi1996ppo
committed
patch scheduler for pd benchmark

underfituu
authored and
ganyi1996ppo
committed
fix oom issue in ci and support heterougeous tp in deepseek
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
update gen_ranktable

liziyu179
authored and
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
update kv_connector ut

underfituu
authored and
ganyi1996ppo
committed
fix lint

underfituu
authored and
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix patch_scheduler

underfituu
authored and
ganyi1996ppo
committed
fix local_rank after torch_npu updated
ganyi1996ppo
committed
fix oom issue
ganyi1996ppo
committed
fix ci crash caused by vllm update
ganyi1996ppo
committed
fix ci crash caused by vllm update
ganyi1996ppo
committed
ut bugfix

liziyu179
authored and
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
[0.9.1][Bugfix] fix oom issue in mla and enable mla_pa for deepseek mla decode (#1311)

ganyi1996ppo
and
liziyu179
committed
fix lint
ganyi1996ppo
committed
remove patch scheduler
ganyi1996ppo
committed
update the doc
ganyi1996ppo
committed
update PD README

liziyu179
authored and
ganyi1996ppo
committed
fix ascend scheduler for disaggregated pd
ganyi1996ppo
committed
fix master kv_connector ut

liziyu179
authored and
ganyi1996ppo
committed
fix kv cache allocate isseu
ganyi1996ppo
committed
skip the float deepseek dbo test for which we do not support yet
ganyi1996ppo
committed
skip the deepseek dbo w8a8 test for oom in ci, will re-enable this after the multi-latent-mla kernel is ready
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
re-enable disaggregte pd test in main
ganyi1996ppo
committed
remove the older pd test
ganyi1996ppo
committed
fix rebase mistake
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix ci
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
fix ci issue
ganyi1996ppo
committed
fix ci issue
ganyi1996ppo
committed
fix ci
ganyi1996ppo
committed
fix mypy
ganyi1996ppo
committed
fix ci
ganyi1996ppo
committed
return query if no kv_cache
ganyi1996ppo
committed
fix aclgraph accuracy issue
ganyi1996ppo
committed
fix lint
ganyi1996ppo
committed
allocate memory in normal way when disable disaggregate prefill
ganyi1996ppo
committed
fix typo
ganyi1996ppo
committed
fix cache allocate bug
ganyi1996ppo
committed
fix ut issue of multicard
ganyi1996ppo
committed
fix ut issue of multicard
ganyi1996ppo
committed
fix pangu's accuracy issue
ganyi1996ppo
committed
fix pangu's accuracy issue
ganyi1996ppo
committed
lint fix
ganyi1996ppo
committed
fix rebase error
ganyi1996ppo
committed
remove redundant code
ganyi1996ppo
committed
force prefill node dummy run in prefill stage
ganyi1996ppo
committed
async pullkv && delay-free blocks in prefill nodes

underfituu
authored and
ganyi1996ppo
committed
increase toy proxy httpx limits & support chat/completions

underfituu
authored and
ganyi1996ppo
committed
update Disaggregate prefill README

underfituu
authored and
ganyi1996ppo
committed
fix lint

underfituu
authored and
ganyi1996ppo
committed
fix isort
ganyi1996ppo
committed
fix format
ganyi1996ppo
committed
fix some of the ut
ganyi1996ppo
committed
fix the format
ganyi1996ppo
committed
fix the ut
ganyi1996ppo
committed
fix the ut
ganyi1996ppo
committed
fix the ut
ganyi1996ppo
committed