feat: support EPD disaggregation#12263

Merged

mickqian merged 76 commits intosgl-project:mainfrom

gty111:epd_rebase

Dec 14, 2025

Commits on Dec 7, 2025

Implement EPD disaggregation
gty111
committed
Fix model weights loading
gty111
committed
Use zmq for transmitting embedding
gty111
committed
Fix MM embedding index
gty111
committed
Fix split mm items
gty111
committed
Fix MM embedding index
gty111
committed
Fix encoder OOM
gty111
committed
Clean up code
gty111
committed
Fix EP colocate and add prefill server ip arg
gty111
committed
Fix race condition
gty111
committed
Batch MM items again (OOM fixed by core binding)
gty111
committed
Enable assign prefill IP on the fly
gty111
committed
Fix rebase
gty111
committed
Fix image resize for Qwen
gty111
committed
Format
gty111
committed
Format
gty111
committed
Support qwen3_vl
gty111
committed
Support qwen3_vl_moe
gty111
committed
Support [E]+[PD colocate]
gty111
committed
Support mooncake for transmission
gty111
committed
Fix embedding shape
gty111
committed
Fix ib_device,mm_embedding shape,format
gty111
committed
Fix router args
gty111
committed
Add mm_transfer_backend (zmq or mooncake)
gty111
committed
Fix embedding_port
gty111
committed
Fix Qwen3-Omni
gty111
committed
Add params check and health check
gty111
committed
Fix gpu_id, send_to_prefill_sockets
gty111
committed
Support dots.vlm
gty111
committed
Save prefill preprocess time for E disaggregation
gty111
committed
feat: remove image URLs from prefill requests in EPD mode to reduce communication overhead

liusy58
authored and
gty111
committed
lint
gty111
committed
Rebase and clean up code
gty111
committed
Fix import
gty111
committed
Fix rebase
gty111
committed
Revert changes
gty111
committed
Fix qwen2_5_vl
gty111
committed
Get rid of the dependency on minlb
gty111
committed
[feat] use `--random-image-count` to generate requests contain images in range [1,image_count]

liusy58
authored and
gty111
committed
fix typo

liusy58
authored and
gty111
committed
Fix OOM for qwen3
gty111
committed
Fix import
gty111
committed
Remove async lock
gty111
committed
Support TP encoder
gty111
committed
lint
gty111
committed
support encoder send mmdata to scheduler directly.

liusy58
authored and
gty111
committed
[fix] support --dist-init-addr

liusy58
authored and
gty111
committed
Refactor and fix bugs (Add zmq_s zmq_t and mooncake backend)
gty111
committed
Fix import
gty111
committed
feat: support prefix_mm_cache

ZhengWG
authored and
gty111
committed
Lint
gty111
committed
Avoid time-consuming CPU concat
gty111
committed
Rename zmq_s and zmq_t; Fix get_local_ip_by_remote
gty111
committed
Add waiting list for zmq_to_scheduler
gty111
committed
Fix duplicate disagg_mode
gty111
committed
use thread for receiving mm data (#4)

liusy58
authored and
gty111
committed
Fix port and transfer backend (#5)
gty111
committed
Fix comments (#6)
gty111
committed
feat: add health/health_generate (#7)

ZhengWG
authored and
gty111
committed
clean code for mm_cache && add para-check (#8)

ZhengWG
authored and
gty111
committed
Fix comments (#10)
gty111
committed
fix: skip init emeb_tokens && skip send_encode without mm_inputs (#11)

ZhengWG
authored and
gty111
committed
[CI]: add EPD disaggregation integration tests (#9)

ZhengWG
authored and
gty111
committed

Commits on Dec 8, 2025

Merge branch 'main' into epd_rebase
ShangmingCai
authored

Commits on Dec 9, 2025

Commits on Dec 10, 2025

Commits on Dec 11, 2025

Merge branch 'main' into epd_rebase
yhyang201
authored

Commits on Dec 12, 2025

ut: speed up epd_dis ut (#12)
ZhengWG
authored

Commits on Dec 13, 2025

Commits on Dec 14, 2025

fix test
liusy58
committed