[Rollout] Add vLLM encoder-prefill disaggregation

[Gianthard-cyh]

This PR adds native vLLM Encoder-Prefill Disaggregation (EPD) to Vime. It uses worker_type: encoder and composes with the existing regular and PD rollout paths.

The implementation has two main parts:

1. Compile the user topology and group overrides into vLLM EC configuration during rollout-server orchestration.
2. Prime encoder engines from the rollout layer before the existing multimodal render/generate flow.

Related to #11.

1. Deployment configuration and override flow

The supported topologies are:

# Encoder + combined prefill/decode
server_groups:
  - worker_type: encoder
    num_gpus: 1
  - worker_type: regular
    num_gpus: 1

# Encoder + separate prefill/decode
server_groups:
  - worker_type: encoder
    num_gpus: 1
  - worker_type: prefill
    num_gpus: 1
  - worker_type: decode
    num_gpus: 1

Vime rejects incomplete or mixed EPD topologies during config resolution.

For each EPD model, the Ray orchestration layer creates one model-scoped /dev/shm/vime-ec-* path. This path is the shared part of the EC configuration and is passed through the generated group overrides to encoder and EC-consumer groups.

_compute_server_args() then combines that shared component with worker-specific vLLM configuration:

Worker type	Generated configuration
encoder	ECExampleConnector, ec_producer, enable_prefix_caching=false, and mm_encoder_only=true
regular	ECExampleConnector and ec_consumer
prefill	ECExampleConnector and ec_consumer
decode	No EC configuration; the existing KV consumer configuration is unchanged

The startup order is also topology-aware. Encoder groups are started and health-checked first, then regular or prefill/decode groups are started. Encoder engines are not registered with the regular or PD router.

Group-level overrides remain the highest-priority user configuration. The orchestration layer first provides the generated shared EC component, then the current group's overrides are applied. At server-argument construction time, Vime supplies the default connector and worker-specific role, while explicitly provided native EC fields override those defaults. This keeps the existing per-group override mechanism available for advanced vLLM configuration.

The current default uses upstream vLLM's ECExampleConnector, so built-in EPD deployment is limited to one node. Vime removes only the automatically generated /dev/shm/vime-ec-* paths during shutdown.

2. Rollout-side encoder priming

Encoder routing is implemented in the rollout layer.

The rollout manager exposes model-scoped encoder endpoints separately from router endpoints. Before a multimodal request enters the existing render/generate path, the new prime_encoder() helper:

• extracts image URLs from chat messages;
• sends one image-only /v1/chat/completions request per image;
• sets max_tokens=1, stream=false, and a unique x-request-id;
• primes multiple images concurrently and distributes them round-robin across encoder endpoints;
• reuses the existing HTTP retry policy and propagates failures;
• becomes a no-op for text-only samples and non-EPD deployments.

After priming completes, the request follows the existing flow:

/v1/chat/completions/render -> /inference/v1/generate

For E+P+D, the existing PD router and KVConnector continue to coordinate prefill and decode. The encoder remains outside that router and publishes embeddings through ECConnector.

The default multimodal rollout, streaming rollout, and the repository's Geo3K multi-turn rollout call prime_encoder() automatically. Custom multimodal rollout functions must call the same helper before render/generate:

from vime.rollout.vllm_rollout import prime_encoder

await prime_encoder(args, messages, model_name='actor')

Validation

Added CPU unit coverage for:

• supported and rejected EPD topologies;
• EC producer/consumer role construction;
• encoder-only option scoping and prefix-caching defaults;
• group override behavior;
• multi-image priming, unique request IDs, and round-robin endpoint selection.

Manual end-to-end validation was completed with Qwen/Qwen2.5-VL-3B-Instruct on 4 x RTX 4090 Plus using 1 encoder + 1 prefill + 1 decode + 1 placeholder. Encoder priming, EC transfer, PD generation, training weight synchronization, and EC cache cleanup all completed successfully.

Added a vLLM-only multimodal GPU acceptance test to the vllm-config Buildkite suite. It runs a regular baseline and 1 encoder + 1 prefill + 1 decode with Qwen/Qwen2.5-VL-3B-Instruct on three GPUs, then checks deterministic token parity, live EC/KV roles, PD router membership, text-only encoder bypass, and EC cache cleanup.

Documentation

Added matching English and Chinese EPD guides covering topology, data flow, connector behavior, defaults, and the custom rollout contract. The vLLM configuration reference now documents the encoder worker type.

AI assistance disclosure

This PR was developed with assistance from OpenAI Codex. I reviewed the changes and validated the implementation end to end.

[read-the-docs-community]

Documentation build overview

📚 vime | 🛠️ Build #33880478 | 📁 Comparing 20debb2 against latest (c0ed6d8)

  🔍 Preview build  

45 files changed · ± 45 modified

± Modified

• genindex.html
• index.html
• search.html
• advanced/arch-support-beyond-megatron.html
• advanced/delta-weight-sync.html
• advanced/external-rollout-engines.html
• advanced/fault-tolerance.html
• advanced/low-precision.html
• advanced/megatron-config.html
• advanced/observability.html
• and 35 more...

[gemini-code-assist]

Code Review

This pull request introduces native support for vLLM Encoder-Prefill-Decode (EPD) disaggregation for multimodal rollouts, including documentation, configuration validation, engine startup orchestration, and image priming logic. The review feedback highlights two important configuration issues: first, a shallow dictionary update when applying user overrides can overwrite the nested ec_transfer_config (which contains the critical generated shared_storage_path); second, command-line --vllm-ec-transfer-config arguments are silently ignored when EPD overrides are active, necessitating a merge with the base configuration.

[Gianthard-cyh]

[aoshen02]

Could you please review the code? I've simplified it.

[Gianthard-cyh]

Could you please review the code? I've simplified it.

Thanks for the update. I've reviewed the simplified version, and it looks good to me. @aoshen02

[princepride]

lgtm