Skip to content

feat(vllm): complete Prime RL protocol integration - #4

Draft
biswapanda wants to merge 1 commit into
base/connor-pr-13066-currentfrom
feat/dyn-prl-goal-v1
Draft

feat(vllm): complete Prime RL protocol integration#4
biswapanda wants to merge 1 commit into
base/connor-pr-13066-currentfrom
feat/dyn-prl-goal-v1

Conversation

@biswapanda

Copy link
Copy Markdown
Owner

Summary

  • expose model-scoped RL worker discovery and sidecar-mediated control routes for separately deployed Prime-RL trainers
  • support native NCCL, NIXL/ModelExpress, and filesystem compatibility updates without introducing a Prime-specific receiver protocol
  • preserve policy-version cache salts and carry typed, preprocessed multimodal features through exact request routing and vLLM-RS lowering
  • add generic routed-experts request and terminal response transport through Dynamo and the vLLM sidecar
  • synchronize the vendored inference protocol with the exact vLLM-RS contract used for validation

Base and relationship to earlier work

This PR is intentionally based on acc5ee96e39c4c84506f87aa2fd15589682deb4e, the exact head of ai-dynamo#13066 used for end-to-end validation. It is a fresh review delta in this fork and does not update or reuse #3.

The multimodal routing design selectively reuses the generic capability and routing contracts from ai-dynamo#11588 and ai-dynamo#11532 while retaining the newer native sidecar implementation from ai-dynamo#13066.

Impact

Prime-RL can discover and control a stable Dynamo worker cohort, generate through aggregated or prefill/decode deployments, update independently deployed vLLM-RS workers, and train with multimodal or routed-experts data while the ordinary Dynamo paths remain unchanged unless the new fields and routes are used.

Validation

  • cargo fmt --all -- --check
  • cargo test -p dynamo-vllm-sidecar (33 unit tests plus the executable doctest)
  • cargo test -p dynamo-llm protocols::openai::generate (28 tests)
  • cargo test -p dynamo-kv-router mm_identifier_hash_preserves_vllm_and_opaque_identifiers
  • native token-generation, RL discovery/control, filesystem reload, and post-update generation smokes
  • three-step NIXL/ModelExpress training
  • three-step aggregated native-NCCL training on separate trainer and inference GPUs
  • three-step 1P/1D native-NCCL training with directly observed KV handoff
  • three-step exact Qwen/Qwen3-VL-4B-Instruct multimodal training and post-update generation
  • three-step routed-experts capture/replay training and post-update generation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant