Skip to content

feat(vllm): route RL controls through sidecar - #9

Draft
connorcarpenter15 wants to merge 5 commits into
mainfrom
feat/vllm-sidecar-rl
Draft

feat(vllm): route RL controls through sidecar#9
connorcarpenter15 wants to merge 5 commits into
mainfrom
feat/vllm-sidecar-rl

Conversation

@connorcarpenter15

Copy link
Copy Markdown
Owner

Summary

  • Route Dynamo vLLM sidecar RL controls and updates through the companion native vllm.Control gRPC API instead of a development HTTP admin server.
  • Advertise only server-reported pause/resume, sleep/wake, weight-transfer, draft-update, and weight-version capabilities; preserve vLLM's RL HTTP request schemas at the Dynamo /engine/* boundary.
  • Add opt-in Rust worker RL discovery at dyn://<namespace>.<component>.rl, including safe endpoint shutdown and discovery unregister/register ordering around pause, sleep, resume, and wake operations.
  • Keep the generic vLLM mocker fail-closed while adding a focused sidecar fake that verifies route payloads reach the expected protobuf calls.

Dependencies and overlap

  • Companion vLLM implementation: connorcarpenter15/vllm#22, pinned in the vendored protocol README at commit 7f3ab290464ac319e867b0d011d11dd6b2ff37f4.
  • This branch is based on current main and includes the four commits from ai-dynamo/dynamo#12734 because the split native Control/Inference protocol is a prerequisite. Once that PR merges, GitHub should reduce this PR to the RL-specific diff.
  • ai-dynamo/dynamo#11804 also explores RL discovery but publishes a vLLM HTTP admin URL. This PR is materially different: it keeps administration on the native gRPC sidecar channel, capability-gates every route, and publishes only Dynamo engine routes plus the existing system URL.

Validation

  • cargo fmt --all -- --check — passed.
  • cargo check -p dynamo-vllm-sidecar -p dynamo-backend-common -p dynamo-vllm-mocker -p dynamo-trtllm-sidecar -p dynamo-mocker-backend — passed.
  • cargo check --manifest-path lib/bindings/python/Cargo.toml — passed.
  • cargo clippy -p dynamo-vllm-sidecar -p dynamo-backend-common -p dynamo-vllm-mocker -p dynamo-trtllm-sidecar -p dynamo-mocker-backend --all-targets -- -D warnings — passed.
  • cargo test -p dynamo-vllm-sidecar rl_engine_routes_preserve_lifecycle_payloads_and_version -- --nocapture — passed, 1 test.
  • cargo test -p dynamo-vllm-sidecar aggregated_generation_converts_request_stream_and_usage -- --nocapture — passed, 1 test.
  • cargo test -p dynamo-backend-common routes_request_describes_the_worker_engine_surface -- --nocapture — passed, 1 test.
  • cargo test -p dynamo-backend-common engine_control_policy_wraps_discovery_mutating_controls -- --nocapture — passed, 1 test.

Not run

  • No GPU/real-engine test was run locally. The cross-process request and lifecycle contracts are covered with the in-process tonic fake; end-to-end validation requires a vLLM build containing the companion draft PR.

AI assistance disclosure

This implementation was prepared with OpenAI Codex assistance and requires human review before merge.

Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant