feat(vllm): add Prime RL protocol parity

[biswapanda]

Overview:

Add the remaining generic Dynamo contracts needed for Prime-RL to train against independently deployed vLLM-RS workers through the native sidecar path. This is intentionally a stacked draft on #13066 so the review contains only the Prime-RL integration and protocol-parity delta.

Details:

• expose model-scoped RL worker discovery and sidecar-mediated control routes for stable trainer-selected worker cohorts
• preserve native NCCL and NIXL/ModelExpress updates and add filesystem reload compatibility through HTTP /collective_rpc; this is a development compatibility endpoint, not the intended production control API
• promote Prime policy-version cache salts into Dynamo's canonical routing namespace while rejecting conflicting values
• carry typed, preprocessed multimodal features through exact routing, the vendored inference protocol, and vLLM-RS lowering without reprocessing media in Dynamo
• preserve distinct routing identities for different images that share the same placeholder layout
• add generic routed-experts request fields and compact terminal response transport through Dynamo and the vLLM sidecar
• synchronize the vendored inference protocol with the exact vLLM-RS contract used for validation

Stack

• Depends on fix(backend): harden administrative route lifecycle #13259, feat(backend): add RL route discovery endpoint #13260, feat(vllm): sync RL control protocol #13261, and feat(vllm): route RL controls through sidecar #13066.
• The base is feat/vllm-sidecar-rl at acc5ee96e39c4c84506f87aa2fd15589682deb4e, the exact feat(vllm): route RL controls through sidecar #13066 head used for end-to-end validation.
• This is a fresh upstream PR and does not update the older fork PR feat(rl): integrate native vLLM sidecar biswapanda/dynamo#3.
• The multimodal routing design selectively reuses the generic contracts from feat(frontend): add exact MM routing for generate #11588 and feat(vllm): complete exact MM routing for generate #11532 while retaining the newer native sidecar implementation from feat(vllm): route RL controls through sidecar #13066.

Validation

• cargo fmt --all -- --check
• cargo test -p dynamo-vllm-sidecar (33 unit tests plus the executable doctest)
• cargo test -p dynamo-llm protocols::openai::generate (28 tests)
• cargo test -p dynamo-kv-router mm_identifier_hash_preserves_vllm_and_opaque_identifiers
• native token-generation, RL discovery/control, filesystem reload, and post-update generation smokes
• three-step NIXL/ModelExpress training
• three-step aggregated native-NCCL training on separate trainer and inference GPUs
• three-step 1P/1D native-NCCL training with directly observed KV handoff
• three-step exact Qwen/Qwen3-VL-4B-Instruct multimodal training and post-update generation
• three-step routed-experts capture/replay training and post-update generation

Where should the reviewer start?

• lib/backend-common/src/rl.rs for the RL route and request contracts
• lib/llm/src/http/service/generate.rs and lib/llm/src/protocols/openai/generate.rs for canonical routing, typed multimodal data, and routed-experts accumulation
• lib/sidecar/vllm/proto/inference.proto and lib/sidecar/vllm/src/convert.rs for the generic vLLM-RS wire and lowering boundary
• lib/sidecar/vllm/src/tests.rs for the protocol and execution-identity contracts

Related Issues

This PR is NOT linked to an issue:

• Confirmed — no related issue

[dynamo-ops]

/ok to test b197fd5

[github-actions]

👋 Hi biswapanda! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀