feat(backend): add RL route discovery endpoint - #13260
Conversation
This comment has been minimized.
This comment has been minimized.
6b25855 to
129e32e
Compare
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
129e32e to
5acd0f7
Compare
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
There was a problem hiding this comment.
Devin Review found 2 potential issues.
⚠️ 1 issue in files not directly in the diff
⚠️ The new RL toggle is silently ignored by the SGLang sidecar (lib/sidecar/sglang/src/engine.rs:121)
The SGLang sidecar builds its worker settings without carrying over the new RL toggle (..Default::default() at lib/sidecar/sglang/src/engine.rs:121), so an operator who turns RL on for that backend gets no RL endpoint and no error telling them it was ignored.
Impact: Operators enabling RL fleet-wide (the toggle is also read from the shared environment variable) see SGLang workers silently missing from RL discovery, with no startup warning or rejection.
How the flag reaches SGLang but is dropped on the floor
CommonArgs now defines --enable-rl / DYN_ENABLE_RL (lib/backend-common/src/args.rs:93-96), and SGLang flattens CommonArgs through lib/sidecar/common/src/args.rs, so the flag and env var are accepted on the SGLang CLI. However from_parsed in lib/sidecar/sglang/src/engine.rs:98-122 never sets enable_rl, so ..Default::default() leaves it false.
Contrast with the sibling sidecars that do propagate it (lib/sidecar/vllm/src/engine.rs:135, lib/sidecar/trtllm/src/engine.rs:113) and with route_to_encoder, which SGLang explicitly rejects with invalid_arg (lib/sidecar/sglang/src/engine.rs:67-71) precisely because it is unsupported. Following that existing convention, SGLang should either honor the flag or reject it at startup instead of silently discarding it. Note DYN_ENABLE_RL is also consumed by the Python vLLM path (components/src/dynamo/vllm/backend_args.py:120), so a deployment-wide env setting will reach SGLang.
Overview:
Add an opt-in RL request-plane endpoint that discovers the administrative routes exposed by a Dynamo backend worker.
This is PR 2 of 4 in the vLLM RL-control stack.
Details:
--enable-rl/DYN_ENABLE_RLfor Rust backends..rlendpoint.Where should the reviewer start?
Start with
lib/backend-common/src/rl.rs, then review the startup/shutdown wiring inlib/backend-common/src/worker.rs.Validation:
cargo fmt --all -- --checkcargo test --locked -p dynamo-backend-common rl_dispatch_only_describes_the_worker_engine_surfacecargo test --locked -p dynamo-sglang-sidecar --libcargo check --locked -p dynamo-backend-common -p dynamo-sglang-sidecar -p dynamo-vllm-sidecar -p dynamo-trtllm-sidecarcargo check --locked --manifest-path lib/bindings/python/Cargo.tomlStack:
Linear: DIS-2671
Related Issues
🚫 This PR is NOT linked to a GitHub issue: