Skip to content

feat(backend): add RL route discovery endpoint - #13260

Merged
connorcarpenter15 merged 2 commits into
fix/backend-engine-route-lifecyclefrom
feat/rl-route-discovery
Aug 18, 2026
Merged

feat(backend): add RL route discovery endpoint#13260
connorcarpenter15 merged 2 commits into
fix/backend-engine-route-lifecyclefrom
feat/rl-route-discovery

Conversation

@connorcarpenter15

@connorcarpenter15 connorcarpenter15 commented Aug 14, 2026

Copy link
Copy Markdown
Member

Overview:

Add an opt-in RL request-plane endpoint that discovers the administrative routes exposed by a Dynamo backend worker.

This is PR 2 of 4 in the vLLM RL-control stack.

Details:

  • Add --enable-rl / DYN_ENABLE_RL for Rust backends.
  • Publish a sibling discovery endpoint that returns the worker's engine-route names and system-server URL.
  • Start the RL endpoint only after the primary serving endpoint is callable and stop it before worker teardown.
  • Enable the endpoint for the vLLM, TensorRT-LLM, and SGLang sidecars, plus the Rust mocker.
  • SGLang currently advertises its existing common route surface; SGLang-specific RL controls can be added separately.
  • Keep Python workers disabled because Python vLLM already owns its .rl endpoint.

Where should the reviewer start?

Start with lib/backend-common/src/rl.rs, then review the startup/shutdown wiring in lib/backend-common/src/worker.rs.

Validation:

  • cargo fmt --all -- --check
  • cargo test --locked -p dynamo-backend-common rl_dispatch_only_describes_the_worker_engine_surface
  • cargo test --locked -p dynamo-sglang-sidecar --lib
  • cargo check --locked -p dynamo-backend-common -p dynamo-sglang-sidecar -p dynamo-vllm-sidecar -p dynamo-trtllm-sidecar
  • cargo check --locked --manifest-path lib/bindings/python/Cargo.toml

Stack:

  1. #13259 — fix(backend): harden administrative route lifecycle
  2. #13260 — feat(backend): add RL route discovery endpoint (this PR)
  3. #13261 — feat(vllm): sync RL control protocol
  4. #13066 — feat(vllm): route RL controls through sidecar

Linear: DIS-2671

Related Issues

🚫 This PR is NOT linked to a GitHub issue:

  • Confirmed — no related GitHub issue

@datadog-official

This comment has been minimized.

@connorcarpenter15
connorcarpenter15 force-pushed the feat/rl-route-discovery branch 2 times, most recently from 6b25855 to 129e32e Compare August 14, 2026 19:37
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
@connorcarpenter15
connorcarpenter15 marked this pull request as ready for review August 14, 2026 21:33
@connorcarpenter15
connorcarpenter15 requested review from a team as code owners August 14, 2026 21:33
Signed-off-by: Connor Carpenter <connorc@nvidia.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

⚠️ 1 issue in files not directly in the diff

⚠️ The new RL toggle is silently ignored by the SGLang sidecar (lib/sidecar/sglang/src/engine.rs:121)

The SGLang sidecar builds its worker settings without carrying over the new RL toggle (..Default::default() at lib/sidecar/sglang/src/engine.rs:121), so an operator who turns RL on for that backend gets no RL endpoint and no error telling them it was ignored.
Impact: Operators enabling RL fleet-wide (the toggle is also read from the shared environment variable) see SGLang workers silently missing from RL discovery, with no startup warning or rejection.

How the flag reaches SGLang but is dropped on the floor

CommonArgs now defines --enable-rl / DYN_ENABLE_RL (lib/backend-common/src/args.rs:93-96), and SGLang flattens CommonArgs through lib/sidecar/common/src/args.rs, so the flag and env var are accepted on the SGLang CLI. However from_parsed in lib/sidecar/sglang/src/engine.rs:98-122 never sets enable_rl, so ..Default::default() leaves it false.

Contrast with the sibling sidecars that do propagate it (lib/sidecar/vllm/src/engine.rs:135, lib/sidecar/trtllm/src/engine.rs:113) and with route_to_encoder, which SGLang explicitly rejects with invalid_arg (lib/sidecar/sglang/src/engine.rs:67-71) precisely because it is unsupported. Following that existing convention, SGLang should either honor the flag or reject it at startup instead of silently discarding it. Note DYN_ENABLE_RL is also consumed by the Python vLLM path (components/src/dynamo/vllm/backend_args.py:120), so a deployment-wide env setting will reach SGLang.

Open in Devin Review

Comment thread lib/backend-common/src/rl.rs
@connorcarpenter15
connorcarpenter15 requested a review from a team as a code owner August 14, 2026 21:38
@connorcarpenter15
connorcarpenter15 merged commit 5306c82 into main Aug 18, 2026
122 checks passed
@connorcarpenter15
connorcarpenter15 deleted the feat/rl-route-discovery branch August 18, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants