feat(RL): add worker discovery endpoint

[biswapanda]

Overview

Adds the frontend worker discovery surface for RL frameworks. This is stacked on the worker-side RL admin routes PR and exposes only GET /v1/rl/workers; admin execution remains direct-to-worker through workers[*].system_url and /engine/<route>.

Changes

• Add the dynamo-rl crate with read-only GET /v1/rl/workers.
• Discover live dyn://<namespace>.<component>.rl instances and query them with {"method":"routes"}.
• Wire the frontend dedicated RL discovery listener through DYN_ENABLE_RL=true and DYN_RL_PORT defaulting to 8001.
• Add focused vLLM E2E pytest coverage for discovery plus direct worker /engine/* admin calls.

Validation

• cargo check -p dynamo-rl
• cargo check -p dynamo-llm
• .venv/bin/python -m pytest tests/rl/test_worker_discovery.py::test_rl_worker_discovery_and_engine_admin_routes -q
• Prime-rl full-weight filesystem E2E passed in /home/biswaranjanp/dev/rl/work/bis-dev/may-18/fullweight-fs-e2e-simple

Notes

supersedes #9382

Summary by CodeRabbit

Release Notes

• New Features

  • Added RL worker discovery API enabling discovery of live worker instances via /v1/rl/workers endpoint
  • HTTP service now supports configurable RL discovery listener on a separate port
  • Worker metadata includes routes, component information, and optional model names

• Tests

  • Added end-to-end tests for RL worker discovery and engine admin routes

[coderabbitai]

Walkthrough

This PR introduces a new dynamo-rl crate providing HTTP-based discovery of live RL worker instances, integrates it into the existing HTTP service with a separate listener port, and includes comprehensive end-to-end tests validating worker discovery and engine admin route access.

Changes

RL Worker Discovery Feature

Layer / File(s)	Summary
Workspace and crate integration Cargo.toml, lib/llm/Cargo.toml, lib/rl/Cargo.toml	Workspace adds lib/rl member and dynamo-rl dependency; lib/llm depends on dynamo-rl; manifest defines RL crate with workspace-managed dependencies.
RL worker discovery core implementation lib/rl/src/lib.rs	Public endpoint GET /v1/rl/workers discovers workers by querying Dynamo endpoint/model instances, filtering by configured RL endpoint and component allowlist, querying each worker's routes via request-plane direct calls, and returning sorted/deduplicated worker metadata with routes, optional system URL, and model association.
HTTP service RL configuration and struct updates lib/llm/src/http/service/service_v2.rs	HttpService struct adds rl_router and rl_port fields; HttpServiceConfig builder adds enable_rl, rl_port, and optional runtime; router building conditionally creates RL router when enabled; startup spawns separate listener for RL discovery on configured port.
Service entry point and runtime wiring lib/llm/src/http/service/openai.rs, lib/llm/src/entrypoint/input/http.rs	openai.rs adds rl_router constructor building RL state from environment; http.rs wires distributed runtime to service builder.
End-to-end integration tests tests/rl/test_worker_discovery.py, tests/rl/weight_update_worker.py, tests/rl/__init__.py	Test module spins up frontend and vLLM worker with RL enabled, polls discovery endpoint until worker appears, validates worker metadata and route exposure, exercises admin endpoints via discovered admin_url, and confirms inference works; includes test worker extension stubs and fixture/helper infrastructure.

---

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: adding a worker discovery endpoint for the RL framework, which is the central focus of this PR.
Description check	✅ Passed	The description includes Overview, Details (Changes section), and explicit Related Issues statement. All required template sections are present and filled with substantive information about the PR's scope and validation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/llm/src/http/service/service_v2.rs`:
- Around line 477-508: The RL discovery listener currently only logs bind
failures in spawn_rl_listener_if_configured, leaving the main HTTP service
running; change the Err(e) branch so the process fails fast: after logging the
error (address = %rl_addr, error = %e) call std::process::exit(1) (or otherwise
trigger global shutdown) so the service does not start silently without RL
discovery; update the Err branch inside spawn_rl_listener_if_configured (the
match on tokio::net::TcpListener::bind) to perform the exit/shutdown
immediately.

In `@lib/rl/src/lib.rs`:
- Around line 242-243: The call to wait_for_client_targets currently hardcodes
Duration::from_secs(5) and ignores its result, so the configured request_timeout
isn't respected and timeouts are swallowed; update the calls (the one at
wait_for_client_targets(&client, &[target.instance_id],
Duration::from_secs(5)).await and the similar block at lines ~276-287) to use
the configured request_timeout Duration (e.g., request_timeout variable or
client.request_timeout) instead of 5s, await the Result and propagate or return
an Err on timeout rather than discarding it, and adjust call sites/signatures if
needed so wait_for_client_targets returns a Result that callers check and handle
consistently.

In `@tests/rl/test_worker_discovery.py`:
- Around line 25-26: The test is currently marked as pre_merge
(pytest.mark.pre_merge) but is an expensive e2e with `@pytest.mark.timeout`(900);
change the marker to pytest.mark.post_merge on the test decorator(s) (replace
pytest.mark.pre_merge with pytest.mark.post_merge) or, if it truly must remain
pre_merge, add an explicit justification comment above the decorator(s)
explaining why this long-running test is critical for pre-merge gating; apply
the same change or justification to the other occurrences around the block
referenced (the markers at the later lines 210-213).
- Around line 90-93: _prepare_log_dir currently creates/deletes a relative
directory under the repo (f"{request.node.name}_{suffix}") which can collide and
writes into the tree; replace that with pytest's temporary directory by
obtaining the tmp_path fixture (use request.getfixturevalue("tmp_path") if you
don't want to change the call signature) and create/return a subdirectory under
it (e.g., tmp_path / f"{request.node.name}_{suffix}"), removing the manual
rmtree and returning the absolute path string so logs are written to
pytest-managed temp locations instead of the repository.
- Around line 24-31: The pytest markers on the pytestmark list are missing the
required VRAM and vLLM KV sizing markers; update the pytestmark list (the
variable pytestmark that decorates this test and references TEST_MODEL) to
include pytest.mark.profiled_vram_gib(N) with the correct GiB value for
TEST_MODEL and the vLLM KV sizing marker pytest.mark.vllm_kv_sizing(...) (or the
project’s canonical vLLM KV marker) so CI schedulers can place it safely.

In `@tests/rl/weight_update_worker.py`:
- Around line 33-35: Replace the runtime assert in update_weights_from_path with
explicit validation: in the update_weights_from_path(self, weight_path: str)
method check if not weight_path (and optionally if not isinstance(weight_path,
str)) and raise a clear exception (e.g., raise ValueError("weight_path must be a
non-empty string")) instead of using assert, then proceed with the existing
logic and return None as before.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 50522a81-a15a-43e2-8ef9-051af1b95cd9

📥 Commits

Reviewing files that changed from the base of the PR and between 7ec3512 and e487c9e.

⛔ Files ignored due to path filters (3)

• Cargo.lock is excluded by !**/*.lock
• lib/bindings/kvbm/Cargo.lock is excluded by !**/*.lock
• lib/bindings/python/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (10)

• Cargo.toml
• lib/llm/Cargo.toml
• lib/llm/src/entrypoint/input/http.rs
• lib/llm/src/http/service/openai.rs
• lib/llm/src/http/service/service_v2.rs
• lib/rl/Cargo.toml
• lib/rl/src/lib.rs
• tests/rl/__init__.py
• tests/rl/test_worker_discovery.py
• tests/rl/weight_update_worker.py