feat(RL): add RL worker admin routes

[biswapanda]

Overview

Adds the worker-side RL admin surface needed by prime-rl on top of bis/nvext-tito-vllm. This PR keeps the frontend out of admin execution and prepares workers to advertise route metadata over the request plane.

Changes

• Add dynamo.common.rl helpers for RL route descriptors and LoRA request validation.
• Register vLLM RL admin routes on the worker system server under /engine/<route>.
• Register dyn://<namespace>.<component>.rl only as a route-descriptor request-plane endpoint.
• Advertise worker system_url through the Python runtime binding.
• Reuse existing vLLM load_lora / unload_lora handlers when LoRA is enabled.

Validation

• .venv/bin/python -m pytest components/src/dynamo/common/tests/test_rl_admin.py -q
• .venv/bin/python -m py_compile components/src/dynamo/common/rl/admin.py components/src/dynamo/vllm/handlers.py components/src/dynamo/vllm/worker_factory.py

Notes

supersedes #9382

---

Summary by CodeRabbit

• New Features

  • Added RL admin request-plane endpoints: health, pause/resume generation, flush cache, abort request, route dispatch, and weight-update lifecycle
  • Weight management: updates from disk/distributed/tensor and coordinated update groups
  • LoRA hot-swap with explicit hot_swap reporting and stricter request validation
  • More robust abort handling during generation to avoid cancellation side effects

• Tests

  • New unit tests covering RL admin routing, validation, endpoints, and deferred-abort behavior

[biswapanda]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

components/src/dynamo/vllm/handlers.py (1)

3303-3304: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don't signal first_token on the synthetic no-output error path.

generate_tokens() emits a synthetic error chunk when res.outputs is empty. This loop calls abort_guard.signal_first_token() for every yielded chunk, so a queued admin/disconnect abort can wake up and execute engine_client.abort() even though no real token was produced yet. That reopens the exact pre-first-token abort window the deferred guard is meant to avoid.

Suggested fix

-                        if abort_guard is not None:
+                        if abort_guard is not None and tok.get("token_ids"):
                             abort_guard.signal_first_token()

If you want to preserve the logic in one place, moving the signal into the normal-output branch inside generate_tokens() is even safer.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/src/dynamo/vllm/handlers.py` around lines 3303 - 3304, The current
loop calls abort_guard.signal_first_token() for every yielded chunk including
the synthetic no-output error from generate_tokens() (when res.outputs is
empty), which reopens the pre-first-token abort window; fix it by only signaling
the abort guard when a real token/output is produced—i.e., guard the call to
abort_guard.signal_first_token() with a check that res.outputs (or the actual
token) is non-empty or move the signal into the normal-output branch inside
generate_tokens() so synthetic error chunks never trigger signal_first_token();
update the code paths that yield chunks to use this check and reference
abort_guard.signal_first_token() and generate_tokens() when making the change.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/src/dynamo/vllm/handlers.py`:
- Around line 299-301: The defer-abort guard is being removed from registry
before awaiting guard.close(), causing a race where abort_request() can miss the
guard and call engine_client.abort(request_id); change the teardown so the guard
remains registered until close() completes — i.e., await guard.close() while the
registry still maps request_id to the guard, then remove
registry.pop(request_id, None) after the await (or wrap close() in a try/finally
that pops in the finally) so the guard is always discoverable by abort_request()
until close() finishes.

In `@components/src/dynamo/vllm/tests/test_vllm_worker_handler.py`:
- Around line 1310-1388: Add a "unit" test-type marker to these new tests by
applying pytest.mark.unit at module or class scope (e.g., set pytestmark =
[pytest.mark.unit] near the top of the file or add `@pytest.mark.unit` above class
TestRLAdminRouteHardening) so the test suite's marker validation
(tests/report_pytest_markers.py) recognizes the required Test Type category;
keep existing markers intact and ensure the new marker is imported from pytest
if necessary.

---

Outside diff comments:
In `@components/src/dynamo/vllm/handlers.py`:
- Around line 3303-3304: The current loop calls abort_guard.signal_first_token()
for every yielded chunk including the synthetic no-output error from
generate_tokens() (when res.outputs is empty), which reopens the pre-first-token
abort window; fix it by only signaling the abort guard when a real token/output
is produced—i.e., guard the call to abort_guard.signal_first_token() with a
check that res.outputs (or the actual token) is non-empty or move the signal
into the normal-output branch inside generate_tokens() so synthetic error chunks
never trigger signal_first_token(); update the code paths that yield chunks to
use this check and reference abort_guard.signal_first_token() and
generate_tokens() when making the change.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ffe9e48e-91ac-4779-8c97-7be2b14372f2

📥 Commits

Reviewing files that changed from the base of the PR and between 489c4a0 and 319b14e.

📒 Files selected for processing (4)

• components/src/dynamo/common/rl/admin.py
• components/src/dynamo/common/tests/test_rl_admin.py
• components/src/dynamo/vllm/handlers.py
• components/src/dynamo/vllm/tests/test_vllm_worker_handler.py

🚧 Files skipped from review as they are similar to previous changes (2)

• components/src/dynamo/common/tests/test_rl_admin.py
• components/src/dynamo/common/rl/admin.py