feat(inference): Dynamo integration with prime-rl

[biswapanda]

Summary

Why: Dynamo owns the inference workers and their admin endpoints, while Prime-RL normally knows only the frontend URL. Hard-coding worker URLs is fragile and does not provide an authoritative inference topology for weight synchronization.

Changes:

• add optional dynamo_discovery_url alongside the existing inference base_url
• discover admin endpoints and per-engine world sizes through /v1/rl/workers
• validate the discovery protocol, worker identities, URLs, and total inference world size
• construct a Dynamo inference pool while leaving static and elastic Prime-RL paths unchanged
• propagate explicit weight_broadcast.inference_world_size for externally managed inference

Scope

This is the minimal root integration for Dynamo-managed inference. It follows current upstream behavior and does not restore the removed pool_size plumbing.

Test plan

• ruff check on config, orchestrator, client, discovery, and test files
• focused discovery, orchestrator setup, and Dynamo configuration tests

---

Note

Medium Risk
Touches weight broadcast topology, NCCL initialization, and orchestrator startup paths; misconfigured inference_world_size or discovery could deadlock or mis-route admin/weight updates on external pools.

Overview
Adds Dynamo-managed external inference so Prime-RL can discover vLLM admin endpoints and topology without hard-coded admin_base_url lists.

Orchestrator clients gain optional dynamo_discovery_url, mutually exclusive with elastic DNS discovery and static admin_base_url. When set, DynamoInferencePool polls /v1/rl/workers, validates worker identity and summed world_size, and wires train/eval traffic through base_url while weight/health admin calls hit discovered per-engine URLs.

inference_world_size is now an explicit knob on shared/orchestrator weight-broadcast configs (filesystem, NCCL, NIXL). With inference: null, RL resolution propagates that value into trainer/orchestrator broadcast settings instead of inferring from local parallel.dp * tp. Dynamo orchestrator configs must set weight_broadcast.inference_world_size explicitly; NCCL’s “≥2 local GPUs” check is skipped when external world size is provided.

^{Reviewed by Cursor Bugbot for commit 322fd68. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 322fd68. Configure here.}

[cursor]

Unused per-engine world sizes

High Severity

Discovered per-engine world_size values are stored in _admin_world_sizes but never exposed or applied. init_nccl_broadcast and init_nixl_broadcast still assume equal GPUs per admin client, so heterogeneous Dynamo topologies (common in P/D) get incorrect rank_offsets and broken weight broadcast groups.

Additional Locations (1)

• src/prime_rl/utils/client.py#L547-L577

 

^{Reviewed by Cursor Bugbot for commit 322fd68. Configure here.}

[cursor]

Admin clients drop auth headers

Medium Severity

_setup_control_clients builds Dynamo admin AsyncClients without the API key, static headers, or headers_from_env that setup_admin_clients always attaches. Secured or header-gated admin endpoints will reject pause, resume, and weight-update calls even though discovery succeeded.

Additional Locations (1)

• src/prime_rl/utils/dynamo.py#L108-L111

 

^{Reviewed by Cursor Bugbot for commit 322fd68. Configure here.}

[S1ro1]

Just a minor nit, looks good to me, would wait for @samsja though