feat(sc): support non-colocated MInf

docs/guides/single-controller.md

@@ -25,7 +25,7 @@ uv run examples/run_grpo_single_controller.py --config <your-sc.yaml>
       enabled: true
     ```
 
-2. **Enable vLLM async engine** and **disable colocated inference** (SC drives rollout via `RolloutManager.generate_and_push`, which is only supported on the disaggregated async engine):
+2. **Pick a generation backend** and **disable colocated inference**. With vLLM, enable the async engine (SC drives rollout via `RolloutManager.generate_and_push`, which is only supported on the disaggregated async engine):
 
     ```yaml
     policy:
@@ -40,6 +40,21 @@ uv run examples/run_grpo_single_controller.py --config <your-sc.yaml>
             gpus_per_node: 4  # inference GPUs; remainder go to training
     ```
 
+    Megatron generation is also supported, non-colocated only. It requires the Megatron trainer (`policy.megatron_cfg.enabled: true`) and NeMo-Gym rollouts additionally require `policy.generation.mcore_generation_config.expose_http_server: true`. The exemplar — a NeMo-Gym run with the OpenAI server exposed — lives at [examples/nemo_gym/grpo_qwen3_0_6b_megatron_generation_single_controller.yaml](../../examples/nemo_gym/grpo_qwen3_0_6b_megatron_generation_single_controller.yaml):
+
+    ```yaml
+    policy:
+      megatron_cfg:
+        enabled: true
+      generation:
+        backend: "megatron"
+        colocated:
+          enabled: false
+          resources:
+            num_nodes: 1
+            gpus_per_node: 4
+    ```
+
 3. **One RL step = one optimizer step.** The SC train pump does not support multi-mini-step inside a single RL step (see `validate_single_controller_config` in [nemo_rl/algorithms/single_controller_utils/config.py](../../nemo_rl/algorithms/single_controller_utils/config.py)):
 
     ```python
@@ -171,7 +186,7 @@ SC reads its async knobs from `async_rl:` and **requires `grpo.async_grpo: null`
 The SC path is still under active development. Feature gaps are tracked in [issue #2625](https://github.com/NVIDIA-NeMo/RL/issues/2625). Notable items:
 
 - Train backend: only Megatron is supported and validated; the AutoModel training path has not been tested on SC.
-- Generation backend: only vLLM is supported and validated; Megatron generation, SGLang, and TRT-LLM have not been tested on SC.
-- Checkpointing and validation are not yet supported (setup raises if enabled).
+- Generation backend: vLLM and Megatron generation are supported (Megatron non-colocated only — colocated Megatron generation raises at setup); SGLang and TRT-LLM have not been tested on SC.
+- Validation is not yet supported (setup raises if enabled); checkpointing is.
 - The `windowed` sampler has no `over_sampling_ratio` cap — over-produced groups aged past the window are evicted, wasting rollout compute.
 - The drain gate in refit is not yet supported.

examples/nemo_gym/grpo_qwen3_0_6b_megatron_generation_single_controller.yaml

@@ -0,0 +1,70 @@
+# GRPO on the NeMo-Gym workplace-assistant environment via the SingleController path,
+# using non-colocated Megatron Inference.
+# Gym rollouts go through the persistent Megatron engine's OpenAI-compatible server,
+# Smoke-test scale; Qwen3-0.6B on one node with 2 GPUs (1 training + 1 generation);
+# the CI-run variant of this setup is
+# tests/functional/grpo_megatron_generation_gym_single_controller.sh.
+defaults: "grpo_qwen3_30ba3b_instruct.yaml"
+
+grpo:
+  # SC requires one optimizer step per RL step:
+  # num_prompts_per_step * num_generations_per_prompt == policy.train_global_batch_size
+  num_prompts_per_step: 4
+  num_generations_per_prompt: 2
+  max_num_steps: 10  # short demo; raise for a real run
+  # SC does not support validation yet (setup raises when it is enabled).
+  val_period: 0
+  val_at_start: false
+  # SC replaces the legacy async-GRPO path.
+  async_grpo: null
+
+policy:
+  model_name: Qwen/Qwen3-0.6B
+  train_global_batch_size: 8
+  # Full workplace-assistant prompts (all tools attached) run past 4k tokens.
+  max_total_sequence_length: 8192
+
+  megatron_cfg:
+    tensor_model_parallel_size: 1
+    expert_model_parallel_size: 1
+    context_parallel_size: 1
+    sequence_parallel: false
+
+  generation:
+    backend: "megatron"
+    mcore_generation_config:
+      # NeMo-Gym drives rollouts through the engine's OpenAI server.
+      expose_http_server: true
+    colocated:
+      enabled: false
+      resources:
+        num_nodes: 1
+        gpus_per_node: 1
+
+# The gym family sets no data_plane block, so the full one is spelled out here
+# (values as documented in examples/configs/grpo_math_1B.yaml).
+data_plane:
+  enabled: true
+  impl: transfer_queue
+  backend: "simple"                  # TQ storage backend ('simple' or 'mooncake_cpu')
+  storage_capacity: 1000000          # max samples retained per partition
+  num_storage_units: 2               # storage shards
+  claim_meta_poll_interval_s: 0.5    # blocking-claim poll cadence
+  global_segment_size: 549755813888  # 512 GiB — used when backend == "mooncake_cpu"
+  local_buffer_size:   68719476736   # 64 GiB  — used when backend == "mooncake_cpu"
+
+async_rl:
+  sampler:
+    name: in_order
+    # 0 = fully synchronous. No off-policy data is admitted, so the inherited
+    # use_importance_sampling_correction=false stays valid.
+    max_lookahead_versions: 0
+  min_groups_for_streaming_train: ${grpo.num_prompts_per_step}
+  max_inflight_prompts: ${grpo.num_prompts_per_step}
+
+checkpointing:
+  # SC does not support checkpointing yet (setup raises when it is enabled).
+  enabled: false
+
+cluster:
+  gpus_per_node: 2

nemo_rl/algorithms/grpo.py

@@ -1391,12 +1391,9 @@ def init_nemo_gym():
             setup_timing_metrics.generation_init_load_time_s = megatron_gen_time
             setup_timing_metrics.nemo_gym_init_time_s = nemo_gym_time
 
-            served_urls = policy_generation.dp_openai_server_base_urls
-            if served_urls != [reserved_url]:
-                raise RuntimeError(
-                    "Megatron server came up at a different address than the one "
-                    f"pre-published to NeMo Gym: reserved {reserved_url}, serving {served_urls}."
-                )
+            MegatronGeneration.verify_served_address(
+                policy_generation.dp_openai_server_base_urls, reserved_url
+            )
         else:
             # Initialize training first so checkpoint conversion completes before inference starts.
             policy, policy_time = init_policy()

nemo_rl/algorithms/single_controller.py

@@ -74,14 +74,15 @@
 from nemo_rl.environments.nemo_gym import should_use_nemo_gym
 from nemo_rl.experience.failures import RolloutStall
 from nemo_rl.experience.rollout_manager import RolloutOutcome
+from nemo_rl.models.generation.megatron.megatron_generation import MegatronGeneration
 from nemo_rl.models.generation.sglang.sglang_generation import SGLangGeneration
 from nemo_rl.models.generation.vllm import VllmGeneration
 from nemo_rl.models.policy.tq_policy import TQPolicy
 from nemo_rl.utils.checkpoint import CheckpointManager, PathLike
 from nemo_rl.utils.logger import Logger
 from nemo_rl.utils.timer import TimeoutChecker, Timer
 
-Generation = Union[VllmGeneration, SGLangGeneration]
+Generation = Union[VllmGeneration, SGLangGeneration, MegatronGeneration]
 
 # Named `log` rather than `logger` to keep it distinct from the experiment
 # Logger this module also uses as `self._logger`.