feat(models): add Qwen3.8-27B Megatron support

docs/about/model-support.md

@@ -14,7 +14,7 @@ for model sizes under 70B at up to 32k sequence length.
 
 **LLMs**:
 
-- **Qwen**: Qwen3.5-9B/35B-A3B/397B-A17B, Qwen3-1.5B/8B/32B, Qwen3-30B-A3B, Qwen3-235B-A22B, Qwen2.5-1.5B/7B/32B
+- **Qwen**: Qwen3.8-27B, Qwen3.5-9B/35B-A3B/397B-A17B, Qwen3-1.5B/8B/32B, Qwen3-30B-A3B, Qwen3-235B-A22B, Qwen2.5-1.5B/7B/32B
 - **GLM**: GLM-4.7-Flash, GLM-5.1, GLM-5.2
 - **Llama**: Llama 3.1/3.3-8B, Llama 3.1/3.3-70B, Llama 3.2-1B
 - **DeepSeek**: DeepSeek-V3/R1-671B

docs/guides/models/index.md

@@ -15,9 +15,9 @@ For the full list of supported models, see
 - **[Nemotron](nemotron/index.md)** — post-training recipes for Nemotron 3
   Nano, Nano Omni, Super, Ultra, and Nemotron 3.5 Lightning, spanning the
   Megatron and AutoModel backends.
-- **[Qwen](qwen/index.md)** — Qwen3.5 LLM and VLM recipes (dense and MoE) on the
-  Megatron and AutoModel backends, including thinking-mode generation-length
-  guidance.
+- **[Qwen](qwen/index.md)** — Qwen3.5 and Qwen3.8 LLM and VLM recipes (dense and
+  MoE), with backend availability documented per version and thinking-mode
+  generation-length guidance.
 
 Other model-specific guides currently live directly under
 [Guides](../../index.md) (for example, [DeepSeek](../deepseek.md)) and are

docs/guides/models/qwen/index.md

@@ -9,6 +9,8 @@ For the full list of supported Qwen models, see
 
 ## Version Guides
 
+- **[Qwen3.8](qwen3-8.md)** — initial dense `Qwen3.8-27B` text-only GRPO
+  support on the Megatron backend, with a low-cost functional nightly recipe.
 - **[Qwen3.5](qwen3-5.md)** — LLM and VLM recipes for `Qwen3.5-9B-Base`,
   `Qwen3.5-35B-A3B-Base`, and `Qwen3.5-397B-A17B` on the Megatron and AutoModel
   backends. This guide covers backend and parallelism support, example recipes, and the
@@ -27,4 +29,5 @@ truth for those models.
 :hidden:
 
 qwen3-5.md
+qwen3-8.md
 ```

docs/guides/models/qwen/qwen3-8.md

@@ -0,0 +1,66 @@
+# Qwen3.8
+
+This page describes the initial NeMo RL support for the dense
+`Qwen/Qwen3.8-27B` model.
+
+## Support Status
+
+`Qwen/Qwen3.8-27B` is **Functionally Ready** for text-only GRPO on the Megatron
+(MBridge) training backend, with vLLM inference. The shipped recipe is a short
+functional smoke test. It validates model loading, rollout generation, weight
+refit, log-probability computation, and an optimizer step; it is not a long-run
+convergence recipe.
+
+## What's Supported
+
+| Model | Modality | Training backend | Parallelism | Inference |
+| --- | --- | --- | --- | --- |
+| `Qwen/Qwen3.8-27B` | LLM (dense) | Megatron | TP + PP + CP | vLLM |
+
+The model follows the same dense hybrid-attention integration path as the
+supported Qwen3.5 dense models. Context parallelism requires sequence packing,
+which is enabled in the example recipe below.
+
+> [!NOTE]
+> AutoModel dependencies now include the Qwen3.5-family dense adapter via
+> [NeMo RL PR #3498](https://github.com/NVIDIA-NeMo/RL/pull/3498).
+> Qwen3.8 AutoModel validation and an example recipe remain tracked in
+> [issue #3675](https://github.com/NVIDIA-NeMo/RL/issues/3675).
+
+## Example Recipes
+
+The recipe below is an example starting point. Recipe YAML files under
+`examples/configs/recipes/` are the source of truth; check the YAML file for the
+authoritative settings.
+
+| Model | Modality | Algorithm | Backend | Scale | Recipe |
+|---|---|---|---|---|---|
+| Qwen3.8-27B | LLM | GRPO | Megatron | 2n8g | [`grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.yaml`](../../../../examples/configs/recipes/llm/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.yaml) |
+
+## Choose a Recipe
+
+### 27B GRPO (Megatron)
+
+Use the Megatron recipe to validate the setup, launch mechanics, logging, and
+checkpointing.
+
+```sh
+uv run examples/run_grpo.py \
+  --config examples/configs/recipes/llm/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.yaml
+```
+
+This is a functional validation recipe rather than a long-run convergence
+recipe; validate longer training separately for the target workload.
+
+#### 100-Step Functional Validation Results
+
+The reference curves below were produced by running the example recipe for more
+than 100 steps.
+
+The recipe's OpenMathInstruct-2 dataset is relatively easy for Qwen3.8-27B, and
+the maximum sequence length is limited to 4,096 tokens. Consequently, the
+reward curve does not show a pronounced upward trend. These curves are provided
+only as a functional validation reference for training beyond 100 steps, not as
+evidence of long-run convergence.
+
+![100-step functional validation curves for grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2: validation accuracy, training reward, mean generated tokens per sample, approximate entropy, generation KL error, and gradient norm](../../../assets/qwen3-8/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2-100steps.png)

examples/configs/recipes/llm/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.yaml

@@ -0,0 +1,19 @@
+defaults: grpo-qwen3.5-9b-1n8g-megatron.yaml
+grpo:
+  num_generations_per_prompt: 8
+checkpointing:
+  checkpoint_dir: results/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2
+policy:
+  model_name: Qwen/Qwen3.8-27B
+  train_global_batch_size: 256
+  sequence_packing:
+    enabled: true
+  make_sequence_length_divisible_by: 16
+  megatron_cfg:
+    pipeline_model_parallel_size: 2
+    context_parallel_size: 2
+logger:
+  wandb:
+    name: grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2
+cluster:
+  num_nodes: 2

tests/test_suites/llm/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd)
+source $SCRIPT_DIR/common.env
+
+# ===== BEGIN CONFIG =====
+NUM_NODES=2
+GPUS_PER_NODE=8
+STEPS_PER_RUN=10
+MAX_STEPS=10
+NUM_RUNS=$(( (MAX_STEPS + STEPS_PER_RUN - 1) / STEPS_PER_RUN ))  # Round up
+NUM_MINUTES=60
+# ===== END CONFIG =====
+
+exit_if_max_steps_reached
+
+# Run the experiment
+cd $PROJECT_ROOT
+uv run examples/run_grpo.py \
+    --config $CONFIG_PATH \
+    grpo.max_num_steps=$MAX_STEPS \
+    logger.log_dir=$LOG_DIR \
+    logger.wandb_enabled=True \
+    logger.wandb.project=nemo-rl \
+    logger.wandb.name=$EXP_NAME \
+    logger.monitor_gpus=True \
+    logger.tensorboard_enabled=True \
+    checkpointing.enabled=True \
+    checkpointing.checkpoint_dir=$CKPT_DIR \
+    $@ \
+    2>&1 | tee $RUN_LOG
+
+# Convert tensorboard logs to json
+uv run tests/json_dump_tb_logs.py $LOG_DIR --output_path $JSON_METRICS
+
+# Only run metrics if the target step is reached
+if [[ $(jq 'to_entries | .[] | select(.key == "train/loss") | .value | keys | map(tonumber) | max' $JSON_METRICS) -ge $MAX_STEPS ]]; then
+    uv run tests/check_metrics.py $JSON_METRICS \
+        'mean(data["train/gen_kl_error"]) < 0.002' \
+        'max(data["train/reward"]) > 0.5'
+
+    # Clean up checkpoint directory after successful run to save space.
+    rm -rf "$CKPT_DIR"
+fi

tests/test_suites/nightly.txt

@@ -32,6 +32,9 @@ tests/test_suites/llm/grpo-moonlight-16ba3b-4n8g-megatron.sh
 # Functional Qwen3.5-9B run (text-only task)
 tests/test_suites/llm/grpo-qwen3.5-9b-1n8g-megatron.sh
 
+# Functional Qwen3.8-27B run (text-only task)
+tests/test_suites/llm/grpo-qwen3.8-27b-2n8g-megatron-tp4pp2cp2.sh
+
 # Functional Qwen3.5-35B run (text-only task)
 # Re-enabled with the vLLM 0.25.1 bump: the AutoModel variant no longer hits the
 # hang from https://github.com/vllm-project/vllm/issues/36237.

tests/unit/test_recipes_and_test_suites.py

@@ -256,7 +256,7 @@ def test_all_recipe_yamls_accounted_for_in_test_suites(
     )
 
 
-def test_nightly_compute_stays_below_4181_hours(nightly_test_suite, tracker):
+def test_nightly_compute_stays_below_4197_hours(nightly_test_suite, tracker):
     command = f"DRYRUN=1 HF_HOME=... HF_DATASETS_CACHE=... CONTAINER= ACCOUNT= PARTITION= ./tools/launch {' '.join(nightly_test_suite)}"
 
     print(f"Running command: {command}")
@@ -288,8 +288,8 @@ def test_nightly_compute_stays_below_4181_hours(nightly_test_suite, tracker):
         f"Last line of output was not as expected: '{last_line}'"
     )
     total_gpu_hours = float(last_line.split(":")[-1].strip())
-    assert total_gpu_hours <= 4181, (
-        f"Total GPU hours exceeded 4181: {last_line}. We should revisit the test suites to reduce the total GPU hours."
+    assert total_gpu_hours <= 4197, (
+        f"Total GPU hours exceeded 4197: {last_line}. We should revisit the test suites to reduce the total GPU hours."
     )
     tracker.track("total_nightly_gpu_hours", total_gpu_hours)