fix: Cherry-pick #14571

docs/fern/pages/use-cases/reinforcement-learning/verl.md

@@ -24,6 +24,7 @@ The native router and ThunderAgent are different scheduling paths. Choose one be
 
 - A Linux GPU environment that satisfies the selected verl, Dynamo, vLLM, CUDA, and PyTorch versions
 - `git`, Python, `etcd`, and `nats-server`
+- `flash-attn`, installed explicitly as its own step under Prepare the Source. The selected verl core reaches `flash_attn.bert_padding` through `verl/utils/attention_utils.py` on the CUDA path with no fallback, but declares the dependency in neither `requirements.txt` nor `pyproject.toml`, and the installer does not add it. Without it the training iteration stops at `ModuleNotFoundError: No module named 'flash_attn'`
 - Model and dataset paths visible on every participating node
 - Enough GPUs for the trainer and rollout layout; the validation smoke is not a full training run
 
@@ -45,6 +46,22 @@ test -f ../verl/recipe/dynamo/main_dynamo.py
 
 The recipe does not pin a complete Dynamo/vLLM image for the native-router path. Build one clean environment, record its immutable image and package versions, and keep the Dynamo and nested recipe checkouts clean during validation.
 
+> [!IMPORTANT]
+> Build that environment on vLLM `0.26.0`, which is what `ai-dynamo[vllm]==1.4.x` pins. The verl commit selected by this snapshot imports `FusedMoE` from `vllm.model_executor.layers.fused_moe.layer` at import time in `verl/utils/vllm/vllm_fp8_utils.py`. vLLM exports that name up to and including `0.26.0` and renamed it to `FusedMoEFactory` in `0.27.0`, so the import fails from `0.27.0` onward. On `ai-dynamo[vllm]==1.5.0`, which pins `vllm[flashinfer,runai,otel]==0.28.0`, loading the Dynamo rollout backend raises `ImportError: FP8 quantization not available` and the Dynamo stack never starts.
+
+### Install flash-attn
+
+`install_verl.sh` does not install `flash-attn`, so add it to the same environment before running anything below:
+
+```bash
+pip install packaging psutil ninja
+MAX_JOBS=4 pip install flash-attn --no-build-isolation
+```
+
+Run this after PyTorch is installed. `--no-build-isolation` is what lets the build compile against the PyTorch already in the environment; upstream requires PyTorch `2.2` or newer and CUDA `12.0` or newer. Confirm `ninja --version` exits `0` first — a broken `ninja` leaves the build single-threaded and it can then take hours instead of minutes. Lower `MAX_JOBS` if the machine runs out of RAM during the build. See the [FlashAttention project](https://github.com/Dao-AILab/flash-attention) for platform notes and prebuilt wheels.
+
+The selected verl also lists `flash-attn` in the `gpu` extra of its `setup.py` (`GPU_REQUIRES`), so `pip install -e '.[gpu]'` from the verl checkout is an equivalent route that additionally installs `liger-kernel`. Neither installer path the recipe uses requests that extra.
+
 ## Run the Validation Smoke
 
 Run the upstream validation-only smoke from the resulting verl checkout:
@@ -56,9 +73,14 @@ MODEL_PATH=/models/Qwen2.5-0.5B-Instruct \
 TRAIN_FILE=/data/dapo-math-17k.parquet \
 TEST_FILE=/data/aime-2024.parquet \
 RAY_DATA_HOME=/data/verl \
-bash recipe/dynamo/smoke_dynamo_v1.sh
+bash recipe/dynamo/smoke_dynamo_v1.sh \
+  actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1 \
+  actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1 \
+  actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1
 ```
 
+The three overrides are required because the selected verl leaves `use_dynamic_bsz` at `false` and defaults every micro-batch field to `null`, and neither the smoke script nor the recipe's trainer configuration sets one, so the trainer rejects the run during config validation. The smoke script forwards any extra arguments to the trainer, so pass them on the command line instead of editing the upstream script. Use the `_per_gpu` form: upstream marks the global form deprecated.
+
 The smoke starts recipe-managed etcd, NATS, a Dynamo vLLM worker, and the shared frontend. `PASS: Dynamo validation smoke completed` confirms the validation command completed; it does not prove an optimizer step or policy refresh.
 
 ## Run a Training Iteration
@@ -83,8 +105,12 @@ python3 -m recipe.dynamo.main_dynamo \
   data.train_files=/data/gsm8k/train.parquet \
   data.val_files=/data/gsm8k/test.parquet \
   actor_rollout_ref.model.path=Qwen/Qwen2.5-0.5B-Instruct \
+  actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1 \
   actor_rollout_ref.rollout.name=dynamo \
   actor_rollout_ref.rollout.mode=async \
+  actor_rollout_ref.rollout.tensor_model_parallel_size=2 \
+  actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1 \
+  actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1 \
   actor_rollout_ref.rollout.engine_kwargs.dynamo.router_mode=kv \
   ++actor_rollout_ref.rollout.engine_kwargs.dynamo.thunderagent.enabled=false \
   ++actor_rollout_ref.rollout.engine_kwargs.dynamo.enable_worker_system_metrics=true \
@@ -93,6 +119,8 @@ python3 -m recipe.dynamo.main_dynamo \
   trainer.total_training_steps=2
 ```
 
+The three micro-batch overrides are required here for the same reason they are required in the smoke. `actor_rollout_ref.rollout.tensor_model_parallel_size=2` states the selected verl's own default rather than changing it: this example runs one rollout that is tensor-parallel across both requested GPUs. Set it to `1` for two independent rollout workers instead.
+
 Adjust model, data, and resource values for your environment. A passing run must include rollout generation, reward or advantage computation, an actor update, policy synchronization, and generation after the update.
 
 ## Verify the Run