Add Qwen 3-8 flash multimodal support

README.md

@@ -33,7 +33,7 @@ prime-rl is a framework for large-scale reinforcement learning. It is designed t
 3. Native integration with [`verifiers`](https://github.com/PrimeIntellect-ai/verifiers) environments through the [Environments Hub](https://app.primeintellect.ai/dashboard/environments?ex_sort=most_stars), including built-in support for SWE and agentic environments.
 4. End-to-end post-training: SFT, RL training, and evals.
 5. Multi-node deployment with Slurm support.
-6. Multimodal support for VLMs such as Qwen3.5.
+6. Multimodal support for Qwen3.5 and Qwen3.8 VLMs, with [image RL and SFT examples](examples/extra/vlm/README.md).
 7. Hackable, modular, and extensible by design.
 8. One-line SLURM deployment for frontier models — e.g. [`GLM-5` FP8 with P/D disaggregation, the `llm-d` router, and Mooncake KV offload](examples/advanced/glm-5.3/).
 
@@ -50,6 +50,7 @@ The trainer runs the PrimeRL model implementations under `src/prime_rl/trainer/m
 | Qwen3 MoE (`qwen3_moe`) | `Qwen/Qwen3-30B-A3B`, … | yes | ✅ | ✅ |
 | Qwen3.5 MoE (`qwen3_5_moe`) | `Qwen/Qwen3.5-35B-A3B`, … | yes | ✅ | ✅ |
 | Qwen3.5 VLMs | see [advanced.md](docs/advanced.md#multimodal-training) (`qwen3_5`, `qwen3_5_moe`) | MoE only on MoE VLMs | MoE only | ✅ |
+| Qwen3.8 Flash Next VLM (`qwen4_exp`) | `Qwen/Qwen3.8-Flash-Next` | yes | ✅ | ✅ |
 | Poolside Laguna (`laguna`) | `poolside/Laguna-XS.2` | yes | ✅ | ✅ |
 | MiniMax M2 (`minimax_m2`) | `MiniMax/MiniMax-M2` | yes | ✅ | ✅ |
 | Muse Glimmer, text only (`muse_glimmer`) | `meta-models/Muse-Glimmer-30B` | no | — | ✅ |

docs/advanced.md

@@ -26,6 +26,7 @@ The trainer only runs `prime-rl`'s own model implementations, selected from the
 | Qwen3 MoE | `Qwen/Qwen3-30B-A3B`, … | ✅ | ✅ |
 | Qwen3.5 MoE | `Qwen/Qwen3.5-35B-A3B`, … | ✅ | ✅ |
 | Qwen3.5 VLMs | see [Multimodal training](#multimodal-training) | MoE only | ✅ |
+| Qwen3.8 Flash Next VLM | `Qwen/Qwen3.8-Flash-Next` | ✅ | ✅ |
 | Laguna | `poolside/Laguna-XS.2` | ✅ | ✅ |
 | MiniMax M2 | `MiniMax/MiniMax-M2` | ✅ | ✅ |
 | Muse Glimmer (text only) | `meta-models/Muse-Glimmer-30B` | — | ✅ |
@@ -161,16 +162,15 @@ The built-in VLM registry covers:
 |---|---|---|---|
 | Qwen3.5 | `qwen3_5` | `model.visual` | `model.language_model` |
 | Qwen3.5-MoE | `qwen3_5_moe` | `model.visual` | `model.language_model` |
+| Qwen3.8 Flash Next | `qwen4_exp` | `model.visual` | `model.language_model` |
 
 ### Enabling VLM Mode
 
-Add `[model.vlm]` and bfloat16 dtypes:
+Add `[model.vlm]` to your RL or SFT config:
 
 ```toml
 [model]
 name = "Qwen/Qwen3.5-4B"
-optimization_dtype = "bfloat16"
-reduce_dtype = "bfloat16"
 
 [model.vlm]
 vision_encoder_attr = "model.visual"
@@ -185,8 +185,8 @@ VLM training requires a registered custom PrimeRL implementation.
 ### Limitations
 
 - **Vision encoder frozen by default.** The default LoRA targets do not match Qwen3.5 vision modules. Set `freeze_vision_encoder = false` to fine-tune the encoder; this is incompatible with LoRA because LoRA freezes all non-adapter parameters.
-- **bfloat16 mandatory.** The trainer config validator refuses any other `optimization_dtype` / `reduce_dtype` for VLMs — vLLM serves VLMs in bfloat16 and a mismatch breaks the importance ratio.
-- **Higher KL mismatch with multi-image inputs.** Expect noisier `mismatch_kl` than text-only; this is from minor numerical differences between the trainer's and vLLM's image processing.
+- **Precision settings are separate.** The examples serve with `inference.vllm.dtype = "bfloat16"` while retaining `trainer.model.optimization_dtype = "float32"` and `reduce_dtype = "float32"`. VLM mode does not require BF16 optimizer state or gradient reduction.
+- **Monitor trainer–inference mismatch.** Image encoding, parallel reductions and model kernels can contribute numerical differences. Track `mismatch_kl/all/mean`, `masked_mismatch_kl/mean`, `unmasked_mismatch_kl/mean` and `is_masked/mean` alongside reward.
 - **Images aren't logged to monitors.** Sample logging captures the prompt text but not the actual images.
 
 ## LoRA Training

docs/algorithms.md

@@ -607,7 +607,7 @@ tok.apply_chat_template(messages, tokenize=False)
 # (the <think>R1</think> from turn 2 is gone)
 ```
 
-Hand-coded renderers ship for `qwen3`, `qwen3-vl`, `qwen3.5`, `glm-5`, `glm-4.5`, `minimax-m2`, `deepseek-v3`, `kimi-k2`, `kimi-k2.5`, `nemotron-3`, `gpt-oss`; anything else falls back to `DefaultRenderer` (a generic `apply_chat_template` wrapper). Pick one via:
+Hand-coded renderers ship for `qwen3`, `qwen3-vl`, `qwen3.5`, `qwen3.8`, `glm-5`, `glm-4.5`, `minimax-m2`, `deepseek-v3`, `kimi-k2`, `kimi-k2.5`, `nemotron-3`, `gpt-oss`; anything else falls back to `DefaultRenderer` (a generic `apply_chat_template` wrapper). Pick one via:
 
 ```toml
 [orchestrator.renderer]

docs/training.md

@@ -204,9 +204,9 @@ clear_thinking = true
 
 The same config selects the renderer for RL under `[orchestrator.renderer]`. The [`renderers` README](https://github.com/PrimeIntellect-ai/renderers#custom-renderers) shows the class behind this example.
 
-**Renderer-backed tokenization.** SFT tokenization is renderer-only. The [`renderers`](algorithms.md#renderers) package owns message-to-token conversion and loss attribution end-to-end, so position-dependent chat templates (for example templates that strip past `<think>` blocks across user turns) do not corrupt the loss mask. `[renderer]` defaults to `name = "auto"`; set a typed renderer config only when you need model-specific template controls. Hand-coded renderers ship for Qwen3, Qwen3.5, GLM-5, GLM-4.5, Kimi K2/K2.5, MiniMax M2, DeepSeek V3, Nemotron 3, GPT-OSS, and VLM families such as Qwen3-VL/Qwen3.5.
+**Renderer-backed tokenization.** SFT tokenization is renderer-only. The [`renderers`](algorithms.md#renderers) package owns message-to-token conversion and loss attribution end-to-end, so position-dependent chat templates (for example templates that strip past `<think>` blocks across user turns) do not corrupt the loss mask. `[renderer]` defaults to `name = "auto"`; set a typed renderer config only when you need model-specific template controls. Hand-coded renderers ship for Qwen3, Qwen3.5, Qwen3.8, GLM-5, GLM-4.5, Kimi K2/K2.5, MiniMax M2, DeepSeek V3, Nemotron 3, GPT-OSS, and VLM families such as Qwen3-VL/Qwen3.5/Qwen3.8.
 
-**VLM training requires a custom PrimeRL implementation.** Training a model with `[model.vlm]` set (SFT or RL) only works for models with a registered PrimeRL VLM class (currently Qwen3.5 dense and MoE).
+**VLM training requires a custom PrimeRL implementation.** Training a model with `[model.vlm]` set (SFT or RL) only works for models with a registered PrimeRL VLM class: Qwen3.5 dense and MoE, and Qwen3.8 Flash Next (`qwen4_exp`, text and images). See the [multimodal examples](../examples/extra/vlm/README.md) for SFT and multi-node image RL configurations.
 
 See [Algorithms § Multi-Turn Trajectories](algorithms.md#multi-turn-trajectories) for the full picture.
 

examples/README.md

@@ -31,7 +31,7 @@ Walk-throughs for the core environments (baseline eval → optional SFT warmup 
 Examples that don't follow the basic eval → SFT → RL walk-through pattern:
 
 - [`dynamo/`](extra/dynamo/README.md) — five-step Qwen3 math training with external Dynamo inference and NCCL weight updates
-- [`vlm/`](extra/vlm/README.md) — multimodal (VLM) SFT, dense + MoE LoRA configs
+- [`vlm/`](extra/vlm/README.md) — Qwen3.5 / Qwen3.8 multi-node image RL and dense + MoE LoRA SFT
 
 ## Related config folders
 

examples/extra/vlm/README.md

@@ -1,22 +1,39 @@
-# VLM SFT
+# Vision-language training
 
-Multimodal (VLM) supervised fine-tuning configs, dense and MoE. VLM training (`[model.vlm]` set) requires a PrimeRL VLM implementation (see [Advanced § Multimodal Training](../../../docs/advanced.md#multimodal-training)); these configs use the Qwen3.5 dense and Qwen3.6 MoE VLM implementations. Both LoRA-finetune with a frozen vision encoder and assistant-only loss masking.
+RL and SFT examples for the custom [PrimeRL VLM implementations](../../../docs/advanced.md#multimodal-training).
+
+## Image RL
+
+| Config | Model | Batch / group | CP |
+|---|---|---|---|
+| [`rl-qwen3.5.toml`](rl-qwen3.5.toml) | `Qwen/Qwen3.5-122B-A10B` | 336 / 16 | 4 |
+| [`rl-qwen3.8.toml`](rl-qwen3.8.toml) | `Qwen/Qwen3.8-Flash-Next` | 256 / 16 | 8 |
+
+Both target **4 trainer + 4 inference nodes**, each with 8× H200 GPUs.
+
+The examples use `primeintellect/color-codeword` with separate training and evaluation task slices. Replace it with your multimodal environment and compatible harness, and update run/W&B names and the shared output path. Install environments with `uv sync --all-extras --all-packages` and export any credentials they require.
+
+After [setup](../../../README.md#setup), validate from a shared checkout:
+
+```bash
+uv run rl @ examples/extra/vlm/rl-qwen3.8.toml --dry-run
+```
+
+Remove `--dry-run` to submit; swap in `rl-qwen3.5.toml` for Qwen3.5. See [Scaling](../../../docs/scaling.md) for Slurm configuration and [Training](../../../docs/training.md) for monitoring and resume.
+
+## SFT
+
+Both configs use LoRA with a frozen vision encoder and assistant-only loss masking.
 
 | Config | Model | Hardware |
 |---|---|---|
-| [`sft.toml`](sft.toml) | `Qwen/Qwen3.5-0.8B` (dense, LoRA rank 16) | tiny — sanity-scale (20 steps) |
-| [`sft-moe.toml`](sft-moe.toml) | `Qwen/Qwen3.6-35B-A3B` (MoE, LoRA rank 32, `ep = 8`) | one 8-GPU node (`num_train_gpus = 8`) |
-
-## Run it
+| [`sft.toml`](sft.toml) | `Qwen/Qwen3.5-0.8B` (dense, LoRA rank 16) | sanity-scale, 20 steps |
+| [`sft-moe.toml`](sft-moe.toml) | `Qwen/Qwen3.6-35B-A3B` (MoE, LoRA rank 32, EP8) | one 8-GPU node |
 
-Point `data.name` at your multimodal chat dataset (HF dataset with a `messages` column — see [Training § Dataset Format](../../../docs/training.md#dataset-format)), then:
+Set `data.name` to your [multimodal chat dataset](../../../docs/training.md#dataset-format), then:
 
 ```bash
 uv run sft @ examples/extra/vlm/sft.toml
 ```
 
-Notes:
-
-- The MoE config declares `[deployment]` with `num_train_gpus = 8`; keep its `ep` in sync with the GPU count.
-- `micro_batch_size = 1` is required for VLM SFT — do not raise it.
-- Checkpoints land in `<run_dir>/checkpoints/step_<N>`; export HF-format weights with `tools/convert_dcp_to_bf16.py` (see [Training](../../../docs/training.md)).
+Keep `micro_batch_size = 1` for VLM SFT and the MoE config's EP degree aligned with its GPU count. See [Training](../../../docs/training.md) for checkpoint export.

examples/extra/vlm/rl-qwen3.5.toml

@@ -0,0 +1,183 @@
+# Qwen3.5 image RL: 4 trainer + 4 inference nodes, each with 8 H200 GPUs.
+# Customize the environment IDs, W&B names and shared paths before launching.
+seq_len = 237568
+max_steps = 200
+output_dir = "/shared/outputs"
+
+[run]
+name = "your-run-name"
+
+[model]
+name = "Qwen/Qwen3.5-122B-A10B"
+
+[model.vlm]
+language_model_attr = "model.language_model"
+vision_encoder_attr = "model.visual"
+freeze_vision_encoder = true
+
+[deployment]
+type = "multi_node"
+gpus_per_node = 8
+num_train_nodes = 4
+num_infer_nodes = 1
+num_infer_replicas = 4
+orchestrator_on_inference = true
+
+[slurm]
+time = "7-00:00:00"
+job_name = "your-run-name"
+
+[ckpt]
+interval = 10
+keep_last = 1
+keep_interval = 200
+
+[weight_broadcast]
+type = "nccl"
+timeout = 7200
+
+[trainer]
+dist_timeout_seconds = 7200
+enable_router_replay = true
+
+[trainer.model]
+cp = 4
+ep = 8
+attn = "flash_attention_3"
+offload = "optimizer"
+optimization_dtype = "float32"
+reduce_dtype = "float32"
+freeze_moe_router = false
+fused_lm_head_token_chunk_size = 1024
+
+[trainer.model.ac]
+mode = "full"
+
+[trainer.model.ac_offloading]
+max_inflight_activations = 1
+
+[trainer.model.compile]
+fullgraph = false
+
+[trainer.optim]
+type = "muon"
+lr = 1e-6
+mu = 0.95
+betas1 = 0.9
+betas2 = 0.95
+weight_decay = 0.01
+max_norm = 1.0
+
+[trainer.scheduler]
+type = "constant"
+
+[trainer.log]
+level = "info"
+
+[monitors.wandb]
+entity = "your-wandb-entity"
+project = "your-wandb-project"
+name = "your-run-name"
+
+[monitors.file]
+path = "metrics.jsonl"
+compress = true
+float_decimals = 6
+
+[inference]
+enable_fp32_lm_head = true
+enable_fp32_router_logits = true
+
+[inference.vllm]
+seed = 0
+dtype = "bfloat16"
+max_model_len = 237568
+max_num_seqs = 256
+max_num_batched_tokens = 8192
+tensor_parallel_size = 4
+data_parallel_size = 2
+enable_expert_parallel = true
+all2all_backend = "allgather_reducescatter"
+gpu_memory_utilization = 0.85
+mm_encoder_tp_mode = "data"
+mm_processor_cache_gb = 128
+mm_processor_cache_type = "lru"
+mamba_cache_dtype = "float32"
+enable_prefix_caching = true
+enable_auto_tool_choice = true
+tool_call_parser = "qwen3_coder"
+reasoning_parser = "None"
+safetensors_load_strategy = "prefetch"
+enable_return_routed_experts = true
+
+[inference.router]
+type = "vllm-router"
+policy = "sticky_least_loaded"
+
+[inference.env_vars]
+VLLM_ENGINE_READY_TIMEOUT_S = "7200"
+
+[orchestrator]
+batch_size = 336
+max_staleness = 8
+
+# Adaptive episode cap: starts at 96 and grows or backs off with engine pressure.
+[orchestrator.concurrency]
+initial_inflight = 96
+max_inflight = 512
+
+[orchestrator.renderer]
+name = "qwen3.5"
+
+[orchestrator.train]
+group_size = 16
+
+[orchestrator.train.algo]
+type = "grpo"
+
+# Cycle through the task catalog in a seeded shuffled order.
+[orchestrator.train.select]
+shuffle = true
+seed = 0
+
+[orchestrator.train.sampling]
+temperature = 1.0
+top_p = 1.0
+max_completion_tokens = 16384
+
+[orchestrator.train.sampling.extra_body]
+min_p = 0.0
+top_k = -1
+return_token_ids = true
+
+[trainer.loss]
+type = "ipo"
+eps = 0.3
+adv_tau = 1.0
+max_importance_ratio = 10000.0
+
+# Replace color-codeword with your multimodal environment and compatible harness.
+[[orchestrator.train.source]]
+name = "color-codeword"
+select.include.idx = ["0:1024"]
+env.taskset.id = "primeintellect/color-codeword@0.1.0"
+env.agent.harness.id = "null"
+env.agent.harness.compaction.summarize_at_tokens = 184320
+
+[orchestrator.eval]
+interval = 10
+group_size = 2
+skip_first_step = false
+retrigger_on_resume = true
+
+[orchestrator.eval.sampling]
+temperature = 1.0
+top_p = 1.0
+max_tokens = 16384
+
+[[orchestrator.eval.source]]
+name = "color-codeword-eval"
+select.include.idx = ["1024:1152"]
+env.taskset.id = "primeintellect/color-codeword@0.1.0"
+env.agent.harness.id = "null"
+env.agent.harness.compaction.summarize_at_tokens = 184320