From 179e3b57fd18dc458928f5a4e8e987b52bc494b5 Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 14:42:07 -0700 Subject: [PATCH 1/6] docs: restructure homepage Core features per the v0.1 feature list --- docs/advanced/index.md | 5 +-- docs/index.md | 74 ++++++++++++++++++++++++++++-------------- 2 files changed, 53 insertions(+), 26 deletions(-) diff --git a/docs/advanced/index.md b/docs/advanced/index.md index 36e0badb8b1..42bc8c0dff4 100644 --- a/docs/advanced/index.md +++ b/docs/advanced/index.md @@ -3,8 +3,9 @@ title: Advanced Features description: Systems-level features for large-scale and long-running RL. --- This section covers the Miles features that the Core-features section of the -homepage points at: low-precision training (FP8 / MXFP8 / INT4 QAT), Rollout -Routing Replay for MoE, speculative decoding, and LoRA training and serving. +homepage points at: low-precision training (FP8 / MXFP8 / NVFP4 / INT4 QAT), +Rollout Routing Replay for MoE, fast weight updates over P2P RDMA, fault +tolerance, speculative decoding, and LoRA training and serving. diff --git a/docs/index.md b/docs/index.md index bb020b5ac3e..ed6daa7af7a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,30 +11,56 @@ needed to run RL at trillion-parameter scale. ## Core features -- **Fast and stable support for the latest models.** Day-0 enablement of frontier - releases such as DeepSeek-V4, with rapid follow-on support for new architectures - including GLM-5, Qwen 3.6, and Nemotron-3-Super. -- **Unified low-precision training.** Customizable precision across the rollout and - training engines, with unified **BF16**, **FP8**, **MXFP8**, and **INT4 QAT** recipes - available now and an **NVFP4** training recipe in progress. -- **Efficient Rollout Routing Replay (R3).** For MoE models, expert routing captured - during inference is replayed during the trainer's forward pass, eliminating the - mismatch that destabilizes large-scale MoE RL. Optimized with a routing-result cache - and overlapped device-to-host (D2H) copy to reduce overhead in both single-turn and - multi-turn RL. -- **Speculative rollout with online MTP-SFT.** Miles keeps the draft model's acceptance - rate high through training by fine-tuning MTP layers on-policy. -- **LoRA training and serving.** Both SFT and RL recipes support LoRA adapters, - and the same adapters load directly into SGLang for rollout — no separate - merge or conversion step. -- **Native agentic rollout.** Tool use, multi-turn dialogue, search, code - execution, and multi-agent co-evolution are all supported through clean Python - extension points. -- **Minimal core, maximal extension.** Twenty-plus plug-points let you replace the - rollout, reward, loss, or filter without forking the trainer. -- **Broad hardware support.** First-class on NVIDIA Hopper (H100, H200) and - Blackwell (B100, B200, GB200, GB300), with AMD MI300X / MI325 / MI350 / - MI355X also supported via ROCm. +### Efficiency & stability + +- **Fully async RL.** Rollout and training workers are decoupled, with configurable + on- and off-policy schedules, an optimized pipeline with fewer bubbles, and + customizable async rollout and eval modes. See + [Fully Async Rollout](/user-guide/fully-async). +- **Fast agentic rollout.** High-throughput generation on + [SGLang](https://github.com/sgl-project/sglang) behind the SGLang router, tuned + for multi-turn agentic workloads. +- **Fast weight updates.** Updated weights sync back in-loop without pausing + rollout — under 10 seconds for a model like Kimi-K2.6 — with + [P2P RDMA](/advanced/p2p-weight-transfer) as a fast path for disaggregated setups. +- **Unified low-precision training.** [MXFP8 and NVFP4](/advanced/fp8-low-precision) + training with a numerically stable RL recipe that reduces precision-induced + divergence; FP8, [INT4 QAT](/advanced/int4-qat), BF16, and FP16 are also supported. +- **Token-in-token-out (TITO).** Supported for + [all models and any black-box agent harness](/user-guide/agentic-chat-template) — + no detokenize/retokenize round-trips between rollout and training. +- **Rollout Routing Replay (R3).** Expert routing recorded during rollout is + [replayed in the trainer's forward pass](/advanced/miles-router), eliminating the + routing mismatch that destabilizes large-scale MoE RL, with compute and + communication overlapped to minimize overhead. +- **LoRA and multi-LoRA.** [Low-rank adapters](/advanced/lora) train frontier-scale + models on a fraction of the GPUs, and the same adapters load directly into SGLang + for rollout — no separate merge or conversion step. +- **Fault tolerance.** When an SGLang engine dies, Miles + [recovers it and resumes the run in place](/advanced/fault-tolerance) — no + restart, no pause. +- **Day-0 model support.** Day-0 enablement of frontier releases such as + DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron — and beyond day-0, nearly + all frontier models (see [Supported models](#supported-models)). + +### Design, support & user experience + +- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor) and + [OpenEnv](/user-guide/openenv) integrations with local CPU sandboxes or Daytona + cloud sandboxes, plus [AgentENV](https://github.com/kvcache-ai/AgentENV) support. +- **Highly customizable pipeline.** Shape every workload through + [twenty-plus plug-points](/user-guide/customization), from reward computation to + the full rollout function. +- **Megatron or FSDP.** + [Switch training backends](/developer/experimental-features#fsdp-backend) without + rewriting your training loop. +- **Wide recipe support.** RL (GRPO, PPO), [SFT](/examples/openhermes-sft), and + on-policy distillation. +- **Verified on multiple hardware generations.** GB300, GB200, B200, H200, H100, + and AMD MI355X — see [Platforms](/platforms/index). +- **[Miles dashboard](/user-guide/dashboard).** A self-hosted web UI for a run's + training dynamics and compute efficiency: what every GPU was doing during a step, + and what each trajectory contained at the token level. ## Supported models From b12930a14096d7bd517a57fe7d4386b9a569dc6f Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 14:53:27 -0700 Subject: [PATCH 2/6] docs: enumerate sandbox providers on the homepage; drop the SFT link --- docs/index.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/index.md b/docs/index.md index ed6daa7af7a..093eb9acd4f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -45,17 +45,19 @@ needed to run RL at trillion-parameter scale. ### Design, support & user experience -- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor) and - [OpenEnv](/user-guide/openenv) integrations with local CPU sandboxes or Daytona - cloud sandboxes, plus [AgentENV](https://github.com/kvcache-ai/AgentENV) support. +- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor), + [OpenEnv](/user-guide/openenv), and [NeMo-Gym](/user-guide/nemo-gym) integrations, + running local CPU sandboxes or per-episode sandboxes on + [Daytona](https://www.daytona.io/), [E2B](https://e2b.dev/), and self-hosted + [AgentENV](https://github.com/kvcache-ai/AgentENV) — see + [Environments](/user-guide/environments) for the support matrix. - **Highly customizable pipeline.** Shape every workload through [twenty-plus plug-points](/user-guide/customization), from reward computation to the full rollout function. - **Megatron or FSDP.** [Switch training backends](/developer/experimental-features#fsdp-backend) without rewriting your training loop. -- **Wide recipe support.** RL (GRPO, PPO), [SFT](/examples/openhermes-sft), and - on-policy distillation. +- **Wide recipe support.** RL (GRPO, PPO), SFT, and on-policy distillation. - **Verified on multiple hardware generations.** GB300, GB200, B200, H200, H100, and AMD MI355X — see [Platforms](/platforms/index). - **[Miles dashboard](/user-guide/dashboard).** A self-hosted web UI for a run's From da1a5557a206db8cfdf4fd561ea6f0e762c5aa5d Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 15:10:20 -0700 Subject: [PATCH 3/6] docs: address review wording on rollout, weight-update, and TITO bullets --- docs/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/index.md b/docs/index.md index 093eb9acd4f..6e44adc5d4a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -18,16 +18,16 @@ needed to run RL at trillion-parameter scale. customizable async rollout and eval modes. See [Fully Async Rollout](/user-guide/fully-async). - **Fast agentic rollout.** High-throughput generation on - [SGLang](https://github.com/sgl-project/sglang) behind the SGLang router, tuned - for multi-turn agentic workloads. + [SGLang](https://github.com/sgl-project/sglang), optimized for multi-turn + agentic workloads. - **Fast weight updates.** Updated weights sync back in-loop without pausing - rollout — under 10 seconds for a model like Kimi-K2.6 — with + rollout — under 10 seconds for a model with 1 T parameters — with [P2P RDMA](/advanced/p2p-weight-transfer) as a fast path for disaggregated setups. - **Unified low-precision training.** [MXFP8 and NVFP4](/advanced/fp8-low-precision) training with a numerically stable RL recipe that reduces precision-induced divergence; FP8, [INT4 QAT](/advanced/int4-qat), BF16, and FP16 are also supported. - **Token-in-token-out (TITO).** Supported for - [all models and any black-box agent harness](/user-guide/agentic-chat-template) — + [all models and all black-box agent harnesses](/user-guide/agentic-chat-template) — no detokenize/retokenize round-trips between rollout and training. - **Rollout Routing Replay (R3).** Expert routing recorded during rollout is [replayed in the trainer's forward pass](/advanced/miles-router), eliminating the From cdabc6ad5996d9bbb9518164e341ae8f4a020c06 Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 16:26:48 -0700 Subject: [PATCH 4/6] ci: retrigger checks (CodeQL python analyze timed out) From 12d5e392c2bf6c34f0c3597048458f9cdd707192 Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 16:55:18 -0700 Subject: [PATCH 5/6] docs: refresh the homepage supported-models table --- docs/index.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/index.md b/docs/index.md index 6e44adc5d4a..ddd6ecda7b1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -66,18 +66,20 @@ needed to run RL at trillion-parameter scale. ## Supported models -Each model name links to its recipe page. +Each model name links to its recipe page or launch script. The table is not +exhaustive — it highlights recent releases; many more models run on Miles out +of the box, including older generations of the families below. | Family | Models | |---|---| -| **DeepSeek** | [DeepSeek-V4 Pro](/models/deepseek/deepseek-v4-pro)
[DeepSeek-V4 Flash](/models/deepseek/deepseek-v4-flash)
[DeepSeek-R1](/models/deepseek/deepseek)
[DeepSeek-V3](/models/deepseek/deepseek) | -| **Thinking Machines** | [Inkling](/models/thinkingmachines/inkling) | -| **Qwen** | [Qwen3.6 MoE](/models/qwen/qwen3-6-moe)
[Qwen3.6](/models/qwen/qwen3-6)
[Qwen3.5-35B-A3B](/models/qwen/qwen3-5-moe)
[Qwen3.5-4B / 9B / 27B](/models/qwen/qwen3-5)
[Qwen3-Next-80B-A3B-Thinking](/models/qwen/qwen3-next)
[Qwen3-30B-A3B / 235B-A22B](/models/qwen/qwen3-moe)
[Qwen3-0.6B / 1.7B / 4B / 8B / 14B / 32B](/models/qwen/qwen3) | -| **GLM** | [GLM-5.1](/models/glm/glm5)
[GLM-5](/models/glm/glm5)
[GLM-4.7-Flash](/models/glm/glm4-7-flash)
[GLM-4.5](/models/glm/glm4-5)
[GLM-Z1-9B-0414](/models/glm/glm4) | -| **Kimi** | [Kimi-K2.6](/models/kimi/kimi-k2.5)
[Kimi-K2.5](/models/kimi/kimi-k2.5)
[Kimi-K2-Instruct / Thinking](/models/kimi/kimi-k2)
[Moonlight-16B-A3B](/models/kimi/moonlight) | -| **Nemotron** | [Nemotron-3-Super-120B-A12B-FP8](/models/nemotron/nemotron-3-super)
[Nemotron-3-Nano MoE](/models/nemotron/nemotron-3-nano-moe)
[Nemotron-3-Nano](/models/nemotron/nemotron-3-nano) | -| **MiMo** | [MiMo-7B-RL](/models/mimo/mimo) | -| **GPT-OSS** | [gpt-oss-20b](/models/gpt-oss/gpt-oss) | +| **DeepSeek** | [DeepSeek-V4 Pro](/models/deepseek/deepseek-v4-pro)
[DeepSeek-V4 Flash](/models/deepseek/deepseek-v4-flash) | +| **Thinking Machines** | [Inkling](/models/thinkingmachines/inkling)
[Inkling-Small](/models/thinkingmachines/inkling-small) | +| **Qwen** | [Qwen3.6 MoE](/models/qwen/qwen3-6-moe)
[Qwen3.6](/models/qwen/qwen3-6)
[Qwen3.5-35B-A3B](/models/qwen/qwen3-5-moe)
[Qwen3.5-4B / 9B / 27B](/models/qwen/qwen3-5) | +| **GLM** | [GLM-5.2](/models/glm/glm5-2)
[GLM-5.1](/models/glm/glm5)
[GLM-5](/models/glm/glm5)
[GLM-4.7-Flash](/models/glm/glm4-7-flash) | +| **Kimi** | [Kimi-K2.6](/models/kimi/kimi-k2.5)
[Kimi-K2.5](/models/kimi/kimi-k2.5) | +| **Nemotron** | [Nemotron-3-Ultra-550B-A55B](https://github.com/radixark/miles/blob/main/scripts/run_nemotron_3_ultra_550b_a55b.py)
[Nemotron-3-Super-120B-A12B-FP8](/models/nemotron/nemotron-3-super)
[Nemotron-3-Nano MoE](/models/nemotron/nemotron-3-nano-moe)
[Nemotron-3-Nano](/models/nemotron/nemotron-3-nano) | +| **Gemma** | [Gemma-4 26B-A4B](https://github.com/radixark/miles/blob/main/scripts/run_gemma_4_26b_a4b.py)
[Gemma-4 31B](https://github.com/radixark/miles/blob/main/scripts/run_gemma_4_31b.py) | +| **JoyAI** | [JoyAI-LLM-Flash](https://github.com/radixark/miles/blob/main/scripts/run_joy_ai_llm_flash.py) | See [Models](/models/index) for exact conversion commands, launch scripts, and parallelism settings. From c7890b772b91b318d7a9c982f0586af2719151dd Mon Sep 17 00:00:00 2001 From: Shi Dong Date: Fri, 7 Aug 2026 17:01:03 -0700 Subject: [PATCH 6/6] docs: add Kimi-K3 to the supported-models table --- docs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index ddd6ecda7b1..8f998e1bf57 100644 --- a/docs/index.md +++ b/docs/index.md @@ -76,7 +76,7 @@ of the box, including older generations of the families below. | **Thinking Machines** | [Inkling](/models/thinkingmachines/inkling)
[Inkling-Small](/models/thinkingmachines/inkling-small) | | **Qwen** | [Qwen3.6 MoE](/models/qwen/qwen3-6-moe)
[Qwen3.6](/models/qwen/qwen3-6)
[Qwen3.5-35B-A3B](/models/qwen/qwen3-5-moe)
[Qwen3.5-4B / 9B / 27B](/models/qwen/qwen3-5) | | **GLM** | [GLM-5.2](/models/glm/glm5-2)
[GLM-5.1](/models/glm/glm5)
[GLM-5](/models/glm/glm5)
[GLM-4.7-Flash](/models/glm/glm4-7-flash) | -| **Kimi** | [Kimi-K2.6](/models/kimi/kimi-k2.5)
[Kimi-K2.5](/models/kimi/kimi-k2.5) | +| **Kimi** | [Kimi-K3](https://github.com/radixark/miles/pull/1825)
[Kimi-K2.6](/models/kimi/kimi-k2.5)
[Kimi-K2.5](/models/kimi/kimi-k2.5) | | **Nemotron** | [Nemotron-3-Ultra-550B-A55B](https://github.com/radixark/miles/blob/main/scripts/run_nemotron_3_ultra_550b_a55b.py)
[Nemotron-3-Super-120B-A12B-FP8](/models/nemotron/nemotron-3-super)
[Nemotron-3-Nano MoE](/models/nemotron/nemotron-3-nano-moe)
[Nemotron-3-Nano](/models/nemotron/nemotron-3-nano) | | **Gemma** | [Gemma-4 26B-A4B](https://github.com/radixark/miles/blob/main/scripts/run_gemma_4_26b_a4b.py)
[Gemma-4 31B](https://github.com/radixark/miles/blob/main/scripts/run_gemma_4_31b.py) | | **JoyAI** | [JoyAI-LLM-Flash](https://github.com/radixark/miles/blob/main/scripts/run_joy_ai_llm_flash.py) |