diff --git a/docs/advanced/index.md b/docs/advanced/index.md index 36e0badb8b1..42bc8c0dff4 100644 --- a/docs/advanced/index.md +++ b/docs/advanced/index.md @@ -3,8 +3,9 @@ title: Advanced Features description: Systems-level features for large-scale and long-running RL. --- This section covers the Miles features that the Core-features section of the -homepage points at: low-precision training (FP8 / MXFP8 / INT4 QAT), Rollout -Routing Replay for MoE, speculative decoding, and LoRA training and serving. +homepage points at: low-precision training (FP8 / MXFP8 / NVFP4 / INT4 QAT), +Rollout Routing Replay for MoE, fast weight updates over P2P RDMA, fault +tolerance, speculative decoding, and LoRA training and serving. diff --git a/docs/index.md b/docs/index.md index bb020b5ac3e..dd25c531f8a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -11,30 +11,61 @@ needed to run RL at trillion-parameter scale. ## Core features -- **Fast and stable support for the latest models.** Day-0 enablement of frontier - releases such as DeepSeek-V4, with rapid follow-on support for new architectures - including GLM-5, Qwen 3.6, and Nemotron-3-Super. -- **Unified low-precision training.** Customizable precision across the rollout and - training engines, with unified **BF16**, **FP8**, **MXFP8**, and **INT4 QAT** recipes - available now and an **NVFP4** training recipe in progress. -- **Efficient Rollout Routing Replay (R3).** For MoE models, expert routing captured - during inference is replayed during the trainer's forward pass, eliminating the - mismatch that destabilizes large-scale MoE RL. Optimized with a routing-result cache - and overlapped device-to-host (D2H) copy to reduce overhead in both single-turn and - multi-turn RL. -- **Speculative rollout with online MTP-SFT.** Miles keeps the draft model's acceptance - rate high through training by fine-tuning MTP layers on-policy. -- **LoRA training and serving.** Both SFT and RL recipes support LoRA adapters, - and the same adapters load directly into SGLang for rollout — no separate - merge or conversion step. -- **Native agentic rollout.** Tool use, multi-turn dialogue, search, code - execution, and multi-agent co-evolution are all supported through clean Python - extension points. -- **Minimal core, maximal extension.** Twenty-plus plug-points let you replace the - rollout, reward, loss, or filter without forking the trainer. -- **Broad hardware support.** First-class on NVIDIA Hopper (H100, H200) and - Blackwell (B100, B200, GB200, GB300), with AMD MI300X / MI325 / MI350 / - MI355X also supported via ROCm. +### Efficiency & stability + +- **Fully async RL.** Rollout and training workers are decoupled, with configurable + on- and off-policy schedules, an optimized pipeline with fewer bubbles, and + customizable async rollout and eval modes. See + [Fully Async Rollout](/user-guide/fully-async). +- **Fast agentic rollout.** High-throughput generation on + [SGLang](https://github.com/sgl-project/sglang), optimized for multi-turn + agentic workloads. +- **Fast weight updates.** Updated weights sync back in-loop without pausing + rollout — under 10 seconds for a model with 1 T parameters — with + [P2P RDMA](/advanced/p2p-weight-transfer) as a fast path for disaggregated setups. +- **Unified low-precision training.** [MXFP8 and NVFP4](/advanced/fp8-low-precision) + training with a numerically stable RL recipe that reduces precision-induced + divergence; FP8, [INT4 QAT](/advanced/int4-qat), BF16, and FP16 are also supported. +- **Token-in-token-out (TITO).** Supported for + [all models and all black-box agent harnesses](/user-guide/agentic-chat-template) — + no detokenize/retokenize round-trips between rollout and training. +- **Rollout Routing Replay (R3).** Expert routing recorded during rollout is + [replayed in the trainer's forward pass](/advanced/miles-router), eliminating the + routing mismatch that destabilizes large-scale MoE RL, with compute and + communication overlapped to minimize overhead. +- **LoRA and multi-LoRA.** [Low-rank adapters](/advanced/lora) train frontier-scale + models on a fraction of the GPUs, and the same adapters load directly into SGLang + for rollout — no separate merge or conversion step. +- **Fault tolerance.** When an SGLang engine dies, Miles + [recovers it and resumes the run in place](/advanced/fault-tolerance) — no + restart, no pause. +- **Day-0 model support.** Day-0 enablement of frontier releases such as + DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron — and beyond day-0, nearly + all frontier models (see [Supported models](#supported-models)). + +### Design, support & user experience + +- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor), + [OpenEnv](/user-guide/openenv), and [NeMo-Gym](/user-guide/nemo-gym) integrations, + running local CPU sandboxes or per-episode sandboxes on + [Daytona](https://www.daytona.io/), [E2B](https://e2b.dev/), and self-hosted + [AgentENV](https://github.com/kvcache-ai/AgentENV) — see + [Environments](/user-guide/environments) for the support matrix. +- **Highly customizable pipeline.** Shape every workload through + [twenty-plus plug-points](/user-guide/customization), from reward computation to + the full rollout function. +- **Megatron or FSDP.** + [Switch training backends](/developer/experimental-features#fsdp-backend) without + rewriting your training loop. +- **Wide recipe support.** RL (GRPO, PPO), SFT, and on-policy distillation. +- **Verified on multiple hardware generations.** GB300, GB200, B300, B200, H200, + H100, and AMD MI355X / MI300X — see [Platforms](/platforms/index). +- **Comprehensive CI.** Unit suites run on every pull request, and tag-triggered + end-to-end GPU training tests cover the supported model families on both NVIDIA + and AMD runners. +- **[Miles dashboard](/user-guide/dashboard).** A self-hosted web UI for a run's + training dynamics and compute efficiency: what every GPU was doing during a step, + and what each trajectory contained at the token level. ## Supported models