Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/advanced/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ title: Advanced Features
description: Systems-level features for large-scale and long-running RL.
---
This section covers the Miles features that the Core-features section of the
homepage points at: low-precision training (FP8 / MXFP8 / INT4 QAT), Rollout
Routing Replay for MoE, speculative decoding, and LoRA training and serving.
homepage points at: low-precision training (FP8 / MXFP8 / NVFP4 / INT4 QAT),
Rollout Routing Replay for MoE, fast weight updates over P2P RDMA, fault
tolerance, speculative decoding, and LoRA training and serving.

<CardGroup cols={2}>

Expand Down
79 changes: 55 additions & 24 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,30 +11,61 @@ needed to run RL at trillion-parameter scale.

## Core features

- **Fast and stable support for the latest models.** Day-0 enablement of frontier
releases such as DeepSeek-V4, with rapid follow-on support for new architectures
including GLM-5, Qwen 3.6, and Nemotron-3-Super.
- **Unified low-precision training.** Customizable precision across the rollout and
training engines, with unified **BF16**, **FP8**, **MXFP8**, and **INT4 QAT** recipes
available now and an **NVFP4** training recipe in progress.
- **Efficient Rollout Routing Replay (R3).** For MoE models, expert routing captured
during inference is replayed during the trainer's forward pass, eliminating the
mismatch that destabilizes large-scale MoE RL. Optimized with a routing-result cache
and overlapped device-to-host (D2H) copy to reduce overhead in both single-turn and
multi-turn RL.
- **Speculative rollout with online MTP-SFT.** Miles keeps the draft model's acceptance
rate high through training by fine-tuning MTP layers on-policy.
- **LoRA training and serving.** Both SFT and RL recipes support LoRA adapters,
and the same adapters load directly into SGLang for rollout — no separate
merge or conversion step.
- **Native agentic rollout.** Tool use, multi-turn dialogue, search, code
execution, and multi-agent co-evolution are all supported through clean Python
extension points.
- **Minimal core, maximal extension.** Twenty-plus plug-points let you replace the
rollout, reward, loss, or filter without forking the trainer.
- **Broad hardware support.** First-class on NVIDIA Hopper (H100, H200) and
Blackwell (B100, B200, GB200, GB300), with AMD MI300X / MI325 / MI350 /
MI355X also supported via ROCm.
### Efficiency & stability

- **Fully async RL.** Rollout and training workers are decoupled, with configurable
on- and off-policy schedules, an optimized pipeline with fewer bubbles, and
customizable async rollout and eval modes. See
[Fully Async Rollout](/user-guide/fully-async).
- **Fast agentic rollout.** High-throughput generation on
[SGLang](https://github.com/sgl-project/sglang), optimized for multi-turn
agentic workloads.
- **Fast weight updates.** Updated weights sync back in-loop without pausing
rollout — under 10 seconds for a model with 1 T parameters — with
[P2P RDMA](/advanced/p2p-weight-transfer) as a fast path for disaggregated setups.
- **Unified low-precision training.** [MXFP8 and NVFP4](/advanced/fp8-low-precision)
training with a numerically stable RL recipe that reduces precision-induced
divergence; FP8, [INT4 QAT](/advanced/int4-qat), BF16, and FP16 are also supported.
- **Token-in-token-out (TITO).** Supported for
[all models and all black-box agent harnesses](/user-guide/agentic-chat-template) —
no detokenize/retokenize round-trips between rollout and training.
- **Rollout Routing Replay (R3).** Expert routing recorded during rollout is
[replayed in the trainer's forward pass](/advanced/miles-router), eliminating the
routing mismatch that destabilizes large-scale MoE RL, with compute and
communication overlapped to minimize overhead.
- **LoRA and multi-LoRA.** [Low-rank adapters](/advanced/lora) train frontier-scale
models on a fraction of the GPUs, and the same adapters load directly into SGLang
for rollout — no separate merge or conversion step.
- **Fault tolerance.** When an SGLang engine dies, Miles
[recovers it and resumes the run in place](/advanced/fault-tolerance) — no
restart, no pause.
- **Day-0 model support.** Day-0 enablement of frontier releases such as
DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron — and beyond day-0, nearly
all frontier models (see [Supported models](#supported-models)).

### Design, support & user experience

- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor),
[OpenEnv](/user-guide/openenv), and [NeMo-Gym](/user-guide/nemo-gym) integrations,
running local CPU sandboxes or per-episode sandboxes on
[Daytona](https://www.daytona.io/), [E2B](https://e2b.dev/), and self-hosted
[AgentENV](https://github.com/kvcache-ai/AgentENV) — see
[Environments](/user-guide/environments) for the support matrix.
- **Highly customizable pipeline.** Shape every workload through
[twenty-plus plug-points](/user-guide/customization), from reward computation to
the full rollout function.
- **Megatron or FSDP.**
[Switch training backends](/developer/experimental-features#fsdp-backend) without
rewriting your training loop.
- **Wide recipe support.** RL (GRPO, PPO), SFT, and on-policy distillation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned more algorithms here?

- **Verified on multiple hardware generations.** GB300, GB200, B300, B200, H200,
H100, and AMD MI355X / MI300X — see [Platforms](/platforms/index).
- **Comprehensive CI.** Unit suites run on every pull request, and tag-triggered
end-to-end GPU training tests cover the supported model families on both NVIDIA
and AMD runners.
- **[Miles dashboard](/user-guide/dashboard).** A self-hosted web UI for a run's
Comment thread
Shi-Dong marked this conversation as resolved.
training dynamics and compute efficiency: what every GPU was doing during a step,
and what each trajectory contained at the token level.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like fully-async scheduling and token info for each trajectory ?


## Supported models

Expand Down
Loading