Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/advanced/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ title: Advanced Features
description: Systems-level features for large-scale and long-running RL.
---
This section covers the Miles features that the Core-features section of the
homepage points at: low-precision training (FP8 / MXFP8 / INT4 QAT), Rollout
Routing Replay for MoE, speculative decoding, and LoRA training and serving.
homepage points at: low-precision training (FP8 / MXFP8 / NVFP4 / INT4 QAT),
Rollout Routing Replay for MoE, fast weight updates over P2P RDMA, fault
tolerance, speculative decoding, and LoRA training and serving.

<CardGroup cols={2}>

Expand Down
76 changes: 52 additions & 24 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,30 +11,58 @@ needed to run RL at trillion-parameter scale.

## Core features

- **Fast and stable support for the latest models.** Day-0 enablement of frontier
releases such as DeepSeek-V4, with rapid follow-on support for new architectures
including GLM-5, Qwen 3.6, and Nemotron-3-Super.
- **Unified low-precision training.** Customizable precision across the rollout and
training engines, with unified **BF16**, **FP8**, **MXFP8**, and **INT4 QAT** recipes
available now and an **NVFP4** training recipe in progress.
- **Efficient Rollout Routing Replay (R3).** For MoE models, expert routing captured
during inference is replayed during the trainer's forward pass, eliminating the
mismatch that destabilizes large-scale MoE RL. Optimized with a routing-result cache
and overlapped device-to-host (D2H) copy to reduce overhead in both single-turn and
multi-turn RL.
- **Speculative rollout with online MTP-SFT.** Miles keeps the draft model's acceptance
rate high through training by fine-tuning MTP layers on-policy.
- **LoRA training and serving.** Both SFT and RL recipes support LoRA adapters,
and the same adapters load directly into SGLang for rollout — no separate
merge or conversion step.
- **Native agentic rollout.** Tool use, multi-turn dialogue, search, code
execution, and multi-agent co-evolution are all supported through clean Python
extension points.
- **Minimal core, maximal extension.** Twenty-plus plug-points let you replace the
rollout, reward, loss, or filter without forking the trainer.
- **Broad hardware support.** First-class on NVIDIA Hopper (H100, H200) and
Blackwell (B100, B200, GB200, GB300), with AMD MI300X / MI325 / MI350 /
MI355X also supported via ROCm.
### Efficiency & stability

- **Fully async RL.** Rollout and training workers are decoupled, with configurable
on- and off-policy schedules, an optimized pipeline with fewer bubbles, and
customizable async rollout and eval modes. See
[Fully Async Rollout](/user-guide/fully-async).
- **Fast agentic rollout.** High-throughput generation on
[SGLang](https://github.com/sgl-project/sglang), optimized for multi-turn
agentic workloads.
- **Fast weight updates.** Updated weights sync back in-loop without pausing
rollout — under 10 seconds for a model with 1 T parameters — with
[P2P RDMA](/advanced/p2p-weight-transfer) as a fast path for disaggregated setups.
- **Unified low-precision training.** [MXFP8 and NVFP4](/advanced/fp8-low-precision)
training with a numerically stable RL recipe that reduces precision-induced
divergence; FP8, [INT4 QAT](/advanced/int4-qat), BF16, and FP16 are also supported.
- **Token-in-token-out (TITO).** Supported for
[all models and all black-box agent harnesses](/user-guide/agentic-chat-template) —
no detokenize/retokenize round-trips between rollout and training.
- **Rollout Routing Replay (R3).** Expert routing recorded during rollout is
[replayed in the trainer's forward pass](/advanced/miles-router), eliminating the
routing mismatch that destabilizes large-scale MoE RL, with compute and
communication overlapped to minimize overhead.
- **LoRA and multi-LoRA.** [Low-rank adapters](/advanced/lora) train frontier-scale
models on a fraction of the GPUs, and the same adapters load directly into SGLang
for rollout — no separate merge or conversion step.
- **Fault tolerance.** When an SGLang engine dies, Miles
[recovers it and resumes the run in place](/advanced/fault-tolerance) — no
restart, no pause.
- **Day-0 model support.** Day-0 enablement of frontier releases such as
DeepSeek-V4, Kimi-K3, GLM-5.2, Inkling, and Nemotron — and beyond day-0, nearly
all frontier models (see [Supported models](#supported-models)).

### Design, support & user experience

- **Coding-agent sandboxes and examples.** [Harbor](/user-guide/harbor),
[OpenEnv](/user-guide/openenv), and [NeMo-Gym](/user-guide/nemo-gym) integrations,
running local CPU sandboxes or per-episode sandboxes on
[Daytona](https://www.daytona.io/), [E2B](https://e2b.dev/), and self-hosted
[AgentENV](https://github.com/kvcache-ai/AgentENV) — see
[Environments](/user-guide/environments) for the support matrix.
- **Highly customizable pipeline.** Shape every workload through
[twenty-plus plug-points](/user-guide/customization), from reward computation to
the full rollout function.
- **Megatron or FSDP.**
[Switch training backends](/developer/experimental-features#fsdp-backend) without
rewriting your training loop.
- **Wide recipe support.** RL (GRPO, PPO), SFT, and on-policy distillation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioned more algorithms here?

- **Verified on multiple hardware generations.** GB300, GB200, B200, H200, H100,
Comment thread
Shi-Dong marked this conversation as resolved.
Resolved
and AMD MI355X — see [Platforms](/platforms/index).
- **[Miles dashboard](/user-guide/dashboard).** A self-hosted web UI for a run's
Comment thread
Shi-Dong marked this conversation as resolved.
Resolved
training dynamics and compute efficiency: what every GPU was doing during a step,
and what each trajectory contained at the token level.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like fully-async scheduling and token info for each trajectory ?


## Supported models

Expand Down
Loading