Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/en/advanced/pd-disaggregation.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# PD Disaggregation

slime supports Prefill and Decode disaggregation (PD Disaggregation).
vime supports Prefill and Decode disaggregation (PD Disaggregation).

You can set the number of servers used for Prefill by setting the `--prefill-num-servers` argument.

We recommand using PD Disaggregation for multi-turn/agentic RL training.
We recommand using PD Disaggregation for multi-turn/agentic RL training, where rollouts tend to produce long contexts and decode-heavy workloads that benefit from separating prefill and decode resources.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There is a typo in this line: "recommand" should be "recommend".

Suggested change
We recommand using PD Disaggregation for multi-turn/agentic RL training, where rollouts tend to produce long contexts and decode-heavy workloads that benefit from separating prefill and decode resources.
We recommend using PD Disaggregation for multi-turn/agentic RL training, where rollouts tend to produce long contexts and decode-heavy workloads that benefit from separating prefill and decode resources.

10 changes: 5 additions & 5 deletions docs/en/developer_guide/trace.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Trace Viewer

slime can attach lightweight execution traces to each rollout sample. These traces capture span-style events such as generation and reward-model calls, and they can be inspected later from a saved rollout debug dump.
vime can attach lightweight execution traces to each rollout sample. These traces capture span-style events such as generation and reward-model calls, and they can be inspected later from a saved rollout debug dump.

![trace timeline viewer](../../_static/image/trace.png)

Expand Down Expand Up @@ -41,7 +41,7 @@ By default it also starts a local static server so you can open the generated HT

## Instrument custom code

For custom rollout or reward code, reuse helpers from `slime.utils.trace_utils`:
For custom rollout or reward code — including custom agent steps, tool calls, sandbox execution, and verifier calls in agentic workflows — reuse helpers from `vime.utils.trace_utils`:

- `trace_span(target, name, attrs=...)`: record a duration span.
- `trace_event(target, name, attrs=...)`: record an instant event.
Expand All @@ -54,10 +54,10 @@ Use `trace_span(...)` when you only want to trace part of a function body, or wh

Use `trace_function(...)` when the whole function should be represented as one span. Internally it resolves the trace target and then opens a `trace_span(...)` around the function call, so it works for both sync and async functions.

The decorator is what slime uses for the main rollout pipeline. For example, `generate_and_rm(...)` is traced per sample and `generate_and_rm_group(...)` is traced per sample group:
The decorator is what vime uses for the main rollout pipeline. For example, `generate_and_rm(...)` is traced per sample and `generate_and_rm_group(...)` is traced per sample group:

```python
from slime.utils.trace_utils import trace_function
from vime.utils.trace_utils import trace_function


@trace_function("generate_and_rm", target="sample")
Expand Down Expand Up @@ -104,7 +104,7 @@ If you need to add attrs after part of the function has executed, use an inner `
For per-turn attrs around an HTTP call, wrap it in `trace_span` directly:

```python
from slime.utils.trace_utils import trace_span
from vime.utils.trace_utils import trace_span

with trace_span(sample, "vllm_generate", attrs={"max_tokens": params["max_new_tokens"]}):
output = await post(url, payload)
Expand Down
31 changes: 24 additions & 7 deletions docs/en/get_started/customization.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Customization Guide

slime provides extensive customization capabilities through function path arguments. These allow you to inject custom logic at various stages of the training and rollout pipeline without modifying the core codebase.
vime provides extensive customization capabilities through function path arguments. These allow you to inject custom logic at various stages of the training and rollout pipeline without modifying the core codebase.

## Overview of Customization Interfaces

Expand Down Expand Up @@ -29,11 +29,28 @@ Below is a summary of all available customization interfaces and their purposes.
| [`--custom-megatron-before-log-prob-hook-path`](#17-megatron-hooks) | Custom logic before log probability computation. |
| [`--custom-megatron-before-train-step-hook-path`](#17-megatron-hooks) | Custom logic before each training step. |

## Agentic workflows through customization interfaces

Agentic workflows — multi-turn tool use, sandbox interaction, environment feedback, verifier/test-based rewards — are an important class of data generation workflows. They plug into vime through the existing customization interfaces; vime does not require a separate agent framework.

For most agentic use cases, **start with `--custom-generate-function-path` plus `--custom-rm-path`**, and only override the full rollout function when the default rollout loop is insufficient.

| If you need to … | Use |
| :--- | :--- |
| Run a custom agent loop, tool calls, RAG, sandbox execution, browser/terminal interaction, or multi-turn generation for each sample, while reusing vime's default rollout loop | [`--custom-generate-function-path`](#2-custom-generate-function---custom-generate-function-path) |
| Compute verifier rewards, test-based rewards, environment success checks, rule-based rewards, or call an external reward service | [`--custom-rm-path`](#3-reward-model---custom-rm-path) |
| Replace the entire rollout orchestration (only when per-sample customization is not enough) | [`--rollout-function-path`](#1-rollout-function---rollout-function-path) |
| Control task sampling, buffering, requeueing, or custom prompt/task sources | [`--data-source-path`](#15-data-source---data-source-path) |
| Attach custom loss masks, metadata, or convert agentic outputs into training data | [`--rollout-data-postprocess-path`](#8-rollout-data-postprocess---rollout-data-postprocess-path), [`--custom-convert-samples-to-train-data-path`](#13-samples-to-train-data-conversion---custom-convert-samples-to-train-data-path) |
| Debug long-running custom generation, verifier calls, tool calls, or sandbox steps | trace utilities in [`vime.utils.trace_utils`](../developer_guide/trace.md) |

Native examples of this pattern: [`examples/multi_agent`](../../../examples/multi_agent/README.md) (a `--rollout-function-path`-based multi-agent pattern) and [`examples/fully_async`](../../../examples/fully_async/README.md) (long-tail agentic generation), both keeping vime's default `vllm_rollout` outer loop.

## Detailed Interface Reference

### 1. Rollout Function (`--rollout-function-path`)

**Default**: `slime.rollout.vllm_rollout.generate_rollout`
**Default**: `vime.rollout.vllm_rollout.generate_rollout`

**Purpose**: Override the entire rollout generation logic.

Expand Down Expand Up @@ -127,7 +144,7 @@ class DynamicFilterOutput:
- Implementing curriculum learning strategies
- Quality-based sample selection

**Example**: `slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std`
**Example**: `vime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std`

---

Expand Down Expand Up @@ -329,11 +346,11 @@ def log_eval_rollout_data(rollout_id, args, data, extra_metrics) -> bool

### 15. Data Source (`--data-source-path`)

**Default**: `slime.rollout.data_source.RolloutDataSourceWithBuffer`
**Default**: `vime.rollout.data_source.RolloutDataSourceWithBuffer`

**Purpose**: Override the data source for rollout prompts.

**Base Class**: `slime.rollout.data_source.DataSource`
**Base Class**: `vime.rollout.data_source.DataSource`

**Required Methods**:
```python
Expand Down Expand Up @@ -406,11 +423,11 @@ Stabilize MoE RL training by recording and replaying expert routing decisions to
| Argument | Description |
| --- | --- |
| `--use-routing-replay` | Forward-backward routing consistency in training. ([arXiv:2507.18071](https://arxiv.org/abs/2507.18071)) |
| `--use-rollout-routing-replay` | R3: Replay routing from rollout during training. Supported by slime's default `vllm_rollout` path. ([arXiv:2510.11370](https://arxiv.org/abs/2510.11370)) |
| `--use-rollout-routing-replay` | R3: Replay routing from rollout during training. Supported by vime's default `vllm_rollout` path. ([arXiv:2510.11370](https://arxiv.org/abs/2510.11370)) |

## Testing Custom Function Paths

slime also provides CPU-only contract tests for customization interfaces. These tests resolve components through import-path strings, so they can validate both built-in hooks and user-defined implementations passed through the same CLI arguments used by training.
vime also provides CPU-only contract tests for customization interfaces. These tests resolve components through import-path strings, so they can validate both built-in hooks and user-defined implementations passed through the same CLI arguments used by training.

The tests live under `tests/plugin_contracts/` and are grouped by hook shape:

Expand Down
4 changes: 2 additions & 2 deletions docs/zh/advanced/pd-disaggregation.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# PD 分离

slime 支持 Prefill 和 Decode 的分离部署 (PD Disaggregation)。
vime 支持 Prefill 和 Decode 的分离部署 (PD Disaggregation)。

可以通过设置 `--prefill-num-servers` 参数来指定用于 Prefill 的服务器数量。

我们推荐在多轮或 agentic RL 训练中开启 PD 分离。
我们推荐在多轮或 agentic RL 训练中开启 PD 分离:这类 rollout 通常会产生较长 context 与 decode-heavy 的负载,更适合把 prefill 与 decode 拆开部署
10 changes: 5 additions & 5 deletions docs/zh/developer_guide/trace.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Trace 可视化

slime 可以为每条 rollout sample 挂上轻量级执行 trace。它会记录生成、奖励模型等 span 事件,并且可以在保存下来的 rollout debug dump 中离线查看。
vime 可以为每条 rollout sample 挂上轻量级执行 trace。它会记录生成、奖励模型等 span 事件,并且可以在保存下来的 rollout debug dump 中离线查看。

![trace 时间线查看器](../../_static/image/trace.png)

Expand Down Expand Up @@ -41,7 +41,7 @@ python tools/trace_timeline_viewer.py /path/to/debug/rollout_0.pt

## 给自定义代码打点

在自定义 rollout 或 reward 逻辑中可以直接复用 `slime.utils.trace_utils` 里的工具:
在自定义 rollout 或 reward 逻辑中——包括 agentic workflow 里的 agent step、tool call、sandbox 执行、verifier 调用等——可以直接复用 `vime.utils.trace_utils` 里的工具:

- `trace_span(target, name, attrs=...)`:记录一段持续时间。
- `trace_event(target, name, attrs=...)`:记录一个瞬时事件。
Expand All @@ -54,10 +54,10 @@ python tools/trace_timeline_viewer.py /path/to/debug/rollout_0.pt

如果整个函数调用都应该对应一个 span,就用 `trace_function(...)`。它本质上会先解析 trace target,然后在函数调用外层自动套一层 `trace_span(...)`,因此同步函数和异步函数都能直接用。

slime 主 rollout 流程里就是这样用的。例如 `generate_and_rm(...)` 按 sample 打点,而 `generate_and_rm_group(...)` 按 group 打点:
vime 主 rollout 流程里就是这样用的。例如 `generate_and_rm(...)` 按 sample 打点,而 `generate_and_rm_group(...)` 按 group 打点:

```python
from slime.utils.trace_utils import trace_function
from vime.utils.trace_utils import trace_function


@trace_function("generate_and_rm", target="sample")
Expand Down Expand Up @@ -104,7 +104,7 @@ async def custom_rollout_batch(samples, **kwargs):
如果想在某个 HTTP 调用周围记录每轮的 attrs,可以直接套一层 `trace_span`:

```python
from slime.utils.trace_utils import trace_span
from vime.utils.trace_utils import trace_span

with trace_span(sample, "vllm_generate", attrs={"max_tokens": params["max_new_tokens"]}):
output = await post(url, payload)
Expand Down
31 changes: 24 additions & 7 deletions docs/zh/get_started/customization.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 自定义指南

slime 通过函数路径参数提供了广泛的自定义能力。这些参数允许你在训练和推理流程的各个阶段注入自定义逻辑,而无需修改核心代码库。
vime 通过函数路径参数提供了广泛的自定义能力。这些参数允许你在训练和推理流程的各个阶段注入自定义逻辑,而无需修改核心代码库。

## 自定义接口概览

Expand Down Expand Up @@ -29,11 +29,28 @@ slime 通过函数路径参数提供了广泛的自定义能力。这些参数
| [`--custom-megatron-before-log-prob-hook-path`](#17-megatron-hook) | log probability 计算前的自定义逻辑。 |
| [`--custom-megatron-before-train-step-hook-path`](#17-megatron-hook) | 每个训练步骤前的自定义逻辑。 |

## 通过 customization 接口实现 agentic workflow

agentic workflow——multi-turn tool use、sandbox interaction、environment feedback、verifier/test-based reward——是一类重要的训练数据生成 workflow。它们通过 vime 已有的 customization 接口接入,vime 本身并不需要变成一个单独的 agent framework。

绝大多数 agentic 场景下,**建议从 `--custom-generate-function-path` 加 `--custom-rm-path` 开始**,只有在默认 rollout 循环无法满足需求时再去覆盖整个 rollout function。

| 想做的事 | 应使用的接口 |
| :--- | :--- |
| 让每条 sample 跑自定义的 agent loop、tool call、RAG、sandbox 执行、browser/terminal 交互或多轮生成,同时复用 vime 默认 rollout loop | [`--custom-generate-function-path`](#2-自定义生成函数---custom-generate-function-path) |
| 实现 verifier reward、test-based reward、environment 成功判定、rule-based reward 或调用外部 reward 服务 | [`--custom-rm-path`](#3-奖励模型---custom-rm-path) |
| 替换整个 rollout 编排(只在 per-sample 自定义不够用时使用) | [`--rollout-function-path`](#1-rollout-函数---rollout-function-path) |
| 控制任务采样、缓冲、回填,或自定义 prompt / task 数据源 | [`--data-source-path`](#15-数据源---data-source-path) |
| 给 agentic 输出附加自定义 loss mask、metadata,或转换成训练数据 | [`--rollout-data-postprocess-path`](#8-rollout-数据后处理---rollout-data-postprocess-path)、[`--custom-convert-samples-to-train-data-path`](#13-样本转训练数据---custom-convert-samples-to-train-data-path) |
| 调试长耗时的 custom generation、verifier、tool call 或 sandbox 调用 | [`vime.utils.trace_utils`](../developer_guide/trace.md) 中的 trace 工具 |

这一模式的原生示例:[`examples/multi_agent`](../../../examples/multi_agent/README.md) 中基于 `--rollout-function-path` 的多 agent 模式,以及 [`examples/fully_async`](../../../examples/fully_async/README.md) 中适合 long-tail agentic 场景的 fully-async rollout,两者外层都走 vime 默认的 `vllm_rollout`。

## 详细接口参考

### 1. Rollout 函数 (`--rollout-function-path`)

**默认值**: `slime.rollout.vllm_rollout.generate_rollout`
**默认值**: `vime.rollout.vllm_rollout.generate_rollout`

**用途**: 覆盖整个 rollout 生成逻辑。

Expand Down Expand Up @@ -127,7 +144,7 @@ class DynamicFilterOutput:
- 实现课程学习策略
- 基于质量的样本选择

**示例**: `slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std`
**示例**: `vime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std`

---

Expand Down Expand Up @@ -329,11 +346,11 @@ def log_eval_rollout_data(rollout_id, args, data, extra_metrics) -> bool

### 15. 数据源 (`--data-source-path`)

**默认值**: `slime.rollout.data_source.RolloutDataSourceWithBuffer`
**默认值**: `vime.rollout.data_source.RolloutDataSourceWithBuffer`

**用途**: 覆盖 rollout 提示词的数据源。

**基类**: `slime.rollout.data_source.DataSource`
**基类**: `vime.rollout.data_source.DataSource`

**必需方法**:
```python
Expand Down Expand Up @@ -408,11 +425,11 @@ def custom_hook(args, rollout_id, step_id, model, optimizer, opt_param_scheduler
| 参数 | 说明 |
| --- | --- |
| `--use-routing-replay` | 训练中前向-反向路由一致性。([arXiv:2507.18071](https://arxiv.org/abs/2507.18071)) |
| `--use-rollout-routing-replay` | R3:在训练时重放 rollout 阶段的路由。slime 默认的 `vllm_rollout` 路径支持该功能。([arXiv:2510.11370](https://arxiv.org/abs/2510.11370)) |
| `--use-rollout-routing-replay` | R3:在训练时重放 rollout 阶段的路由。vime 默认的 `vllm_rollout` 路径支持该功能。([arXiv:2510.11370](https://arxiv.org/abs/2510.11370)) |

## 自定义函数路径的测试

slime 现在也提供了一组 CPU 契约测试,用于校验这些 customization 接口。测试会通过字符串形式的导入路径来动态加载组件,因此既能回归仓库内置 hook,也能验证用户通过和训练时完全相同的 CLI 参数传入的自定义实现。
vime 现在也提供了一组 CPU 契约测试,用于校验这些 customization 接口。测试会通过字符串形式的导入路径来动态加载组件,因此既能回归仓库内置 hook,也能验证用户通过和训练时完全相同的 CLI 参数传入的自定义实现。

这些测试统一放在 `tests/plugin_contracts/` 目录下,并按 hook 形态归并成少数几个文件:

Expand Down
Loading