diff --git a/README.md b/README.md index b400b4f9b..7174c2b3a 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ [中文版](./README_zh.md) · [Repository](https://github.com/vllm-project/vime) +[![Documentation](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](https://docs.vllm.ai/projects/vime/en/latest/) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/vllm-project/vime) **Vime** is an LLM post-training framework for RL scaling, built on [slime](https://github.com/THUDM/slime). It keeps slime's training stack and data-generation design while using [**vLLM**](https://github.com/vllm-project/vllm) (with [vllm-router](https://github.com/vllm-project/router)) as the default rollout backend. Vime provides two core capabilities: diff --git a/README_zh.md b/README_zh.md index f2f6721fc..3cf742186 100644 --- a/README_zh.md +++ b/README_zh.md @@ -2,6 +2,7 @@ [English](./README.md) · [代码仓库](https://github.com/vllm-project/vime) +[![文档](https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat)](https://docs.vllm.ai/projects/vime/zh-cn/latest/) [![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/vllm-project/vime) **Vime** 是基于 [slime](https://github.com/THUDM/slime) 的 RL scaling 用 LLM post-training 框架。在保留 slime 训练栈与数据生成设计的同时，默认以 [**vLLM**](https://github.com/vllm-project/vllm)（配合 [vllm-router](https://github.com/vllm-project/router)）作为 rollout 后端。Vime 提供两大核心能力： diff --git a/docs/_static/image/logo.ico b/docs/_static/image/logo.ico index 78153d1e2..a41978640 100644 Binary files a/docs/_static/image/logo.ico and b/docs/_static/image/logo.ico differ diff --git a/docs/_static/image/logo.jpg b/docs/_static/image/logo.jpg index c1257d72b..22db26759 100644 Binary files a/docs/_static/image/logo.jpg and b/docs/_static/image/logo.jpg differ diff --git a/docs/en/advanced/vllm-config.md b/docs/en/advanced/vllm-config.md index 21b7975c6..08c7c7c6b 100644 --- a/docs/en/advanced/vllm-config.md +++ b/docs/en/advanced/vllm-config.md @@ -289,8 +289,8 @@ You can configure the routing policy: ```bash --router-policy round_robin # Simple round-robin ---router-policy consistent_hash # Session affinity for multi-turn ---router-policy cache_aware # Cache-aware routing (default) +--router-policy consistent_hash # Session affinity for multi-turn (default) +--router-policy cache_aware # Cache-aware routing ``` ### Session-Affinity Routing for Multi-Turn Agents diff --git a/docs/en/developer_guide/profiling.md b/docs/en/developer_guide/profiling.md index 4db02cf8b..cfa77e2f1 100644 --- a/docs/en/developer_guide/profiling.md +++ b/docs/en/developer_guide/profiling.md @@ -124,8 +124,8 @@ python tools/profile_rollout.py \ While `sleep_rollout` is waiting: 1. `profile_rollout.py --action start` -2. Send a few completion requests to the router or **directly to a worker** (2–4 is enough; traces get large) -3. (Optional) `profile_rollout.py --action stop`; or wait for `max_iterations` to auto-flush +2. Send a few completion requests to the router or **directly to a worker** (2-4 is usually enough; traces get large) +3. If relying on auto-flush, remember that `max_iterations` stops after `> N` steps. For example, `max_iterations=3` needs 4 requests; otherwise call `profile_rollout.py --action stop` manually. 4. Inspect traces under `torch_profiler_dir` Example request (`model` is the HF checkpoint path): @@ -162,7 +162,7 @@ python tools/analyze_profile.py --profile-dir /root/logs/vllm_profile --all-rank | Symptom | Fix | |------|------| | `POST /start_profile` 404 | Pass `--vllm-profiler-config` as JSON; restart the job | -| Start OK but empty output dir | Confirm curl hits a worker and returns 200; increase `max_iterations` or send more requests | +| Start OK but empty output dir | Confirm curl hits a worker and returns 200; if `max_iterations=3`, send 4 requests or call `stop_profile` manually | | Router 503 | Confirm the current job's router port; connect directly to a worker | | Slow or timed-out stop | Increase `VLLM_RPC_TIMEOUT`; reduce request count | @@ -290,16 +290,15 @@ run_profiling_session() { echo "=== 1/3 start_profile (all workers via router) ===" python tools/profile_rollout.py --router-url "${router_url}" --action start - echo "=== 2/3 send completions (direct to worker; 3 requests) ===" - for i in 1 2 3; do - curl -sS -X POST "${worker_url}/v1/completions" \ + echo "=== 2/3 send completions (direct to worker; 4 requests so max_iterations=3 can auto-flush) ===" + for i in 1 2 3 4; do + response="$(curl -sS -X POST "${worker_url}/v1/completions" \ -H "Content-Type: application/json" \ - -d "{\"model\":\"${model}\",\"prompt\":\"Hello ${i}\",\"max_tokens\":32}" \ - | head -c 400 - echo + -d "{\"model\":\"${model}\",\"prompt\":\"Hello ${i}\",\"max_tokens\":32}")" + printf '%s\n' "${response:0:400}" done - echo "=== 3/3 list trace files (max_iterations=3 auto-stop; add --action stop if needed) ===" + echo "=== 3/3 list trace files (max_iterations=3 auto-stop uses > N; add --action stop if needed) ===" sleep 2 find "${PROFILE_DIR}" -type f \( -name '*.json*' -o -name 'profiler_out_*' \) | sort echo "Open *.trace.json.gz in https://ui.perfetto.dev/ or run:" diff --git a/docs/en/get_started/qa.md b/docs/en/get_started/qa.md index 53d6ab62c..47f03b0f8 100644 --- a/docs/en/get_started/qa.md +++ b/docs/en/get_started/qa.md @@ -65,4 +65,4 @@ 13. **Gradient becomes NaN or Inf during training.** - You can try setting the `--no-check-for-nan-in-loss-and-grad` flag to skip the corresponding training steps. \ No newline at end of file + You can try setting the `--no-check-for-nan-in-loss-and-grad` flag to skip the corresponding training steps. diff --git a/docs/en/get_started/quick_start.md b/docs/en/get_started/quick_start.md index c9bdac830..5e8d6fdd9 100644 --- a/docs/en/get_started/quick_start.md +++ b/docs/en/get_started/quick_start.md @@ -11,7 +11,7 @@ Since vime may contain temporary patches for vllm/megatron, to avoid potential e **vime** supports multiple NVIDIA GPU hardware platforms: -- **B200 Series**: Fully supported with identical setup steps as H-series GPUs +- **GB200 / GB300 / B200 / 300 Series**: Fully supported with identical setup steps as H-series GPUs - **H-Series (H100/H200)**: Official support with comprehensive CI testing and stable performance **Important Notes**: @@ -20,8 +20,6 @@ Since vime may contain temporary patches for vllm/megatron, to avoid potential e - B-series basic functionality is stable and suitable for development/testing, but currently lacks CI protection - Both hardware platforms use identical installation and startup procedures -- For scenarios where Docker is not convenient, please refer to [build_conda.sh](https://github.com/vllm-project/vime/blob/main/build_conda.sh). - ### Pull and Start Docker Container Please execute the following commands to pull the latest image and start an interactive container: @@ -520,7 +518,7 @@ CUSTOM_ARGS=( ## Multi-Node Training for Large-Scale MOE Models -To start a multi-node task, you need to first start a Ray cluster. On node 0, run: +If you use Ray for multi-node training, one option is to start the cluster as follows: ```bash # Node0 (HEAD) diff --git a/docs/en/get_started/usage.md b/docs/en/get_started/usage.md index 84fafe43f..68a952cc1 100644 --- a/docs/en/get_started/usage.md +++ b/docs/en/get_started/usage.md @@ -31,9 +31,8 @@ Additionally, vime supports Prefill and Decode disaggregation (PD Disaggregation ### Choosing Training Backend -vime supports multiple training backends, which can be selected via the `--train-backend` parameter: - -- `megatron` (default): Uses Megatron-LM as the training backend, supporting efficient training of large-scale models. +vime currently supports Megatron-LM as its training backend for efficient +large-scale model training. ### Loading Megatron @@ -145,7 +144,7 @@ Note: - Before the first training step, vime will synchronize the parameters from Megatron to vLLM. Therefore, the `--hf-checkpoint` does not need to contain the latest training parameters, and you do not need to change the HF checkpoint when resuming training. - By default, vLLM reads the maximum context length from the `config.json` in the Hugging Face checkpoint. You can use the `--vllm-max-model-len` parameter to override this value to support longer inference. - During co-located training and inference, although Megatron and vLLM will offload sequentially, they still need to leave some memory for each other. You need to adjust vLLM's total VRAM usage by reducing `--vllm-gpu-memory-utilization`. - - vime supports passing through vllm-router parameters by adding a `router` prefix to the original parameter name. For example, vllm-router's `--balance-abs-threshold` parameter should be set as `--router-balance-abs-threshold`. Since vllm-router uses cache-aware routing by default, it may cause uneven request distribution. You can set `--router-balance-abs-threshold 0` to force balanced distribution, but this may affect prefix cache hit rate in multi-turn conversation scenarios. + - vime supports passing through vllm-router parameters by adding a `router` prefix to the original parameter name. For example, vllm-router's `--balance-abs-threshold` parameter should be set as `--router-balance-abs-threshold`. vime uses `consistent_hash` routing by default. cache-aware routing is not supported for now. You can set `--router-balance-abs-threshold 0` to force balanced distribution, but this may affect prefix cache hit rate in multi-turn conversation scenarios. For details on some of vLLM's customizations and the principles behind how vime incorporates vLLM, please see the "How to Use vLLM" section. diff --git a/docs/zh/advanced/vllm-config.md b/docs/zh/advanced/vllm-config.md index 2e6e2c9e3..5f83fd25b 100644 --- a/docs/zh/advanced/vllm-config.md +++ b/docs/zh/advanced/vllm-config.md @@ -289,8 +289,8 @@ python train.py \ ```bash --router-policy round_robin # 简单轮询 ---router-policy consistent_hash # 多轮会话亲和 ---router-policy cache_aware # 缓存感知路由（默认） +--router-policy consistent_hash # 多轮会话亲和（默认） +--router-policy cache_aware # 缓存感知路由 ``` ### 多轮 Agent 的会话亲和路由 diff --git a/docs/zh/developer_guide/profiling.md b/docs/zh/developer_guide/profiling.md index 7e43793f6..92142f227 100644 --- a/docs/zh/developer_guide/profiling.md +++ b/docs/zh/developer_guide/profiling.md @@ -9,7 +9,7 @@ - 从日志确认router/worker地址 - start_profile - 发送少量推理请求 --（可选）stop_profile；或达到max_iterations后自动落盘 +-（可选）stop_profile；或达到max_iterations后自动写入 trace - 在torch_profiler_dir查看trace文件 @@ -44,10 +44,10 @@ vLLM只有在启动时配置了`--profiler-config`，才会注册`/start_profile |------|------| | `profiler` | `"torch"` 或 `"cuda"` | | `torch_profiler_dir` | trace输出目录（绝对路径） | -| `max_iterations` | worker记录超过N步后自动stop并落盘（条件为`> N`） | +| `max_iterations` | worker记录超过N步后自动stop并写入 trace（条件为`> N`） | | `ignore_frontend` | 建议`true`，仅profile worker，降低前端开销 | -**防止`stop_profile`时RPC超时：** vLLM APIServer与EngineCore/worker之间通过内部RPC通信。手动调用`stop_profile`触发trace落盘可能耗时数分钟，而默认`VLLM_RPC_TIMEOUT`仅**10秒**（10000 ms），容易导致flush中断或trace不完整。Profiling时建议设为**30分钟**（1800000 ms）。 +**防止`stop_profile`时RPC超时：** vLLM APIServer与EngineCore/worker之间通过内部RPC通信。手动调用`stop_profile`把 trace 写出来可能耗时数分钟，而默认`VLLM_RPC_TIMEOUT`仅**10秒**（10000 ms），容易导致flush中断或trace不完整。Profiling时建议设为**30分钟**（1800000 ms）。 该变量须在**启动train、拉起vLLM之前**传入Ray worker环境（仅在本机shell `export`不一定会进入Ray job）。在`ray job submit`的`runtime-env-json`中写入，例如： @@ -111,7 +111,7 @@ python tools/profile_rollout.py \ ### 停止Profiling（可选） -若在`--vllm-profiler-config`中设置了`max_iterations`，worker在记录足够步数后会**自动stop并落盘**，实践中发完推理后常可直接在`torch_profiler_dir`看到trace，**不必**再手动`stop_profile`。需要提前结束采集时再执行： +若在`--vllm-profiler-config`中设置了`max_iterations`，worker在记录足够步数后会**自动stop并写入 trace**，实践中发完推理后常可直接在`torch_profiler_dir`看到trace，**不必**再手动`stop_profile`。需要提前结束采集时再执行： ```bash python tools/profile_rollout.py \ @@ -124,8 +124,8 @@ python tools/profile_rollout.py \ 在sleep_rollout等待期间，执行步骤如下： 1. `profile_rollout.py --action start` -2. 向router或**直连worker**发送少量completion请求（2～4条即可，trace会很大） -3. （可选）`profile_rollout.py --action stop`；或等待`max_iterations`触发自动落盘 +2. 向router或**直连worker**发送少量completion请求（通常2～4条即可，trace会很大） +3. 如果依赖自动写入 trace，要注意 `max_iterations` 的停止条件是 `> N`。例如 `max_iterations=3` 时，需要发 4 条请求；否则请手动执行 `profile_rollout.py --action stop` 4. 在`torch_profiler_dir`查看trace 请求示例（`model`使用HF checkpoint路径）： @@ -162,7 +162,7 @@ python tools/analyze_profile.py --profile-dir /root/logs/vllm_profile --all-rank | 现象 | 处理 | |------|------| | `POST /start_profile` 404 | 用JSON传`--vllm-profiler-config`；重启job | -| start成功但目录为空 | 确认curl打到worker且返回200；适当增大`max_iterations`或补发推理 | +| start成功但目录为空 | 确认curl打到worker且返回200；若 `max_iterations=3`，请发 4 条请求，或手动执行 `stop_profile` | | router 503 | 确认当前job的router端口；改直连worker | | stop很慢或超时 | 增大`VLLM_RPC_TIMEOUT`；减少请求条数 | @@ -290,16 +290,15 @@ run_profiling_session() { echo "=== 1/3 start_profile (all workers via router) ===" python tools/profile_rollout.py --router-url "${router_url}" --action start - echo "=== 2/3 send completions (direct to worker; 3 requests) ===" - for i in 1 2 3; do - curl -sS -X POST "${worker_url}/v1/completions" \ + echo "=== 2/3 send completions (direct to worker; 4 requests so max_iterations=3 can auto-flush) ===" + for i in 1 2 3 4; do + response="$(curl -sS -X POST "${worker_url}/v1/completions" \ -H "Content-Type: application/json" \ - -d "{\"model\":\"${model}\",\"prompt\":\"Hello ${i}\",\"max_tokens\":32}" \ - | head -c 400 - echo + -d "{\"model\":\"${model}\",\"prompt\":\"Hello ${i}\",\"max_tokens\":32}")" + printf '%s\n' "${response:0:400}" done - echo "=== 3/3 list trace files (max_iterations=3 auto-stop; add --action stop if needed) ===" + echo "=== 3/3 list trace files (max_iterations=3 auto-stop uses > N; add --action stop if needed) ===" sleep 2 find "${PROFILE_DIR}" -type f \( -name '*.json*' -o -name 'profiler_out_*' \) | sort echo "Open *.trace.json.gz in https://ui.perfetto.dev/ or run:" diff --git a/docs/zh/get_started/quick_start.md b/docs/zh/get_started/quick_start.md index 8078ccb0f..7a1b76cdd 100644 --- a/docs/zh/get_started/quick_start.md +++ b/docs/zh/get_started/quick_start.md @@ -10,7 +10,7 @@ **vime** 支持多种 NVIDIA GPU 硬件平台： -- **B200 系列**：完全支持，运行步骤与 H 系列完全相同 +- **GB200 / GB300 / B200 / 300 系列**：完全支持，运行步骤与 H 系列完全相同 - **H 系列 (H100/H200)**：官方支持，具有完整的 CI 测试保护，运行稳定可靠 **重要说明**： @@ -19,8 +19,6 @@ - B 卡基本功能稳定，可作为开发和测试参考，但暂无 CI 保护 - 两种硬件平台使用完全相同的安装和启动流程 -- 对于不方便使用 docker 的场景，请参考 [build_conda.sh](https://github.com/vllm-project/vime/blob/main/build_conda.sh)。 - ### 拉取并启动 Docker 容器 请执行以下命令，拉取最新镜像并启动一个交互式容器： @@ -525,7 +523,7 @@ CUSTOM_ARGS=( ## 大规模 MOE 模型的多机训练 -为了启动多机任务，首先需要启动一个 ray 集群，即在 node 0 运行： +如果使用 Ray 进行多机训练，可以参考下面的方式启动集群： ```bash # Node0（HEAD） @@ -536,7 +534,7 @@ ray start --head --node-ip-address ${MASTER_ADDR} \ ray start --address=${MASTER_ADDR}:6379 --num-gpus 8 ``` -在 ray 集群启动后，可以在 node 0 提交任务，例如： +在 Ray 集群启动后，可以在 node 0 提交任务，例如： ```bash ray job submit --address="http://127.0.0.1:8265" \ diff --git a/docs/zh/get_started/usage.md b/docs/zh/get_started/usage.md index 6d6c63b66..1c8e50836 100644 --- a/docs/zh/get_started/usage.md +++ b/docs/zh/get_started/usage.md @@ -33,9 +33,7 @@ ### 选择训练后端 -vime 支持多种训练后端，可以通过 `--train-backend` 参数进行选择： - -- `megatron`（默认）：使用 Megatron-LM 作为训练后端，支持大规模模型的高效训练。 +vime 当前使用 Megatron-LM 作为训练后端，用于支持大规模模型的高效训练。 ### 加载 megatron @@ -148,7 +146,7 @@ vLLM 的加载非常简单，只需要： - 在第一个训练步之前，vime 会把 megatron 里的参数同步给 vLLM，所以 `--hf-checkpoint` 中不需要有最新的训练参数，在续训的时候也不需要更换 hf ckpt； - vLLM 默认会从 huggingface ckpt 中 `config.json` 读取模型的最大 context length，可以使用 `--vllm-max-model-len` 参数来对这个值进行覆盖，从而支持进行更长的推理； - 在训推一体的训练过程中，虽然 megatron 和 vLLM 会先后 offload，但是还是需要为对方留有一些空间，需要通过减小 `--vllm-gpu-memory-utilization` 来调整 vLLM 的显存占用总量。 -- vime 支持透传 vllm-router 的参数，方式是在原参数名前加上 `router` 前缀。例如，vllm-router 的 `--balance-abs-threshold` 参数需要设置为 `--router-balance-abs-threshold`。由于 vllm-router 默认使用 cache-aware routing，可能会导致请求分配不均衡的问题。可以通过设置 `--router-balance-abs-threshold 0` 来强制均衡分配，但这可能会影响多轮对话场景下 prefix cache 的命中率。 +- vime 支持透传 vllm-router 的参数，方式是在原参数名前加上 `router` 前缀。例如，vllm-router 的 `--balance-abs-threshold` 参数需要设置为 `--router-balance-abs-threshold`。vime 默认使用 `consistent_hash` 路由策略。暂时不支持 cache-aware routing。可以通过设置 `--router-balance-abs-threshold 0` 来强制均衡分配，但这可能会影响多轮对话场景下 prefix cache 的命中率。 对于一些 vLLM 的自定义以及 vime 引入 vLLM 的原理，请见 vLLM 使用方法一节。 diff --git a/tools/profile_rollout.py b/tools/profile_rollout.py index c3802d2eb..0709e554a 100644 --- a/tools/profile_rollout.py +++ b/tools/profile_rollout.py @@ -14,18 +14,10 @@ def get_workers(router_url): return [] -def start_profile(worker_url, args): - payload = { - "output_dir": args.output_dir, - "num_steps": args.num_steps, - "activities": args.activities, - "profile_by_stage": args.profile_by_stage, - "with_stack": args.with_stack, - "record_shapes": args.record_shapes, - } +def start_profile(worker_url): try: - print(f"Starting profile on {worker_url} for {args.num_steps} steps...") - response = requests.post(f"{worker_url}/start_profile", json=payload) + print(f"Starting profile on {worker_url}...") + response = requests.post(f"{worker_url}/start_profile", json={}) response.raise_for_status() print(f"Successfully started profile on {worker_url}") except Exception as e: @@ -46,12 +38,6 @@ def main(): parser = argparse.ArgumentParser(description="Automate vLLM profiling across all workers via router.") parser.add_argument("--router-url", type=str, required=True, help="Router URL (e.g., http://127.0.0.1:3000)") parser.add_argument("--action", type=str, choices=["start", "stop"], default="start", help="Action to perform") - parser.add_argument("--output-dir", type=str, default="/tmp/vllm_profile", help="Output directory for traces") - parser.add_argument("--num-steps", type=int, default=3, help="Number of steps to profile (default: 3)") - parser.add_argument("--activities", type=str, nargs="+", default=["GPU"], help="Activities to profile (CPU, GPU)") - parser.add_argument("--profile-by-stage", action="store_true", help="Profile by stage (prefill/decode)") - parser.add_argument("--with-stack", action="store_true", help="Record call stack") - parser.add_argument("--record-shapes", action="store_true", help="Record tensor shapes") args = parser.parse_args() @@ -68,7 +54,7 @@ def main(): continue if args.action == "start": - start_profile(worker_url, args) + start_profile(worker_url) else: stop_profile(worker_url)