Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/add-dynamic-filter/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Use this skill when:

### Step 2: Implement the Function Signature

Dynamic sampling filter (called in `slime/rollout/sglang_rollout.py`):
Dynamic sampling filter (called in `slime/rollout/vllm_rollout.py`):

```python
def filter_function(args, samples, **kwargs):
Expand Down Expand Up @@ -97,5 +97,5 @@ Example wiring:

- Dynamic filter types: `slime/rollout/filter_hub/base_types.py`
- Dynamic filter example: `slime/rollout/filter_hub/dynamic_sampling_filters.py`
- Rollout generation hook points: `slime/rollout/sglang_rollout.py`
- Rollout generation hook points: `slime/rollout/vllm_rollout.py`
- Buffer filter hook point: `slime/rollout/data_source.py`
2 changes: 1 addition & 1 deletion .claude/skills/add-eval-dataset-config/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,5 +87,5 @@ Use a separate eval function when inference/eval behavior must differ from train

- Eval config model: `slime/utils/eval_config.py`
- Eval config resolution: `slime/utils/arguments.py`
- Eval rollout path: `slime/rollout/sglang_rollout.py`
- Eval rollout path: `slime/rollout/vllm_rollout.py`
- Customization docs: `docs/en/get_started/customization.md`
10 changes: 5 additions & 5 deletions .claude/skills/add-rollout-function/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: add-rollout-function
description: Guide for adding a new rollout function in slime and wiring it through --rollout-function-path. Use when user wants to implement custom rollout data generation logic, custom train/eval rollout outputs, or migrate from the default sglang rollout path.
description: Guide for adding a new rollout function in slime and wiring it through --rollout-function-path. Use when user wants to implement custom rollout data generation logic, custom train/eval rollout outputs, or migrate from the default vLLM rollout path.
---

# Add Rollout Function
Expand All @@ -12,7 +12,7 @@ Implement a custom rollout function and integrate it safely with slime training/
Use this skill when:

- User asks to add a new rollout task or rollout generation function
- User asks to replace default `slime.rollout.sglang_rollout.generate_rollout`
- User asks to replace default `slime.rollout.vllm_rollout.generate_rollout`
- User asks to customize train/eval data generation behavior

## Step-by-Step Guide
Expand All @@ -21,10 +21,10 @@ Use this skill when:

Start from one of these references:

- Async RL-style rollout: `slime/rollout/sglang_rollout.py`
- Async RL-style rollout: `slime/rollout/vllm_rollout.py`
- Simple SFT-style rollout: `slime/rollout/sft_rollout.py`

If the task needs engine-based async generation and rewards, use the sglang path as base.
If the task needs engine-based async generation and rewards, use the vLLM path as base.
If the task is file/buffer-driven and simple, use sft path as base.

### Step 2: Create the New Rollout Module
Expand Down Expand Up @@ -100,7 +100,7 @@ The default and signature expectation are documented in:

## Reference Locations

- Default rollout: `slime/rollout/sglang_rollout.py`
- Default rollout: `slime/rollout/vllm_rollout.py`
- Simple custom example: `slime/rollout/sft_rollout.py`
- Output dataclasses: `slime/rollout/base_types.py`
- Wiring/loading: `slime/ray/rollout.py`
Expand Down
90 changes: 0 additions & 90 deletions .github/workflows/conda-ci.yml

This file was deleted.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ We also provide examples for some use cases not covered in the quick start guide
Arguments in Vime are divided into three categories:

1. **Megatron arguments**: Vime reads all arguments in Megatron. You can configure Megatron by passing arguments like `--tensor-model-parallel-size 2`.
2. **vLLM arguments**: vLLM server and engine options are exposed with a `--vllm-` prefix (for example, `--vllm-gpu-memory-utilization`). Router options live under two prefixes: vllm-router's native options are passed with `--router-` (for example, `--router-policy round_robin`), while Vime-side orchestration knobs that tell Vime *where* the router lives use `--vllm-router-` (`--vllm-router-ip`, `--vllm-router-port`, `--vllm-router-request-timeout-secs`). See [slime/backends/vllm_utils/arguments.py](slime/backends/vllm_utils/arguments.py) for the full surface.
2. **vLLM arguments**: vLLM server and engine options are exposed with a `--vllm-` prefix (for example, `--vllm-gpu-memory-utilization`). Router options live under two prefixes: vllm-router's native options are passed with `--router-` (for example, `--router-policy round_robin`, `--router-request-timeout-secs`), while Vime-side orchestration knobs that tell Vime *where* the router lives use `--vllm-router-` (`--vllm-router-ip`, `--vllm-router-port`). See [slime/backends/vllm_utils/arguments.py](slime/backends/vllm_utils/arguments.py) for the full surface.
3. **Framework-specific arguments**: Shared slime/Vime orchestration flags (rollout GPUs, data paths, RL algorithms, etc.). Please refer to [slime/utils/arguments.py](slime/utils/arguments.py).

`--rollout-num-gpus-per-engine` sets the tensor parallel size of each vLLM engine. The default rollout entry is `slime.rollout.vllm_rollout.generate_rollout`.
Expand Down
2 changes: 1 addition & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Vime 继承 slime 的广泛模型支持,包括:
Vime 的参数分为三类:

1. **Megatron 参数**:Vime 会读取 Megatron 中的全部参数,可通过传入如 `--tensor-model-parallel-size 2` 的方式配置 Megatron;
2. **vLLM 参数**:vLLM server 与 engine 相关选项以 `--vllm-` 为前缀(例如 `--vllm-gpu-memory-utilization`)。路由相关选项分两类前缀:vllm-router 自身的选项以 `--router-` 传入(例如 `--router-policy round_robin`),Vime 侧用于告诉 Vime *router 在哪里* 的编排参数则以 `--vllm-router-` 为前缀(`--vllm-router-ip`、`--vllm-router-port`、`--vllm-router-request-timeout-secs`)。完整参数见 [slime/backends/vllm_utils/arguments.py](slime/backends/vllm_utils/arguments.py)。
2. **vLLM 参数**:vLLM server 与 engine 相关选项以 `--vllm-` 为前缀(例如 `--vllm-gpu-memory-utilization`)。路由相关选项分两类前缀:vllm-router 自身的选项以 `--router-` 传入(例如 `--router-policy round_robin`、`--router-request-timeout-secs`),Vime 侧用于告诉 Vime *router 在哪里* 的编排参数则以 `--vllm-router-` 为前缀(`--vllm-router-ip`、`--vllm-router-port`)。完整参数见 [slime/backends/vllm_utils/arguments.py](slime/backends/vllm_utils/arguments.py)。
3. **框架参数**:与 slime/Vime 编排相关的开关(rollout GPU、数据路径、RL 算法等),见 [slime/utils/arguments.py](slime/utils/arguments.py)。

`--rollout-num-gpus-per-engine` 对应每个 vLLM engine 的 tensor parallel size。默认 rollout 入口为 `slime.rollout.vllm_rollout.generate_rollout`。
Expand Down
84 changes: 0 additions & 84 deletions build_conda.sh

This file was deleted.

24 changes: 5 additions & 19 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
ARG BASE_IMAGE=vllm/vllm-openai:v0.21.0-cu129-ubuntu2404
ARG BASE_IMAGE=vllm/vllm-openai:v0.22.0-cu129-ubuntu2404
FROM ${BASE_IMAGE}

# ======================================== Arguments =============================================

ARG PATCH_VERSION=latest
ARG MEGATRON_COMMIT=1dcf0dafa884ad52ffb243625717a3471643e087
ARG SGLANG_VERSION=0.5.10.post1

ARG ENABLE_CUDA_13=0

Expand Down Expand Up @@ -74,29 +73,17 @@ RUN if [ "$ENABLE_CUDA_13" = "1" ]; then \
(cd /root && git clone -b feat/v350_plus_8045 https://github.com/fzyzcjy/triton.git && cd triton && pip install -r python/requirements.txt && pip install --verbose -e .); \
fi

# Skip sglang lines from requirements (we install --no-deps below; the full
# transitive set conflicts with the vllm base environment).
COPY requirements.txt /tmp/requirements.txt
RUN grep -vE '^[[:space:]]*(sglang|sglang-router)([[:space:]]|[<>=!]|$)' /tmp/requirements.txt > /tmp/requirements-vllm.txt && \
pip install --ignore-installed PyJWT && \
pip install -r /tmp/requirements-vllm.txt

# Temporarily install another sgl-kernel version for GB300 without rebuilding the whole image
RUN if [ "$ENABLE_CUDA_13" = "1" ]; then \
SGL_KERNEL_VERSION=0.3.17.post2 && \
python3 -m pip install https://github.com/sgl-project/whl/releases/download/v${SGL_KERNEL_VERSION}/sgl_kernel-${SGL_KERNEL_VERSION}+cu130-cp310-abi3-manylinux2014_$(uname -m).whl --force-reinstall --no-deps; \
fi
RUN pip install --ignore-installed PyJWT && \
pip install -r /tmp/requirements.txt

# https://github.com/pytorch/pytorch/issues/168167
RUN pip install nvidia-cudnn-cu12==9.16.0.29

# reinstall numpy 1.x for megatron
RUN pip install "numpy<2"

# vime's slime/utils/arguments.py + slime/ray/rollout.py top-level import sglang_router;
# the cu129 vllm base does not ship sglang, so install --no-deps stubs here.
RUN pip install --no-deps "sglang==${SGLANG_VERSION}" sglang-router==0.3.2 && \
pip install IPython
RUN pip install IPython

# Pin vllm-router explicitly so the vllm rollout routing layer is a visible build step
# (also in requirements.txt; pulling it here makes the layer cache-able and fail-fast).
Expand Down Expand Up @@ -135,10 +122,9 @@ RUN cd /root/slime/slime/backends/megatron_utils/kernels/int4_qat && \
# Fail-fast import smoke + flashinfer version pin check. Catches ABI / version
# regressions at build time instead of first GPU run.
RUN python3 -c "\
import vllm, sglang, slime, flashinfer; \
import vllm, slime, flashinfer; \
from slime.backends.vllm_utils.vllm_engine import VLLMEngine; \
print('vllm', vllm.__version__); \
print('sglang', sglang.__version__); \
print('flashinfer', flashinfer.__version__); \
print('VLLMEngine import ok')"

Expand Down
12 changes: 8 additions & 4 deletions docker/justfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
release-primary:
ARG_TAG_POSTFIX="" ARG_BUILD_EXTRA_ARGS="" just _release-raw

# Should be executed on ARM machines
# Should be executed on ARM machines.
# Inherits the Dockerfile's default cu129 BASE_IMAGE; that tag is a multi-arch
# manifest, so docker selects the arm64 image automatically on an ARM host.
release-cu129-arm64:
ARG_TAG_POSTFIX="-cu129-arm64" ARG_BUILD_EXTRA_ARGS='--build-arg SGLANG_IMAGE_TAG=v0.5.5.post3-cu129-arm64 --build-arg ENABLE_SGLANG_PATCH=0' just _release-raw
ARG_TAG_POSTFIX="-cu129-arm64" ARG_BUILD_EXTRA_ARGS="" just _release-raw

# Should be executed on ARM machines
# Should be executed on ARM machines.
# The default-CUDA vLLM tag (no cuXXX suffix) already ships CUDA 13.0;
# ENABLE_CUDA_13 then builds the CUDA-13 TransformerEngine/Triton on top.
release-cu13-arm64:
ARG_TAG_POSTFIX="-cu13-arm64" ARG_BUILD_EXTRA_ARGS='--build-arg SGLANG_IMAGE_TAG=dev-arm64-cu13-20251122 --build-arg ENABLE_CUDA_13=1 --build-arg ENABLE_SGLANG_PATCH=0' just _release-raw
ARG_TAG_POSTFIX="-cu13-arm64" ARG_BUILD_EXTRA_ARGS='--build-arg BASE_IMAGE=vllm/vllm-openai:v0.22.0-ubuntu2404 --build-arg ENABLE_CUDA_13=1' just _release-raw

_release-raw:
#!/bin/bash
Expand Down
Loading
Loading