[RL] Add /pull_weights: engine-side pull of published weights into a host-local checkpoint (sglang-miles) - #30366
Merged
yueming-yuan merged 1 commit intoJul 9, 2026
Conversation
nanjiangwill
requested review from
AniZpZ,
BBuf,
CatherineSue,
Edwardf0t1,
FlamingoPg,
Fridge003,
JustinTong0323,
Qiaolin-Yu,
Ying1123,
alphabetc1,
b8zhong,
ch-wan,
fzyzcjy,
hanming-lu,
hnyls2002,
hzh0425,
ispobock,
jybsuper,
key4ng,
kpham-sgl,
lifuhuang,
merrymercy,
mickqian,
pyc96,
slin1237,
xiezhq-hermann,
yhyang201,
yizhang2077,
yuan-luo and
yushengsu-thu
as code owners
July 7, 2026 08:17
nanjiangwill
requested review from
DarkSharpness,
HydraQYH and
celve
as code owners
July 7, 2026 08:17
github-actions
Bot
added
dependencies
This was referenced Jul 7, 2026
Open
Contributor
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
nanjiangwill
force-pushed
the
pull-weights-sglang-miles
branch
2 times, most recently
from
July 7, 2026 18:13
e419790 to
8f0ab41
Compare
…host-local checkpoint A trainer publishes each weight sync as a version directory (a full HF checkpoint, or zstd-compressed per-tensor deltas with checksums) under a shared source_dir. /pull_weights fans out to every rank on every node of the deployment; each host seeds a local checkpoint from the newest full version (or the server's own model path) and applies the delta chain in place, verified per-tensor. The engine then reloads the local checkpoint through the ordinary update_weights_from_disk path, so the weight-loading code never sees the delta format. --custom-pull-weights-pre-read-hook covers object-store-backed shared filesystems that need an explicit refresh before reads. Ported from slime's runtime patch (THUDM/slime#2181) for miles/slime-style disaggregated RL training.
nanjiangwill
force-pushed
the
pull-weights-sglang-miles
branch
from
July 7, 2026 18:22
8f0ab41 to
5de87ba
Compare
yueming-yuan
approved these changes
Jul 7, 2026
nanjiangwill
added a commit
to nanjiangwill/miles
that referenced
this pull request
Jul 8, 2026
…i-weight-update The weights checker (--check-weight-update-equal, armed by --ci-test) resets engine tensors at startup and compares after the first sync, expecting it to rewrite every tensor — but the disk-delta baseline publishes nothing. When the checker is armed, the baseline now also reloads the just-pulled v0 checkpoint via update_weights_from_disk, restoring engine state and setting the weight version the CI equality check expects. Adds tests/e2e/megatron/test_qwen3_4B_disk_delta.py (mirrors the p2p weight-update test: 4 actor + 4 rollout GPUs, --ci-test), registered under labels ["megatron", "weight-update"]; disabled until the CI image ships sglang /pull_weights (sgl-project/sglang#30366).
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 14, 2026
… into a host-local checkpoint (sglang-miles) (#30366)
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 14, 2026
… into a host-local checkpoint (sglang-miles) (#30366)
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 14, 2026
… into a host-local checkpoint (sglang-miles) (#30366)
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 14, 2026
… into a host-local checkpoint (sglang-miles) (#30366)
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 25, 2026
… into a host-local checkpoint (#30366, #28524) Folds in the two fixes #28524 made to this endpoint -- Annotated[..., Body()] on the handler and keyword construction of PullWeightsReqOutput, which is a kw_only msgspec Struct and would raise on the positional form. The rest of #28524 (dumper / dump-comparator) is already upstream in v0.5.16 and is dropped.
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 25, 2026
… into a host-local checkpoint (#30366, #28524) Folds in the two fixes #28524 made to this endpoint -- Annotated[..., Body()] on the handler and keyword construction of PullWeightsReqOutput, which is a kw_only msgspec Struct and would raise on the positional form. The rest of #28524 (dumper / dump-comparator) is already upstream in v0.5.16 and is dropped.
yueming-yuan
pushed a commit
that referenced
this pull request
Jul 25, 2026
… into a host-local checkpoint (#30366, #28524) Folds in the two fixes #28524 made to this endpoint -- Annotated[..., Body()] on the handler and keyword construction of PullWeightsReqOutput, which is a kw_only msgspec Struct and would raise on the positional form. The rest of #28524 (dumper / dump-comparator) is already upstream in v0.5.16 and is dropped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
RL trainers with disaggregated rollout publish each weight sync as a version dir
weight_v{N:06d}/on a shared filesystem — a full HF checkpoint, or zstd-compressed per-tensor byte deltas (xor/overwrite) with per-tensor checksums, packaged as a canonical HF checkpoint dir. This adds the engine-side receiver:POST /pull_weightsbrings a host-local checkpoint up to a target version on every host the deployment spans, so the trainer talks to one endpoint per engine.Trainer PR: radixark/miles#1235 ·
mainvariant: #30367 · Refs: THUDM/slime#2181Behavior
POST /pull_weights {local_checkpoint_dir, source_dir, target_version}fans out to every scheduler rank on every node. Each host seeds from the newest full version ≤ target (or the server's ownmodel_path— version 0 is the engine's base), then applies the delta chain in place via mmap, parallelized across tensors. A checksum mismatch or out-of-order apply raises — never serve bad weights; a per-host flock + applied-version marker collapse co-located ranks to one pull. Success is gathered across the TP group, so the reply covers every host. The trainer then reloads via the ordinary/update_weights_from_disk— weight loading never sees the delta format.--custom-pull-weights-pre-read-hook <import.path>: refresh hook for object-store-backed mounts without cross-host read-after-write consistency (POSIX shared FS needs none).Notes
xxhash+zstandardadded to pyproject;blake3stays an optional lazy import. The module is imported only inside the handler.CI States
Latest PR Test (Base): ❌ Run #28888974230
Latest PR Test (Extra): ❌ Run #28888973534