Repository navigation
fix(rollout): apply rollout patches through sglang's scheduler bootstrap - #284
Merged
Merged
Conversation
Signed-off-by: rockdu <kangrdu@gmail.com>
Rockdu
force-pushed
the
fix/rollout-patches-bootstrap
branch
from
October 4, 2026 19:19
f885d9f to
d4b1905
Compare
OnePunchMonk
pushed a commit
to OnePunchMonk/miles_diffusion
that referenced
this pull request
Oct 5, 2026
…rap (radixark#284) Signed-off-by: rockdu <kangrdu@gmail.com>
OnePunchMonk
pushed a commit
to OnePunchMonk/miles_diffusion
that referenced
this pull request
Oct 5, 2026
…rap (radixark#284) Signed-off-by: rockdu <kangrdu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
launch_server.bootstrap_scheduler_process, the scheduler spawn target on current sglang-miles, so thewan,qwen_imageandltxrollout patch groups run in the scheduler child again.Why
miles applied its rollout patches by rebinding
launch_server.run_scheduler_process, which spawn pickles by qualname. sglang-miles now spawnsworker_bootstrap.bootstrap_scheduler_processinstead, so that rebinding replaced a name nothing calls, and no patch group reached the engine. Nothing errors; the engine just runs its own bf16 fused norm and rope instead of the fp32 shapes the trainer uses.On
test_wan22_pickscore_grpo_17xGPU_single_node_4xGPU_proxythis showed up astrain/model_output_mean_abs_diff0.0 -> 0.020 andtrain/model_output_rel_max0.0 -> 0.057, plus shifted step-0 rewards. Re-recording that standard would have hidden a train/rollout parity break. A probe with CUDA IPC disabled on the scheduler hop reproduced the same numbers bit for bit, which ruled out the sglang transport fixes running alongside.Validation
Full
PR Testdispatch on #274 + this fix + #283's ring-attention fix + #282 (--continue-on-error), with sglang fromRockdu/sglang:ci/sglang-miles-ipc-host-and-port(sglang-miles plus sgl-project/sglang#42436 and #42470): https://github.com/radixark/miles_diffusion/actions/runs/37197860435wan22, qwenimage and ltx23 are the recipes that use a rollout patch group; all three match their standards again.
Files
miles/backends/sglang_diffusion_utils/sglang_diffusion_engine.py: the scheduler wrapper and its rebinding.Checklist
pre-commit run --all-filespasses — not runpytest -xis green — not run locally; GPU CI above is greenpython3 train.py --helpstill parses — n/a