Skip to content

[Refactor] diffusion RL: Rollout controller as a scheduler-step plugin instead of mixins (sglang-miles port) - #44877

Merged
Rockdu merged 15 commits into
sglang-milesfrom
rockdu/rollout-plugin-sglang-miles
Oct 17, 2026
Merged

Rockdu merged 15 commits into
sglang-milesfrom
rockdu/rollout-plugin-sglang-miles

Conversation

@Rockdu

@Rockdu Rockdu commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The sglang-miles port of #44603 (the same 15 commits) on the sglang-miles bump to sglang v0.5.22; it supersedes #44239. The one difference from #44603: sglang-miles keeps its trajectory semantics from #40786 (timesteps cover every step, latents follow rollout_return_step_indices, latent_step_indices is recorded), which miles-diffusion depends on.

Rollout stops being a scheduler / denoising-stage mixin: each latent stream of a rollout request gets a FlowRolloutController whose transition the loop passes into scheduler.step. Serving is untouched; rollout numerics do not change.

It supersedes the design in #37940; @niehen6174 is co-author on the derived commits. @JingwenGu0829 found the issues fixed in commits 13 and 14 (Rockdu#4) and is co-author on commits 13 to 15.

How it works

  • FlowMatchEulerDiscreteScheduler.step(..., transition=None) runs transition or self._transition. Loops pass **rollout_step_kwargs(batch), which is empty for serving requests, so every scheduler keeps its serving call.
  • Hooks in rollout_hooks.py: prepare_rollout_if_enabled (before the SP shard), init_rollout_env_if_enabled, rollout_step_kwargs, finalize_rollout_if_enabled; all no-ops for serving.
  • Wired into generic DenoisingStage, Cosmos3 and LTX-2. A loop without a scheduler (MiniMax H3, next PR) passes its own build_controller and ode_transition.

Review guide

Commits 2, 3, 4 and 6 are mechanical moves verified by the chain verifier (4/4 proofs) and commit 1 is a pure rename; skim them. Review commits 5, 7 to 11 and 13 to 15.

Tests

Checklist


CI States

Latest PR Test (Base): ❌ Run #39022334419
Latest PR Test (Extra): ❌ Run #39022334380
Latest PR Test (AMD ROCm 10): ❌ Run #39022334446

Rockdu and others added 15 commits October 16, 2026 20:42
…troller.py

Pure file rename (git R100); the proof tool has no file-rename primitive.

Refactor-Kind: non_mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
…er.py

Refactor-Kind: mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
Refactor-Kind: mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
Refactor-Kind: mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
… RolloutSessionData

Refactor-Kind: non_mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
Refactor-Kind: mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
…lloutController and drop SchedulerRLMixin

Refactor-Kind: non_mechanical_provable
Signed-off-by: rockdu <kangrdu@gmail.com>
…stead of branching on batch

Refactor-Kind: non_mechanical_provable
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
… of the denoise loops

Refactor-Kind: non_mechanical_provable
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
… hook functions

Refactor-Kind: non_mechanical_provable
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
…ontroller

Refactor-Kind: non_mechanical_provable
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
… miles-diffusion

Refactor-Kind: non_mechanical_provable

Signed-off-by: rockdu <kangrdu@gmail.com>
… keep trajectory timesteps on device

Refactor-Kind: non_mechanical_provable
Co-authored-by: Kangrui Du <89372739+Rockdu@users.noreply.github.com>
Co-authored-by: JingwenGu0829 <75733630+JingwenGu0829@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
…nstead of capturing it

Refactor-Kind: non_mechanical_provable
Co-authored-by: Kangrui Du <89372739+Rockdu@users.noreply.github.com>
Co-authored-by: JingwenGu0829 <75733630+JingwenGu0829@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
…undant generator argument and make the trajectory recorder public

Refactor-Kind: non_mechanical_provable
Co-authored-by: Kangrui Du <89372739+Rockdu@users.noreply.github.com>
Co-authored-by: JingwenGu0829 <75733630+JingwenGu0829@users.noreply.github.com>
Signed-off-by: rockdu <kangrdu@gmail.com>
@github-actions github-actions Bot added the diffusion SGLang Diffusion label Oct 17, 2026
@Rockdu
Rockdu marked this pull request as ready for review October 17, 2026 09:33
@Rockdu
Rockdu force-pushed the rockdu/rollout-plugin-sglang-miles branch 3 times, most recently from 38746af to 928423d Compare October 17, 2026 09:45
@Rockdu Rockdu changed the title [Refactor] diffusion: Rollout controller as a scheduler-step plugin instead of mixins (sglang-miles port) [Refactor] diffusion RL: Rollout controller as a scheduler-step plugin instead of mixins (sglang-miles port) Oct 17, 2026
@Rockdu
Rockdu merged commit 74bd569 into sglang-miles Oct 17, 2026
68 of 80 checks passed
@Rockdu
Rockdu deleted the rockdu/rollout-plugin-sglang-miles branch October 17, 2026 09:47
nanjiangwill pushed a commit to modal-projects/sglang that referenced this pull request Nov 1, 2026
…n instead of mixins (sglang-miles port) (sgl-project#44877)

Signed-off-by: rockdu <kangrdu@gmail.com>
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Co-authored-by: JingwenGu0829 <75733630+JingwenGu0829@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant