Repository navigation

[cherrypick from #36389] [LoRA] Let DP-attention LoRA run idle ranks and eager prefill (partial) - #41507

Open

yushengsu-thu wants to merge 69 commits into

sglang-milesfrom

yusheng/cherrypick-36389-partial-sglang-miles

Commits on Sep 16, 2026
- [cherry-pick][gRPC] Expose native pause status (#37488) (#39863)
  
  ishandhanani
  and
  Aphoh
  authored
  c2b0eb4
Commits on Sep 17, 2026
Commits on Sep 18, 2026
- [Cherry-pick to release/v0.5.20] [gRPC] Stream engine state changes (#39915) (#40137)
  
  authored
  94602c9
Commits on Sep 19, 2026
- [1/54] [sglang-miles] True on-policy training support (FSDP2 + qwen_dense) (#18639)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  fccd475
- [2/54] [sglang-miles] R3 (Rollout Routing Replay) DeepEP and MTP support (#18642)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  0acd77b
- [3/54] [sglang-miles] PD disaggregation for RL (#18646)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  81b5004
- [4/54] [sglang-miles] MTP related fix (#18647)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  c28a3e2
- [5/54] [sglang-miles] VLM training multimodal fallback fixes (#18781)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  0a90b66
- [6/54] [sglang-miles] Fix pause-aware weight update deadlocks (#22754, #22623)
  
  maocheng23
  authored and
  Zhichenzzz
  committed
  8cb97ae
- [7/54] [sglang-miles] R3 support on PD disaggregation mini_lb (#22916)
  
  zyzshishui
  authored and
  Zhichenzzz
  committed
  a5844d6
- [8/54] [sglang-miles] Improve PD pause handling (#23672, #23887)
  
  ByronHsu
  authored and
  Zhichenzzz
  committed
  09761cc
- [9/54] [sglang-miles] Add KimiK2 raw tool call id parser (#25196)
  
  guapisolo
  authored and
  Zhichenzzz
  committed
  a543492
- [10/54] Fix GLM4 MoE Lite shared expert TP flag
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  637b670
- [11/54] [sglang-miles] MoE-LoRA: kimi 2.5/2.6, GLM-5.2 FP8, cuda-graph/RL fixes (#25141, #29874, #31251)
  
  nanjiangwill
  authored and
  Zhichenzzz
  committed
  216f478
- [12/54] [sglang-miles] DeepSeek V4 RL fixes (#27131, #27603, #27604, #27728)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  6cd0950
- [13/54] [sglang-miles] load_lora_adapter_from_distributed API, with upsert (#27268, #31759, #30913)
  
  gongyisheng
  authored and
  Zhichenzzz
  committed
  ea8c009
- [14/54] [sglang-miles] P2P weight update support and fixes (#21278, #22663)
  
  JD-ETH
  authored and
  Zhichenzzz
  committed
  3091a72
- [15/54] [sglang-miles] rollout indexer replay: raw seq-relative topk capture + meta_info threading
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  3a00422
- [16/54] [sglang-miles] RL weight-update sessions + distributed update for spec draft worker(s) (#27749, #28575, #18565, #22663, #28001, #29675, #27750)
  
  guapisolo
  authored and
  Zhichenzzz
  committed
  a3db53b
- [17/54] [sglang-miles] exclude shared skip-topk layer indexer weights from RL weight check (#29339)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  afca0d3
- [18/54] [sglang-miles] Fix stale _attn_sink_local cache after RL weight updates (#30421)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  68ceb09
- [19/54] [lora] Support GDN in_proj_ba adapters for Qwen3.5 (#30499)
  
  yushengsu-thu
  authored and
  Zhichenzzz
  committed
  f54ceed
- [20/54] [RL] Add /pull_weights: engine-side pull of published weights into a host-local checkpoint (#30366, #28524)
  
  nanjiangwill
  authored and
  Zhichenzzz
  committed
  3a27812
- [21/54] feat(sglang-miles): Support aborting requests by rid prefix - multi-lora needs (#30912)
  
  mathewjhan
  authored and
  Zhichenzzz
  committed
  cac9f81
- [22/54] [sglang-miles] Fix flush_cache() no-op after pause_generation in retract (#31962)
  Zhichenzzz
  committed
  c7d60a1
- [23/54] [sglang-miles] check_weights: make the wire ChecksumInfo match the merged per-role payload
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  8b35006
- [24/54] [sglang-miles] Port RankParallelismConfig / ParallelismContext to v0.5.16's parallel state
  
  JD-ETH
  authored and
  Zhichenzzz
  committed
  fd95c2d
- [25/54] [sglang-miles] Repoint three symbols v0.5.16 renamed or moved
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  e868a3b
- [26/54] [sglang-miles] Repoint two more attributes v0.5.16 relocated
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  4fc2be1
- [27/54] [sglang-miles] Fix iter_runners reaching the draft runner on the wrong object
  
  guapisolo
  authored and
  Zhichenzzz
  committed
  4487241
- [28/54] fix: honor weight-check skips for quantized entries (#32809)
  
  guapisolo
  authored and
  Zhichenzzz
  committed
  fc84c3f
- [29/54] [Fix] Make RowParallelLinear k-size tuple-aware for FP8 (#30742)
  
  XinyuJiangCMU
  authored and
  Zhichenzzz
  committed
  43c5ecf
- [30/54] [sglang-miles] Warn instead of silently dropping one-sided MoE expert LoRA targets (#32376)
  
  yushengsu-thu
  authored and
  Zhichenzzz
  committed
  7530706
- [31/54] [sglang-miles] Back up the CUDA graph pool across TMS pause/resume (#32718)
  
  zyzshishui
  authored and
  Zhichenzzz
  committed
  2c09249
- [32/54] Redesign multi-LoRA (#33299)
  
  yushengsu-thu
  authored and
  Zhichenzzz
  committed
  4e71e2b
- [33/54] Fix NextN weight loading by sharing the unified-loader mappings (#33478)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  a161e3a
- [34/54] Put the DSA cuda-graph page table in the pausable memory region (#33479)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  1c4540a
- [35/54] [sglang-miles] Nemotron support on sglang-miles (#27110)
  Zhichenzzz
  committed
  ef0159a
- [36/54] [sglang-miles] RDT/NIXL weight sync support for Ray scheduler actors (#27723)
  
  xyuzh
  authored and
  Zhichenzzz
  committed
  e539347
- [37/54] [sglang-miles] Restore the transporter's server_args field
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  2c60092
- [38/54] [sglang-miles] Restore _update_bucketed_weights_from_distributed
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  bb5d6e2
- [39/54] [sglang-miles] Guard the skip-topk indexer weight exemption
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  435b783
- [40/54] moe: mask padded rows before capturing routed experts for R3 (#36404)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  7f6df98
- [41/54] args: drop DeepseekV4 from the FlashInfer allreduce-fusion auto-enable
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  71d6e25
- [42/54] capture: slice the dp-gathered buffer by the layout this forward used
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  10a2bbe
- [43/54] [sglang-miles] Add SGLANG_DISABLE_MULTIMEM_AG to force the NCCL all-gather path (#36110)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  7604992
- [44/54] [sglang-miles] Extract Anthropic conversion into standalone utils (#35127)
  
  authored and
  Zhichenzzz
  committed
  b4dd9c2
- [45/54] Release the weight-checker snapshot once compare passes (#36561)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  75a9014
- [46/54] [LoRA] Restore GDN in_proj target-module choices lost in the rebase (#37466)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  1b08216
- [47/54] [RL] Flag the from_numpy sincos position tables as not weight-checked (#37649)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  9d15a3b
- [48/54] [sglang-miles] Streamed LoRA weight updates: register RPC, session scope, LoRA stash (#36384)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  13b2ce7
- [49/54] Allow the GDN out_proj LoRA target in the CLI (#37887)
  
  fzyzcjy
  authored and
  Zhichenzzz
  committed
  03cfffb
- [50/54] fix(qwen3_5): let the GDN gated norm follow the model's device (#37954)
  Zhichenzzz
  committed
  bbcfcd4
- [51/54] [sglang-miles] Flag the MRoPE axis_map buffer as not weight-checked
  Zhichenzzz
  committed
  168a113
- [52/54] [sglang-miles] Pass Nemotron-H layer ID to routing replay (#38718)
  
  Shi-Dong
  authored and
  Zhichenzzz
  committed
  8364067
- [53/54] [LoRA] Publish fresh streamed versions alongside generation (#38165)
  
  authored and
  Zhichenzzz
  committed
  e9e6a18
- [54/54] [sglang-miles] Compare MXFP4 Marlin experts in dequantized space in the weight checker (#40028)
  
  yueming-yuan
  authored and
  Zhichenzzz
  committed
  5a8da8c
Commits on Sep 23, 2026
- [sglang-miles] Kimi K3 colocated RL: LoRA fixes and in-place MXFP4 Marlin reload (#37704)
  yueming-yuan
  authored
  67e4cd1
- Resolve HF LoRA targets through model-aware normalization (#40242)
  yueming-yuan
  authored
  201f3e0
Commits on Sep 24, 2026
- [cherrypick from #40932] [Sampling] Add selected/support sampling logprob modes (#41047)
  nanjiangwill
  authored
  ae04cb1
Commits on Sep 25, 2026
Commits on Sep 26, 2026
- [cherrypick from #40986] [Sampling] Stream sampling masks as per-request arrays (#41270)
  nanjiangwill
  authored
  ab4f767
- [cherrypick from #41235] [PD] Keep the sampling mask of a replayed rebootstrap token (#41271)
  
  nanjiangwill
  and
  yueming-yuan
  authored
  571212b
Commits on Sep 28, 2026
- [LoRA] Let DP-attention LoRA run idle ranks and eager prefill (partial #36389)
  
  erikwijmans
  and
  yushengsu-thu
  committed
  7f6c583

Commits

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 19, 2026

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 25, 2026

Commits on Sep 26, 2026

Commits on Sep 28, 2026