Repository navigation
Redesign multi-LoRA - #33299
Merged
Merged
Redesign multi-LoRA#33299
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
There was a problem hiding this comment.
Copilot wasn't able to review any files in this pull request.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Lease release now goes through a single _finalize_lora_lease guarded by ReqState.lora_lease_released, closing both the leak (aborted/errored requests never releasing) and the double-release (finish + abort both decrementing) that each wedge wait_for_zero_leases. Registry acquire performs its lookup and counter increment inside the writer lock, removing the lookup/increment race. 14 unit tests cover the lease state machine.
Distributed-loaded adapters register as reloadable=False: eviction keeps the catalog entry but implicit reload of a non-reloadable adapter is a hard error instead of silently serving stale weights, and explicit unload drops the lora_ref_cache entry. The two LoRA control endpoints now run the API-key check. Abort-prefix unit-test stubs track upstream scheduler fields (dllm_config, pp_size, last_batch).
yushengsu-thu
requested review from
CatherineSue,
Fridge003,
JustinTong0323,
Ying1123,
hnyls2002,
ispobock,
jybsuper,
lifuhuang,
merrymercy,
slin1237 and
xiezhq-hermann
as code owners
August 4, 2026 00:03
yushengsu-thu
force-pushed
the
sglang-redesign-multi-lora
branch
from
August 4, 2026 00:15
a0f40e5 to
fa4caa8
Compare
This was referenced Aug 4, 2026
yueming-yuan
pushed a commit
that referenced
this pull request
Aug 25, 2026
Kh4L
pushed a commit
to Kh4L/sglang
that referenced
this pull request
Sep 8, 2026
(cherry picked from commit 642106b)
8 of 33 tasks
Zhichenzzz
pushed a commit
that referenced
this pull request
Sep 19, 2026
HJSang
pushed a commit
to HJSang/sglang
that referenced
this pull request
Sep 21, 2026
(cherry picked from commit 642106b)
ziang-and
pushed a commit
to zianglih/sglang
that referenced
this pull request
Oct 2, 2026
…lication (sgl-project#27268, sgl-project#31759, sgl-project#30913, sgl-project#30912, sgl-project#33299, sgl-project#36384, sgl-project#38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
nv-hwoo
pushed a commit
to nv-hwoo/sglang
that referenced
this pull request
Oct 20, 2026
…lication (sgl-project#27268, sgl-project#31759, sgl-project#30913, sgl-project#30912, sgl-project#33299, sgl-project#36384, sgl-project#38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
nanjiangwill
pushed a commit
to modal-projects/sglang
that referenced
this pull request
Oct 21, 2026
…lication (sgl-project#27268, sgl-project#31759, sgl-project#30913, sgl-project#30912, sgl-project#33299, sgl-project#36384, sgl-project#38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com> (cherry picked from commit c40cd81) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VDwTQn4gGFgVhzGXFs4aJx
Rockdu
pushed a commit
that referenced
this pull request
Oct 30, 2026
…lication (#27268, #31759, #30913, #30912, #33299, #36384, #38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
Rockdu
pushed a commit
that referenced
this pull request
Oct 30, 2026
…lication (#27268, #31759, #30913, #30912, #33299, #36384, #38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
Rockdu
pushed a commit
that referenced
this pull request
Oct 31, 2026
…lication (#27268, #31759, #30913, #30912, #33299, #36384, #38165) Register adapter identities, stream their tensors within weight-update sessions, and publish checksum-verified fresh versions alongside generation. Preserve per-rank failure cleanup and request-prefix cancellation for adapter lifecycle management. Co-authored-by: Mathew Han <mathewjhan@gmail.com> Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com> Co-authored-by: Yueming Yuan <yym022502@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hardening for miles multi-LoRA serving (slot oversubscription with registration-scoped adapters):
_finalize_lora_lease+ReqState.lora_lease_releasedclose both the lease leak on abort/error paths and the finish+abort double-release; either wedgeswait_for_zero_leases.LoRARef.reloadable: eviction keeps the catalog entry; implicit reload of a non-reloadable (distributed-loaded) adapter is a hard error instead of silently serving stale weights; explicit unload dropslora_ref_cache.Used by miles PR: radixark/miles#2137
CI States
Latest PR Test (Base): ❌ Run #30865020522
Latest PR Test (Extra): ❌ Run #30865020325