Repository navigation
[Refactor] scheduler: Share one interface across request lists and abort through each list - #43907
Conversation
53b6bfd to
965b933
Compare
2449c2c to
dae91d1
Compare
f0bab9a to
c9817c8
Compare
dae91d1 to
ab1ed7a
Compare
…ed and held, and dllm to request lists; abort through each list
ab1ed7a to
12a039b
Compare
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
…release_aborted_request public; read checker lists by kind; fix moved abort comments
|
/rerun-test test_disaggregation_optimistic_prefill.py test_prefill_complete_allocation.py test_retract_decode.py test_scripted_core_1gpu.py test_scripted_swa_1gpu.py test_constrained_decoding_spec_reasoning.py test_update_weights_from_tensor.py test_update_weights_from_distributed.py test_session_control.py test_priority_scheduling.py test_weight_version_spans.py |
|
🚀 🚀 🚀 |
…ort through each list (sgl-project#43907)
Every list that holds live requests now shares one interface,
RequestList, and each list's abort lives on the list.scheduler_components/request_list.py:RequestListis an abstract class (iteration,join,leave,take,clear,abort), andListKindnames each list._RequestListis the shared implementation: requests in join order, held by identity.move(reqs, src, dst)hands requests from one list to another and raises if one is not insrc.running_reqs,prefill_launched_reqs, the grammar queue, the PD prefill bootstrap and transfer queues, the PD decode retracted and held lists, and the dLLM waiting queue are_RequestLists, changed only throughjoin,leave,take,clearandmove. The PD prefill transfer queue always exists, empty outside PD prefill, instead of beingNone.request_lists()is keyed byListKind;is_fully_idleand the request-list invariant check read kinds.abort_requestcalls each list'sabort. The waiting queue, the chunked request, the PD decode prealloc and transfer queues, and requests in flight keep their own abort code._make_abort_reqmoves tomanagers/utils.pyasmake_abort_req,Scheduler._release_aborted_requestbecomesrelease_aborted_request, andAbortReq.matches(rid)replaces theabort_all or rid.startswith(...)test repeated at each abort site.RequestList, apart from an allowlist of the fields not migrated yet.Behavior changes:
request_lists()order, so the outputs of different requests can be sent in a different order.Test plan
test/registered/unit/managers/,test/registered/unit/disaggregation/,test/registered/unit/dllm/and the grammar tests pass locally withSGLANG_IS_IN_CI=1, apart from failures main has too.CI States
Latest PR Test (Base): 🚫 Run #41590426729
Latest PR Test (Extra): ❌ Run #41590426552
Latest PR Test (AMD ROCm 10): 🚫 Run #41590426937
Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: