Skip to content

[Refactor] scheduler: Share one interface across request lists and abort through each list - #43907

Merged
hnyls2002 merged 13 commits into
mainfrom
lsyin/sched-pr13
Nov 3, 2026
Merged

hnyls2002 merged 13 commits into
mainfrom
lsyin/sched-pr13

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Every list that holds live requests now shares one interface, RequestList, and each list's abort lives on the list.

  • scheduler_components/request_list.py: RequestList is an abstract class (iteration, join, leave, take, clear, abort), and ListKind names each list. _RequestList is the shared implementation: requests in join order, held by identity. move(reqs, src, dst) hands requests from one list to another and raises if one is not in src.
  • running_reqs, prefill_launched_reqs, the grammar queue, the PD prefill bootstrap and transfer queues, the PD decode retracted and held lists, and the dLLM waiting queue are _RequestLists, changed only through join, leave, take, clear and move. The PD prefill transfer queue always exists, empty outside PD prefill, instead of being None.
  • request_lists() is keyed by ListKind; is_fully_idle and the request-list invariant check read kinds.
  • abort_request calls each list's abort. The waiting queue, the chunked request, the PD decode prealloc and transfer queues, and requests in flight keep their own abort code.
  • List owners build and release aborts through shared entry points: _make_abort_req moves to managers/utils.py as make_abort_req, Scheduler._release_aborted_request becomes release_aborted_request, and AbortReq.matches(rid) replaces the abort_all or rid.startswith(...) test repeated at each abort site.
  • A unit test checks that every field of a list owner annotated as holding requests is a RequestList, apart from an allowlist of the fields not migrated yet.

Behavior changes:

  • One abort handles the lists in request_lists() order, so the outputs of different requests can be sent in a different order.
  • Requeueing an optimistic PD prefill request raises if the request is not in the list it leaves; before, that was ignored.

Test plan

  • The unit tests under test/registered/unit/managers/, test/registered/unit/disaggregation/, test/registered/unit/dllm/ and the grammar tests pass locally with SGLANG_IS_IN_CI=1, apart from failures main has too.

CI States

Latest PR Test (Base): 🚫 Run #41590426729
Latest PR Test (Extra): ❌ Run #41590426552
Latest PR Test (AMD ROCm 10): 🚫 Run #41590426937

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@github-actions github-actions Bot added the Multi-modal multi-modal language model label Oct 12, 2026
@hnyls2002
hnyls2002 marked this pull request as draft October 13, 2026 02:01
@hnyls2002
hnyls2002 force-pushed the lsyin/sched-pr12 branch 2 times, most recently from f0bab9a to c9817c8 Compare November 3, 2026 00:25
Base automatically changed from lsyin/sched-pr12 to main November 3, 2026 02:33
@github-actions github-actions Bot added lora hicache Hierarchical Caching for SGLang labels Nov 3, 2026
@hnyls2002 hnyls2002 changed the title [PD] Decode a copy of the running rows and drop finished rows at commit [Refactor] scheduler: Describe request lists by property and ask a registry Nov 3, 2026
@hnyls2002 hnyls2002 changed the title [Refactor] scheduler: Describe request lists by property and ask a registry [Refactor] scheduler: Share one interface across request lists and abort through each list Nov 3, 2026
@hnyls2002
hnyls2002 marked this pull request as ready for review November 3, 2026 04:44

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@hnyls2002
hnyls2002 requested a review from liusy58 as a code owner November 3, 2026 05:09
@hnyls2002 hnyls2002 added highest-priority CI: all three control labels, plus never batch-cancelled or stale-closed run-ci CI: run the baseline test suite on this PR labels Nov 3, 2026
@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_disaggregation_optimistic_prefill.py test_prefill_complete_allocation.py test_retract_decode.py test_scripted_core_1gpu.py test_scripted_swa_1gpu.py test_constrained_decoding_spec_reasoning.py test_update_weights_from_tensor.py test_update_weights_from_distributed.py test_session_control.py test_priority_scheduling.py test_weight_version_spans.py

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

🚀 2-gpu-h100 (5 tests): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_optimistic_prefill.py
cd test/ && python3 registered/e2e/disaggregation/test_prefill_complete_allocation.py
cd test/ && python3 registered/spec/test_constrained_decoding_spec_reasoning.py
cd test/ && python3 registered/rl/test_update_weights_from_distributed.py
cd test/ && python3 registered/rl/test_weight_version_spans.py

🚀 1-gpu-5090 (4 tests): ✅ View workflow run

cd test/ && python3 registered/scheduler/test_retract_decode.py
cd test/ && python3 registered/chunked_prefill/test_scripted_core_1gpu.py
cd test/ && python3 registered/rl/test_update_weights_from_tensor.py
cd test/ && python3 registered/scheduler/test_priority_scheduling.py

🚀 1-gpu-h100 (2 tests): ✅ View workflow run

cd test/ && python3 registered/chunked_prefill/test_scripted_swa_1gpu.py
cd test/ && python3 registered/sessions/test_session_control.py

@hnyls2002
hnyls2002 merged commit ed8d232 into main Nov 3, 2026
85 of 145 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/sched-pr13 branch November 3, 2026 06:32
Yuxingwang-intel pushed a commit to Yuxingwang-intel/sglang that referenced this pull request Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hicache Hierarchical Caching for SGLang highest-priority CI: all three control labels, plus never batch-cancelled or stale-closed lora Multi-modal multi-modal language model run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant