[guptaishaan]

Motivation

Fixes #35537.

batch_is_full could remain set after the final chunked-prefill pass. The next scheduler pass returned before recalculating available request slots, which left a waiting request queued until a running request finished.

Thanks to @atlantistony for the report and detailed trace.

Modifications

Reset batch_is_full at the start of each prefill admission pass. Add a regression test that starts with the stale post-chunk state and verifies that the waiting request reaches admission.

Accuracy Tests

Not applicable. This does not change model output computation.

Speed Tests and Profiling

Not run. The focused scheduler test passed on a Linux node with 8 NVIDIA A40 GPUs, with CUDA hidden for the CPU unit test. The full reported Qwen3.8 NVFP4 setup could not run because A40 is sm_86.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations. Not needed for this internal scheduler fix.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed. Not applicable to model accuracy, and the reported NVFP4 setup is unsupported on A40.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ❌ Run #38208264284
Latest PR Test (Extra): ❌ Run #38208264108
Latest PR Test (AMD ROCm 10): ❌ Run #38208264211

[guptaishaan]

Rebased onto current main. The scheduler change is unchanged. In the regression test I added scheduler.processed_tokens_counter = 0, since calc_priority now takes it and the bare Scheduler in the test would otherwise fail on that instead of the behavior under test.

The pr-test-finish failure came from call-gate / pr-gate ("Require run-ci label"). No test job ran or failed, so it is not caused by the change. It will need the run-ci label or /tag-and-rerun-ci from a maintainer.

I re-ran test_scheduler_chunked_req_gate.py on CPU: it passes, and the new test fails when the scheduler change is reverted.

[guptaishaan]

Rebased onto current main.

One conflict in _get_new_batch_prefill_raw: #41872 added a batch_is_full reset when running_batch.is_empty(). That does not cover this issue, where the running batch is non-empty and the flag was latched by the last chunked-prefill pass, so I kept the unconditional reset, which also covers the cases main handles.

I also set running_batch.is_empty.return_value = False in the regression test. With the mock's default truthy is_empty() the test would pass on main even without the fix. With that change it passes with the fix and errors without it. test_scheduler_chunked_req_gate.py passes on CPU.

[hnyls2002]

Thanks @guptaishaan, the symptom was real. Its cause was a slot double count rather than the latch itself: a chunked request keeps its req_to_token row across chunks, so available_size() already excluded it while can_run_list counted it again, which set batch_is_full on the final chunk with a row still free. #44401 fixes that count on main and keeps the latch, so resetting batch_is_full every pass is no longer needed. Closing as superseded; please reopen if #35537 still reproduces on main.