[BugFix] scheduler: Fix ordering preserving of skipped requests#32173
[BugFix] scheduler: Fix ordering preserving of skipped requests#32173njhill merged 1 commit intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request correctly fixes a bug where the order of skipped requests was not preserved by the scheduler. The fix involves changing the implementation of FCFSRequestQueue.prepend_requests to correctly handle the reversed order of skipped requests. The removal of the now-unused __reversed__ method is a good cleanup. I have one suggestion to improve the clarity of the prepend_requests method's docstring to prevent potential confusion in the future, as its behavior regarding ordering might be unexpected.
|
Hi @orozery, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
8c08c30 to
bc525fc
Compare
This commit fixes the order preserving of requests skipped by the scheduler. A unit test is added to verify the fix. Signed-off-by: Or Ozeri <oro@il.ibm.com>
bc525fc to
43019fa
Compare
…-project#32173) Signed-off-by: Or Ozeri <oro@il.ibm.com> Signed-off-by: Tomer Natan <tbarnatan@computelab-frontend-8.nvidia.com>
…-project#32173) Signed-off-by: Or Ozeri <oro@il.ibm.com>
…-project#32173) Signed-off-by: Or Ozeri <oro@il.ibm.com>
…-project#32173) Signed-off-by: Or Ozeri <oro@il.ibm.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
…-project#32173) Signed-off-by: Or Ozeri <oro@il.ibm.com>
This PR fixes the order preserving of requests skipped by the scheduler.
A unit test is added to verify the fix.
I came across this bug while testing #29087, and the test there requires this fix.
Note
Cursor Bugbot is generating a summary for commit 8c08c30fef37d620043c4221960e04d6704ba1dc. Configure here.
Note
Fixes request ordering when skipped requests are re-queued.
FCFSRequestQueue.prepend_requeststoextendleft(requests)(remove reverse) to maintain original order when prepending__reversed__definitions from request queuestest_prepend_skipped_requests_orderto verify waiting order is preserved when some requests wait for remote KVsWritten by Cursor Bugbot for commit 8c08c30fef37d620043c4221960e04d6704ba1dc. This will update automatically on new commits. Configure here.
Note
Ensures scheduler preserves request order when skipped requests are re-queued.
FCFSRequestQueue.prepend_requeststo prepend without reversing to maintain original order__reversed__definitions fromRequestQueue,FCFSRequestQueue, andPriorityRequestQueuetest_prepend_skipped_requests_orderto verify waiting order is preserved when some requests wait for remote KVsWritten by Cursor Bugbot for commit bc525fcc2dbb1c060a48b593dfab8d93d2d8697b. This will update automatically on new commits. Configure here.
Note
Ensures the scheduler preserves waiting order when skipped requests are re-queued.
FCFSRequestQueue.prepend_requeststo prepend without reversing to maintain original waiting order__reversed__fromRequestQueue,FCFSRequestQueue, andPriorityRequestQueuetest_prepend_skipped_requests_orderto verify waiting order is preserved when some requests wait for remote KVsWritten by Cursor Bugbot for commit 43019fa. This will update automatically on new commits. Configure here.