Repository navigation
[Bugfix] Backport grouped chunk interaction latency fix to v0.32.0 - #9377
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
This PR matches CODEOWNERS paths: /tests/. Code owners: @NickCao @yenuo26 Routing: @NickCao via CODEOWNERS; @yenuo26 via CODEOWNERS @maithilijoshi20, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer. Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment. |
Omni ReviewBot triage noteAutomated triage of commit
These are automated triage suggestions only — the final decision belongs to the maintainers. |
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com> (cherry picked from commit aea37f3)
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com> (cherry picked from commit 54f4ef7)
With the boundary step left for a fresh call, a two-step chunk runs one step per call, so tests whose stubs use two-step chunks no longer reach the grouped continuation. Use three-step chunks where a test is about grouped steps: - grouped steps see the same inputs as per-step calls; - a failing step ends the grouped call. Dropping a continuation step's output passed the module before this change and now fails this test; - the interaction test queues its update after two grouped steps, and the boundary call runs exactly one step; - the call-pattern test covers two- and three-step chunks. Also make the _execute_stepwise_core docstring match where the grouped call now stops. Signed-off-by: windyskr <54269955+windyskr@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 94e01a0)
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com> (cherry picked from commit 545cc70)
0202d44 to
dd42361
Compare
Backport of #8810 to
release/v0.32.0, requested in Gaohan123's approval.Purpose
Streaming AR diffusion pipelines with chunk-step grouping process an interaction received during chunk k only after preparing chunk k+1, delaying the update until chunk k+2. Leave the boundary-finishing step for a fresh scheduler invocation so queued interaction RPCs can be handled before boundary application. Chunks with at least two denoise steps benefit; one-step chunks retain their existing latency.
This is a clean cherry-pick of the four implementation/test commits from #8810, with source commit references preserved. It includes the three-step grouping coverage and the regression using
runner.submit_interaction()with a minimal coordinator fake. It does not include the source branch's merges from main.The changes affect only the runner and its regression tests. Public configuration and APIs are unchanged; the runner docstrings describe the corrected boundary behavior. Original reproduction and expected/observed behavior: #8756.
Test Plan
vLLM Version: 0.32.0; Python 3.12.3 and PyTorch 2.14.0+cu132 under WSL2, using the existing
~/.venvs/vllm-py312environment.vLLM-Omni Base:
release/v0.32.0.python -m pytest -q tests/diffusion/test_diffusion_step_pipeline.py tests/diffusion/streaming_interactive_generation -k "not parallel"Test Result