Skip to content

[BugFix] Fixed the issue where --no-async-chunk was not working.#2934

Merged
lishunyang12 merged 1 commit intovllm-project:mainfrom
amy-why-3459:bugfix_tests
Apr 20, 2026
Merged

[BugFix] Fixed the issue where --no-async-chunk was not working.#2934
lishunyang12 merged 1 commit intovllm-project:mainfrom
amy-why-3459:bugfix_tests

Conversation

@amy-why-3459
Copy link
Copy Markdown
Contributor

@amy-why-3459 amy-why-3459 commented Apr 20, 2026

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

fix #2933

Test Plan

vllm serve /home/models/Qwen3-Omni-30B-A3B-Instruct --omni --port 28889 --no-async-chunk

Test Result

(Worker pid=3867183) INFO 04-20 04:03:12 [qwen3_omni.py:1006] ============self.vllm_config.model_config.async_chunk:False
(APIServer pid=3866749) INFO 04-20 04:03:12 [stage_engine_core_client.py:172] [StageEngineCoreClient] Stage-2 adding request: chatcmpl-84dfae6f31fc3eb1
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] [Overall Summary]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] +-----------------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | Field                       |      Value |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] +-----------------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_requests                |          1 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_wall_time_ms            | 11,939.418 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_total_tokens            |        301 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_avg_time_per_request_ms | 11,939.418 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_avg_tokens_per_s        |     25.211 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_stage_0_wall_time_ms    |  1,953.011 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_stage_1_wall_time_ms    |  3,227.079 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] | e2e_stage_2_wall_time_ms    |  1,626.541 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:502] +-----------------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] [RequestE2EStats [request_id=chatcmpl-84dfae6f31fc3eb1]]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] +------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] | Field            |      Value |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] +------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] | e2e_total_ms     | 11,875.375 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] | e2e_total_tokens |        301 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:528] +------------------+------------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] [StageRequestStats [request_id=chatcmpl-84dfae6f31fc3eb1]]
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] +-------------------+-----------+-----------+-----------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | Field             |         0 |         1 |         2 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] +-------------------+-----------+-----------+-----------+
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | batch_id          |         1 |         1 |         1 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | batch_size        |         1 |         1 |         1 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | num_tokens_in     |        67 |         0 |         0 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | num_tokens_out    |        30 |       204 |         0 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] | stage_gen_time_ms | 1,951.266 | 3,226.583 | 1,625.796 |
(APIServer pid=3866749) INFO 04-20 04:03:14 [stats.py:581] +-------------------+-----------+-----------+-----------+


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
  • The test results. Please paste the results comparison before and after, or the e2e results.
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
  • (Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
@hsliuustc0106 hsliuustc0106 added the ready label to trigger buildkite CI label Apr 20, 2026
@amy-why-3459
Copy link
Copy Markdown
Contributor Author

@ZeldaHuang @lishunyang12 PTAL

Copy link
Copy Markdown
Collaborator

@lishunyang12 lishunyang12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your quick fix.

# Provisional value (mirrors the CLI/caller kwarg); the engine resolves
# pipeline + deploy YAML + CLI precedence below and the final value is
# re-assigned from ``self.engine.async_chunk`` after init.
self.async_chunk = bool(async_chunk) if async_chunk is not None else False
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have updated this logic in #2383 with introduction of a new flag --no-async-chunk

@lishunyang12 lishunyang12 merged commit 0393c58 into vllm-project:main Apr 20, 2026
8 checks passed
qinganrice pushed a commit to qinganrice/vllm-omni that referenced this pull request Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Passing --no-async-chunk to the command line had no effect.

3 participants