Repository navigation
Add sp_min_token_num=0 to E2E correctness tests for SP and AsyncTP - #38151
Closed
ProExpertProg with Copilot wants to merge 1 commit into
Closed
ProExpertProg with Copilot wants to merge 1 commit into
ProExpertProg with Copilot wants to merge 1 commit into
Conversation
Copilot
AI
changed the title
[WIP] Add CompilationConfig.pass_config.enable_sp_min_token to E2E tests
Add Mar 25, 2026
sp_min_token_num=0 to E2E correctness tests for SP and AsyncTP
Co-authored-by: ProExpertProg <11367180+ProExpertProg@users.noreply.github.com> Agent-Logs-Url: https://github.com/vllm-project/vllm/sessions/f8e4f0ad-7a0f-4253-8a28-7a0100eacae5
ProExpertProg
force-pushed
the
copilot/add-sp-min-token-to-e2e-tests
branch
from
April 29, 2026 18:11
6cf77e4 to
b65206a
Compare
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
6 tasks done
Collaborator
|
Closing as superseded by #60517, which removed the torch.compile-based sequence parallelism pass and AsyncTP (along with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When
sp_min_token_numis unset, vLLM auto-computes an SP threshold viaget_sequence_parallelism_threshold(). For small models (e.g.hmellor/tiny-random-LlamaForCausalLM) or dummy-loaded models with overriddenhidden_size, this returnsNone(belowSP_MIN_HIDDEN_SIZE), silently disabling SP — meaning the correctness tests weren't actually exercising SP or AsyncTP paths.Changes
test_sequence_parallel.py: Add"sp_min_token_num": 0topass_configin_compare_sp()to force SP enabled regardless of model size.test_async_tp.py: Add"sp_min_token_num": 0topass_configintest_async_tp_pass_correctness()— AsyncTP internally requires SP, same silent-disable risk applies.Consistent with the existing pattern in
tests/compile/fusions_e2e/test_tp2_async_tp.pywhich already setssp_min_token_num=512for the same reason. Using0is more aggressive: SP applies at any batch size.⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.