skip cudagraph/DP padding in topk - #48979
Merged
Merged
Conversation
gnovack
marked this pull request as ready for review
July 17, 2026 20:19
gnovack
requested review from
AndreasKaratzas,
WoosukKwon,
mgoin,
pavanimajety,
tlrmchlsmth,
yewentao256 and
zyongye
as code owners
July 17, 2026 20:19
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
WoosukKwon
approved these changes
Jul 17, 2026
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: gnovack <novackgm@gmail.com>
gnovack
force-pushed
the
topk-skip-padding
branch
from
July 18, 2026 20:53
b210624 to
6e37c87
Compare
4 tasks
|
@WoosukKwon hey guys, I'm experimenting with Intel B70 and main branch code, it looks like after this PR some function call in XPU has changed from 6 to 7 arguments and it's failing now. I understand it's main branch and so on, just FYI |
Member
|
@jikunshang yeah just saw it, that was quick, thx! |
12 tasks
Tejas-Raj01
pushed a commit
to Tejas-Raj01/vllm
that referenced
this pull request
Jul 22, 2026
Signed-off-by: gnovack <novackgm@gmail.com> Signed-off-by: Tejas-Raj01 <rajtejas.xyz@gmail.com>
vx120
pushed a commit
to vx120/vllm
that referenced
this pull request
Jul 23, 2026
Signed-off-by: gnovack <novackgm@gmail.com>
vrdn-23
added a commit
to vrdn-23/vllm
that referenced
this pull request
Jul 23, 2026
Resolves the merge of origin/main (46f01a5) into vrdn-23/refactor-envs-to-use-pydantic-settings (base 34e6dfc). vllm/envs.py and docs/configuration/env_vars.md conflicted structurally: main edits the legacy TYPE_CHECKING block + environment_variables dict, which this branch already replaced with a tree of pydantic BaseSettings subclasses. Took ours for both (branch structure wins), then ported the semantic delta from the 11 main-side commits field-by-field: Additions (12 vars): - vllm-project#47845 VLLM_MAX_COMPLETION_PROMPTS (ServerSettings) - vllm-project#47388 VLLM_ENABLE_STARTUP_PLAN (PathSettings) - vllm-project#45313 VLLM_BUILD_COMMIT/PIPELINE/URL, VLLM_IMAGE_TAG (BuildSettings) - vllm-project#48268 VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS (FlashInferSettings; comma-list field_validator) - vllm-project#42433 VLLM_EC_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#47636 VLLM_P2P_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#45964 VLLM_DCP_Q_REPLICATE (QuantSettings) Modification: - vllm-project#48979 VLLM_MOE_SKIP_PADDING default False -> True Rename (required; merged callers read the new name): - vllm-project#45781 VLLM_TRITON_ATTN_USE_TD -> VLLM_TRITON_USE_TD (rename only, no deprecation shim) Deletions (vllm-project#44749): VLLM_ROCM_USE_AITER_PAGED_ATTN, VLLM_TPU_BUCKET_PADDING_GAP, VLLM_TPU_MOST_MODEL_LEN, VLLM_CI_USE_S3 (also dropped from the compile_factors ignore-set), VLLM_FLASHINFER_ALLREDUCE_FUSION_THRESHOLDS_MB (+ its validator). compile_factors() ignore-set: added VLLM_ENABLE_STARTUP_PLAN, VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS, VLLM_XLA_CACHE_PATH, VLLM_CONFIG_ROOT (vllm-project#47573); removed VLLM_CI_USE_S3. docs/configuration/env_vars.md: kept ours — the VLLM_PORT/Kubernetes warning lives in the docs generator on this branch, and new fields flow through gen_env_vars.py automatically. Not ported: nothing dropped silently; all 11 commits' envs.py deltas are covered above. Verified: no conflict markers; all 12 ported vars import with correct defaults; all 5 deleted vars gone; pre-commit clean on both files; tests/test_envs.py 11/11 pass (incl. test_p2p_side_channel_defaults_and_override). AI assistance was used to resolve this merge, following the reusable playbook at docs/superpowers/specs/2026-05-14-envs-merge-conflict-resolution-design.md (execution recorded in Appendix D). Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Vinay Damodaran <vrdn@hey.com>
vrdn-23
added a commit
to vrdn-23/vllm
that referenced
this pull request
Jul 23, 2026
Resolves the merge of origin/main (46f01a5) into vrdn-23/refactor-envs-to-use-pydantic-settings (base 34e6dfc). vllm/envs.py and docs/configuration/env_vars.md conflicted structurally: main edits the legacy TYPE_CHECKING block + environment_variables dict, which this branch already replaced with a tree of pydantic BaseSettings subclasses. Took ours for both (branch structure wins), then ported the semantic delta from the 11 main-side commits field-by-field: Additions (12 vars): - vllm-project#47845 VLLM_MAX_COMPLETION_PROMPTS (ServerSettings) - vllm-project#47388 VLLM_ENABLE_STARTUP_PLAN (PathSettings) - vllm-project#45313 VLLM_BUILD_COMMIT/PIPELINE/URL, VLLM_IMAGE_TAG (BuildSettings) - vllm-project#48268 VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS (FlashInferSettings; comma-list field_validator) - vllm-project#42433 VLLM_EC_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#47636 VLLM_P2P_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#45964 VLLM_DCP_Q_REPLICATE (QuantSettings) Modification: - vllm-project#48979 VLLM_MOE_SKIP_PADDING default False -> True Rename (required; merged callers read the new name): - vllm-project#45781 VLLM_TRITON_ATTN_USE_TD -> VLLM_TRITON_USE_TD (rename only, no deprecation shim) Deletions (vllm-project#44749): VLLM_ROCM_USE_AITER_PAGED_ATTN, VLLM_TPU_BUCKET_PADDING_GAP, VLLM_TPU_MOST_MODEL_LEN, VLLM_CI_USE_S3 (also dropped from the compile_factors ignore-set), VLLM_FLASHINFER_ALLREDUCE_FUSION_THRESHOLDS_MB (+ its validator). compile_factors() ignore-set: added VLLM_ENABLE_STARTUP_PLAN, VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS, VLLM_XLA_CACHE_PATH, VLLM_CONFIG_ROOT (vllm-project#47573); removed VLLM_CI_USE_S3. docs/configuration/env_vars.md: kept ours — the VLLM_PORT/Kubernetes warning lives in the docs generator on this branch, and new fields flow through gen_env_vars.py automatically. Not ported: nothing dropped silently; all 11 commits' envs.py deltas are covered above. Verified: no conflict markers; all 12 ported vars import with correct defaults; all 5 deleted vars gone; pre-commit clean on both files; tests/test_envs.py 11/11 pass (incl. test_p2p_side_channel_defaults_and_override). AI assistance was used to resolve this merge. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Vinay Damodaran <vrdn@hey.com>
vrdn-23
added a commit
to vrdn-23/vllm
that referenced
this pull request
Jul 23, 2026
Resolves the merge of origin/main (46f01a5) into vrdn-23/refactor-envs-to-use-pydantic-settings (base 34e6dfc). vllm/envs.py and docs/configuration/env_vars.md conflicted structurally: main edits the legacy TYPE_CHECKING block + environment_variables dict, which this branch already replaced with a tree of pydantic BaseSettings subclasses. Took ours for both (branch structure wins), then ported the semantic delta from the 11 main-side commits field-by-field: Additions (12 vars): - vllm-project#47845 VLLM_MAX_COMPLETION_PROMPTS (ServerSettings) - vllm-project#47388 VLLM_ENABLE_STARTUP_PLAN (PathSettings) - vllm-project#45313 VLLM_BUILD_COMMIT/PIPELINE/URL, VLLM_IMAGE_TAG (BuildSettings) - vllm-project#48268 VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS (FlashInferSettings; comma-list field_validator) - vllm-project#42433 VLLM_EC_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#47636 VLLM_P2P_SIDE_CHANNEL_HOST/PORT (ConnectorSettings) - vllm-project#45964 VLLM_DCP_Q_REPLICATE (QuantSettings) Modification: - vllm-project#48979 VLLM_MOE_SKIP_PADDING default False -> True Rename (required; merged callers read the new name): - vllm-project#45781 VLLM_TRITON_ATTN_USE_TD -> VLLM_TRITON_USE_TD (rename only, no deprecation shim) Deletions (vllm-project#44749): VLLM_ROCM_USE_AITER_PAGED_ATTN, VLLM_TPU_BUCKET_PADDING_GAP, VLLM_TPU_MOST_MODEL_LEN, VLLM_CI_USE_S3 (also dropped from the compile_factors ignore-set), VLLM_FLASHINFER_ALLREDUCE_FUSION_THRESHOLDS_MB (+ its validator). compile_factors() ignore-set: added VLLM_ENABLE_STARTUP_PLAN, VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS, VLLM_XLA_CACHE_PATH, VLLM_CONFIG_ROOT (vllm-project#47573); removed VLLM_CI_USE_S3. docs/configuration/env_vars.md: kept ours — the VLLM_PORT/Kubernetes warning lives in the docs generator on this branch, and new fields flow through gen_env_vars.py automatically. Not ported: nothing dropped silently; all 11 commits' envs.py deltas are covered above. Verified: no conflict markers; all 12 ported vars import with correct defaults; all 5 deleted vars gone; pre-commit clean on both files; tests/test_envs.py 11/11 pass (incl. test_p2p_side_channel_defaults_and_override). AI assistance was used to resolve this merge. Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Vinay Damodaran <vrdn@hey.com>
4 tasks
4 tasks
edwinlim0919
pushed a commit
to chaeminlim-mb/vllm
that referenced
this pull request
Jul 29, 2026
Signed-off-by: gnovack <novackgm@gmail.com>
itej89
pushed a commit
to itej89/vllm
that referenced
this pull request
Aug 4, 2026
Signed-off-by: gnovack <novackgm@gmail.com> Signed-off-by: Tej Kiran <kiran.tej@amd.com>
aditi-amd
pushed a commit
to aditi-amd/vllm
that referenced
this pull request
Aug 4, 2026
Signed-off-by: gnovack <novackgm@gmail.com> Signed-off-by: root <root@smci355-ccs-aus-m02-09.cs-aus.dcgpu>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
This PR modifies the
VLLM_MOE_SKIP_PADDINGlogic introduced by #46428, removing the separate call totorch.whereand instead passing theis_paddingmask into the topk kernel. The topk kernel then reads fromis_paddingto determine whether to set any giventopk_idsentry to-1.Test Plan
Unit tests
Ran Top-K kernel unit tests after adding new test cases to validate handling of pad tokens
End-to-End benchmarks
Benchmarked deepseek-ai/DeepSeek-V4-Flash on 4xGB200 w/ sonnet dataset, 512/512
Allgather ReduceScatter backend
DeepEP V2 backend
DeepGEMM MegaMoE
Top-K kernel microbenchmarks