Conversation
Initialize short-row top-k destinations to -1 and clamp expanded pool IDs to the request completed-pool range before sparse-attention gathers. Assisted-by: OpenAI Codex Signed-off-by: Luca Motz <321921718+lucamotz@users.noreply.github.com>
lucamotz
marked this pull request as ready for review
August 28, 2026 07:20
Author
|
Fresh CUDA regression results from the PR head ( The exact PR-head test and patched Triton module were mounted read-only into disposable containers based on the qualified GLM-5.3 vLLM image: Result on each GB10: Test file SHA-256: |
2 tasks
Author
|
Now that vllm-project#53906 is merged, I have upstreamed this fix against vllm-project/vllm:main as vllm-project#55201. Closing this dependent fork PR in favor of the upstream PR. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
GLM-5.3 kpool selection can produce fewer than the requested
select_kentries for short rows. The current destination uses
torch.empty, so slotsnot written by the top-k backend can retain arbitrary positive values. Pool
expansion then treats any non-negative value as a valid pool ID and can turn an
uninitialized value into an out-of-range sparse-attention gather.
This change adds two defensive layers:
-1, preserving the existinginvalid-index convention for every unwritten slot;
[0, completed_pool_count)during expansion, so amalformed positive ID cannot become a token index outside the request's
completed-pool range.
The regression supplies valid, out-of-range-positive, and negative pool IDs
for a short row and verifies that only the completed pools plus the incomplete
tail are expanded.
This is a focused dependent change for
vllm-project#53906. It targets that PR's source
branch because the GLM kpool implementation is not present on upstream
main.It is independent of #7, which fixes graph
metadata lifetime in different files.
Test Plan
Test Result
glm-releasebase142062f13d16bed254b5d97cc3d371fbd4f7790a.git diff --check: passed.SPDX, forbidden-import, CUDA-API, and repository-specific validators.
capability 12.1):
1 passed, 30 deselectedon each device. The exactPR-head test and patched Triton module SHA-256 receipts are recorded in the
PR conversation.
No private hostnames, paths, credentials, environment files, or raw deployment
logs are included.
This change and PR description were prepared with OpenAI Codex assistance. The
human submitter reviewed the diff and is responsible for the implementation
and reported evidence.
Essential Elements of an Effective PR Description Checklist