Repository navigation
[Test] Handle tied top-k indices in graph-pool logprob regression - #40661
Merged
Merged
Conversation
Collaborator
Author
|
/rerun-test test_graph_pool_borrow.py |
Contributor
|
Results for 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
test_input_logprobs_survive_replay_with_growing_chunkscan fail when random float32 logprobs tie. The implementation computestopk(max_k)and truncates each sequence to its requestedk, while the test computestopk(k)independently and requires identical token IDs. PyTorch does not guarantee which tied indicestopkreturns.This caused the base-b failure on #40638. It also reproduces without that PR on public main commit
00986c81be687d82e66651fb3ce9a096070b06e6: seed 44 fails atrows=2600, choosing token 1783 versus 959 at the same logprob,-5.749078750610352. The failed test and its logprob, pool, and result-processing code remain identical on main ate332e1b84e643d02d6223542941030c03717f62c.Modifications
Validate returned token IDs by gathering their expected scores and comparing those scores exactly against the expected top-k values. Require distinct IDs within each row and explicitly exercise tied scores in both sequences. All graph replay, pinned-copy, sampled-logit, and token-logprob checks remain in place. Only the existing test changes.
Accuracy Tests
On one H200 with PyTorch 2.13.0+cu130, using the public-main baseline above:
test/registered/unit/model_executor/runner_utils/test_graph_pool_borrow.py: 18 tests passed.No model output or inference behavior changes.
Speed Tests and Profiling
Not applicable; test-only change.
CI States
Latest PR Test (Base): ❌ Run #35677282078
Latest PR Test (Extra): ❌ Run #35677281936
Latest PR Test (AMD ROCm 10): ❌ Run #35677282031