Conversation
zihaomu
requested review from
Qiaolin-Yu,
Ying1123,
hnyls2002 and
merrymercy
as code owners
June 23, 2026 06:36
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Contributor
|
I did an independent CPU-side validation of this PR head ( Checks run: git diff --check 067987d86^..067987d86
python3 -m py_compile python/sglang/srt/speculative/eagle_utils.py test/registered/unit/spec/test_eagle_verify_tp_sync.py
docker run --rm -v $PWD:/workspace -w /workspace lmsysorg/sglang:dev \
bash -lc 'PYTHONPATH=python python3 -m pytest test/registered/unit/spec/test_eagle_verify_tp_sync.py -q'Result: the focused test passed ( Also checked CI status: lint passed; the current red PR-test entries are gate/skip finish jobs rather than a focused lint/test failure. |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
predict,accept_index, andnum_correct_draftsaligned before later running-batch updates and collectives.Fixes #28815.
Motivation
On ROCm, EAGLE/NEXTN verify enters the greedy path via
_is_hip. The sampling path already broadcasts verify outputs across TP ranks, but the greedy/HIP path returned local verify results. If ranks accept different numbers of draft tokens, later shape-dependent collectives can diverge and hang.The fix treats TP synchronization as a verify-output invariant rather than a sampling-only behavior.
Testing
python3 -m pytest test/registered/unit/spec/test_eagle_verify_tp_sync.py -qpre-commit run --files python/sglang/srt/speculative/eagle_utils.py test/registered/unit/spec/test_eagle_verify_tp_sync.pypython3 repro_issue_28815_tp_divergence.py --mode repo-fixed --master-port 29622 --join-timeout-seconds 30CI States
Latest PR Test (Base): ❌ Run #28007911393
Latest PR Test (Extra): ❌ Run #28007911148