Repository navigation
Conversation
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
V4 hash routing in layers 0–2 must use the current DP input IDs. A graph of the inner compiled model can retain the address of an earlier IDs gather while
ForCausalLM.forwardpublishes subsequent gathers at new addresses. Current main routes local rows before the DP collectives and retains the outerForCausalLMrecording unit. That unit publishes the graph-owned current IDs to the hash reader.Add a GPU regression test using the real decorated V4 wrapper, a small hash reader and fresh caller ID allocations. It changes IDs, alternates two decode buckets and inserts an eager prefill between replays. Retaining earlier caller allocations prevents allocator reuse from accidentally making an incorrect recording unit pass. The test checks exact current token/expert IDs and that decode actually replays.
Validation on exact head
09888e9f0770522159d8b89d82f69c8eeaa45a02, ROCm GPU and pinned AITER5087367401ca32c6d6bd9d9cc38ceeab744db406:python3 -m pytest -q tests/test_deepseek_v4_hash_graph.py: 1 passed in 3.06 seconds, including exact current-ID checks and the assertion that decode actually replays.git diff --check, non-GPU CI, catalog/docs and offline smoke passed. Model accuracy jobs skipped before approval are not claimed as passed.No production inference code changes.
The related admission throughput fix is #2727. The old-graph acceptance investigation observed 408/408 incorrect and out-of-vocabulary hash IDs versus 0/408 on #2688, with identical-context decode first diverging at the layer-0 hash FFN. A matched full pair runs historical good
ce46f6877with the same extra ID-buffer allocation and changes only the copy of each fresh gathered ID tensor into that buffer: acceptance is 42.37% → 31.81%, output throughput 3971.74 → 3634.33 tok/s. Both legs exit 0, complete 1280 requests and generate 1,182,150 output tokens using the original 8k1k random workload. The historical high-acceptance baseline contains wrong hash routing; this test protects the corrected current recording boundary.