Skip to content

test: guard V4 hash inputs across graph replay - #2735

Closed
yhl-amd wants to merge 1 commit into
mainfrom
test/v4-dp-hash-graph-inputs
Closed

yhl-amd wants to merge 1 commit into
mainfrom
test/v4-dp-hash-graph-inputs

Conversation

@yhl-amd

@yhl-amd yhl-amd commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

V4 hash routing in layers 0–2 must use the current DP input IDs. A graph of the inner compiled model can retain the address of an earlier IDs gather while ForCausalLM.forward publishes subsequent gathers at new addresses. Current main routes local rows before the DP collectives and retains the outer ForCausalLM recording unit. That unit publishes the graph-owned current IDs to the hash reader.

Add a GPU regression test using the real decorated V4 wrapper, a small hash reader and fresh caller ID allocations. It changes IDs, alternates two decode buckets and inserts an eager prefill between replays. Retaining earlier caller allocations prevents allocator reuse from accidentally making an incorrect recording unit pass. The test checks exact current token/expert IDs and that decode actually replays.

Validation on exact head 09888e9f0770522159d8b89d82f69c8eeaa45a02, ROCm GPU and pinned AITER 5087367401ca32c6d6bd9d9cc38ceeab744db406:

  • python3 -m pytest -q tests/test_deepseek_v4_hash_graph.py: 1 passed in 3.06 seconds, including exact current-ID checks and the assertion that decode actually replays.
  • A diagnostic negative control removes the outer recording unit and records only the inner hash reader. It reaches the numerical assertion and fails with 64/64 mismatched elements, maximum absolute difference 101. This is an expected correctness failure, not a setup failure; the control is outside this PR.
  • The 12 existing region-graph GPU tests passed.
  • Black, Ruff, git diff --check, non-GPU CI, catalog/docs and offline smoke passed. Model accuracy jobs skipped before approval are not claimed as passed.

No production inference code changes.

The related admission throughput fix is #2727. The old-graph acceptance investigation observed 408/408 incorrect and out-of-vocabulary hash IDs versus 0/408 on #2688, with identical-context decode first diverging at the layer-0 hash FFN. A matched full pair runs historical good ce46f6877 with the same extra ID-buffer allocation and changes only the copy of each fresh gathered ID tensor into that buffer: acceptance is 42.37% → 31.81%, output throughput 3971.74 → 3634.33 tok/s. Both legs exit 0, complete 1280 requests and generate 1,182,150 output tokens using the original 8k1k random workload. The historical high-acceptance baseline contains wrong hash routing; this test protects the corrected current recording boundary.

@yhl-amd
yhl-amd requested review from valarLip and zufayu October 27, 2026 01:18
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

  • ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

  • ✅ Run after the PR is approved and Pre Checkin passes
  • ✅ Run immediately when an approval review is submitted
  • ✅ Can be requested before approval with labels
Label Tests
ci:full Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom Run native ATOM model accuracy tests
ci:vllm Run ATOM vLLM OOT model accuracy tests
ci:sglang Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2735 --add-label <label>

@yhl-amd
yhl-amd marked this pull request as ready for review October 27, 2026 01:35
@yhl-amd yhl-amd closed this Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant