Repository navigation
[HIP] [BUG fix] Initialize reverse route indices before Opus sorting - #6467
Conversation
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
One backend per PR: PR title tags & labels: |
aiter 9205e105 in nightly_202610131505 picks the scatter-epilog GEMM2 for MiniMax-M3 TP4 MoE at M > 8192, which hits an illegal memory access on unwritten Opus reverse routes (fixed upstream in ROCm/aiter#6467). Cap the TP4 points' max-num-batched-tokens at 8192 so they stay on the reduce rows. nightly_202610131505 中的 aiter 9205e105 会在 M > 8192 时为 MiniMax-M3 TP4 MoE 选择 scatter epilog GEMM2,因 Opus 反向路由未初始化而触发非法内存访问 (上游已在 ROCm/aiter#6467 修复)。将 TP4 点位的 max-num-batched-tokens 限制为 8192,使其使用 reduce 配置。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Motivation
Opus sorting can leave
reverse_sortedentries unwritten when routing repeats an expert with zero weight. The scatter reduction then reads stale allocator contents as sorted row indices, which can cause an illegal GPU memory access.Initialize all reverse route indices to
-1on the current HIP stream before sorting. Valid routes overwrite their entries, while unwritten slots retain the sentinel that scatter reduction skips. Remove the redundant Python-side initialization for EP routing.Validated locally:
torch.compile, non-default stream execution, CUDA Graph replay with changed routing, and existing Opus sorting checks.Technical Details
Test Plan
Test Result
Submission Checklist