[Kernel][SM70] Avoid NVFP4 MoE prefill input expansion - #390
Merged
yangzhuxinyzx merged 2 commits intoAug 28, 2026
Merged
Conversation
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
yangzhuxinyzx
changed the base branch from
codex/v100-qwen38-grouped-page4-prefill-20260828-144759
to
main
August 28, 2026 08:44
yangzhuxinyzx
marked this pull request as ready for review
August 28, 2026 09:00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Stacked on #387. Remove the Qwen3.8 TP4 NVFP4 prefill input-expansion bottleneck without changing routing order, W2, activation, unpermute, decode, or output arithmetic.
The exact E512/K10 W13 route keeps the existing stable expert sort, derives original token-row indices in a lightweight metadata kernel, and uses TurboMind SM70 MatrixLayout.idxs instead of materializing [tokens * 10, 2560] FP16 rows.
Safety
VLLM_SM70_NVFP4_QWEN38_MOE_INDEXED_PREFILL=0restores the old route.=1fails closed when either operator is missing.Test Result
_Cand_moe_Cincremental builds and schema imports pass.15 passed, 5 skipped.git diff --checkpass.6.026752 -> 4.235264 ms(1.423x,1.791488 ms/layer/ranksaved). All routing metadata and W13 output are bitwise equal.5998.65 -> 6507.10 tok/s(+8.47%)5777.43 -> 6241.48 tok/s(+8.03%)5450.92 -> 5871.47 tok/s(+7.71%)6394.74 -> 6970.80 tok/s(+9.01%)Full implementation and benchmark evidence are recorded in
docs/design/sm70_v100_migration_control.md.