[Core][SM70] Promote DFlash2 NVFP4 prefill route - #417
Merged
yangzhuxinyzx merged 2 commits intoAug 29, 2026
Merged
yangzhuxinyzx merged 2 commits into
yangzhuxinyzx merged 2 commits into
Conversation
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Contributor
Author
|
Four-V100 production-service gate passed on head a09e332: health/models HTTP 200; automatic QPN2 M<=8 and QPN2-packed M>=1024 routes hit with both env gates unset; D256/direct-paged/FP8-bridge/context-only prefill routes hit; qwen3_coder tool call returned structured finish_reason=tool_calls with valid get_weather({"city":"广州"}); repeated ~8K prefix reduced TTFB 2.789s -> 0.591s with 9,888 local prefix-cache hit tokens. The public tunnel was also verified. Old service remains disabled as rollback. |
yangzhuxinyzx
marked this pull request as ready for review
August 29, 2026 05:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Automatically select the existing lossless QPN2-packed bounded prefill path for the exact Qwen3.8-27B NVFP4 DFlash2 TP4 single-request contract. The default gate requires
method=dflash, q7, checkpoint selector top-K 16, PP1/TP4, B1, no DBO/microbatching, and the exact dense layer shapes. Explicit environment overrides remain hard rollbacks; DFlash1, target-only, other speculative methods, TP sizes, and concurrent service retain existing routing.This PR changes dispatch only. It does not change kernel arithmetic, DFlash2 proposal/verification, sampling, attention, prefix-cache behavior, or the merged tool/structured-output fixes.
BASE_SHA: 62ad1e0
Test Plan
Test Result
git diff --check: passed.54363ddee68f4a5db81c9d37e5fb738d28f5b67dc7f725ad7333172b1ea157da.Raw artifacts and exact hashes are recorded in
docs/design/sm70_dflash2_nvfp4_prefill_promotion.md.