Skip to content

[Core][SM70] Preserve DFlash2 decode across NVFP4 prefill - #420

Merged
yangzhuxinyzx merged 3 commits into
mainfrom
agent/v100-dflash2-release-audit-20260829-052327
Aug 29, 2026
Merged

yangzhuxinyzx merged 3 commits into
mainfrom
agent/v100-dflash2-release-audit-20260829-052327

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

  • keep the accepted SM70 NVFP4 QPN2 decode graph opaque while adding the QPN2-packed large-M prefill route
  • replace the Python M-dependent branch and persistent 85 MiB workspace with one C++ dispatcher and an operator-local ephemeral workspace
  • source-default the quality-audited 18 ms NVFP4 DFlash2 TP4/B1 practical path, preserving every explicit override and keeping candidate-order disabled

Test Plan

  • focused CPU contract and dispatch tests
  • exact V100 normal/gated M8/M16 operator comparisons
  • restart the four-V100 256K practical API with QPN2/prefill/DFlash2 fastpath booleans removed from the launch file
  • measure three 512-token single-request decode runs, cold 32K prefill, and post-prefill decode
  • smoke tool calls and strict JSON schema output

Test Result

  • 7 passed focused pytest; ruff, compileall, and git diff --check pass
  • V100 operator comparison: max_abs=0 for normal/gated and explicit/ephemeral workspace cases
  • CUDA Graph pool: 0.13 GiB
  • complete DFlash2 round: 17.4035 / 17.3569 / 17.3697 ms, mean 17.3767 ms
  • cold 32K pure prefill: 4039.49 tok/s
  • post-32K complete round: 17.3640 ms
  • tool calling: 3/3; strict JSON schema: 1/1

Detailed evidence is recorded in docs/design/sm70_dflash2_nvfp4_prefill_promotion.md.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…lease-audit-20260829-052327

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>

# Conflicts:
#	docs/design/sm70_v100_migration_control.md
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 29, 2026 07:54
@yangzhuxinyzx
yangzhuxinyzx merged commit e315dd4 into main Aug 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant