Skip to content

[Kernel][SM70] Prescale DeepSeek V4 FP8 decode - #309

Merged
yangzhuxinyzx merged 8 commits into
mainfrom
agent/v100-dsv4-pp2tp4-fp8-prescaled-decode-20260826
Aug 28, 2026
Merged

yangzhuxinyzx merged 8 commits into
mainfrom
agent/v100-dsv4-pp2tp4-fp8-prescaled-decode-20260826

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

Remove the redundant E4M3 exponent-bias FP16 multiply from the exact SM70 DeepSeek V4 M=1 fused-WQA/WKV path by absorbing its power-of-two factor into UE8M0 scales at load time. The route is opt-in and restricted to the measured PP2 x TP4 no-spec single-request contract.

Test Plan

  • Build the SM70 extension for compute capability 7.0.
  • Run real-weight CUDA Graph A/B/B/A on five V4 FP8 shapes with changing inputs.
  • Run the focused Python routing/admission tests.
  • Run matched full-model GSM8K-64 and three pure-decode repeats before promotion.

Test Result

  • Real layer-0 TP4-rank-0 operator screen: 320/320 changing-input comparisons bitwise equal.
  • Fused WQA/WKV: 63.330 us to 20.577 us (3.078x).
  • Five-shape weighted projection: -1.867 ms/token warm, -1.821 ms/token after cache scrub.
  • pytest -q tests/quantization/test_sm70_fp8_prefill_exact_dense.py -k prescaled: 3 passed.
  • Ruff and git diff --check: passed.
  • Full-model endpoint and pinned dataset gates: pending; keep Draft.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Audited against latest main. The original stacked branch was not replayed wholesale because it was opt-in, reused an older prefill-only operator as its capability signal, and prescaled FP16 values without an actual finite/reversibility gate.

The performance work was repaired in #316: generic SM70 M=1 tensor/runtime admission with no model or checkpoint identity, default-on behavior, explicit rollback, safe fallback for older extensions and unsupported scale ranges, a distinct M=1 operator contract, targeted tests, and full local/remote pre-commit. The recorded 320/320 changing-input comparisons remain bitwise equal and the fused projection remains 3.078x faster.

#316 is merged into main at d9d0da2. Closing #309 only because its audited and repaired functionality is now present on main.

@1CatAI 1CatAI locked and limited conversation to collaborators Aug 26, 2026
@yangzhuxinyzx
yangzhuxinyzx deleted the agent/v100-dsv4-pp2tp4-fp8-prescaled-decode-20260826 branch August 26, 2026 07:55
@yangzhuxinyzx
yangzhuxinyzx restored the agent/v100-dsv4-pp2tp4-fp8-prescaled-decode-20260826 branch August 27, 2026 14:35
@1CatAI 1CatAI unlocked this conversation Aug 27, 2026
@yangzhuxinyzx yangzhuxinyzx reopened this Aug 27, 2026
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

公开开发已恢复:本 PR 已取消归档、解锁并重新打开。为避免把旧基线上的问题直接带入最新 main,现统一作为 Draft 审计项;恢复可见性不代表已经批准合并。 该组涉及 SM70/V100 性能或数值路径,需先与当前主线及 #344#351 去重,再用同条件的聚焦质量和性能证据决定修复后合并。

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
#316 已合入受保护且默认开启的安全 prescale 实现。

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…-20260828-0310

[CI][SM70] Sync #309 to audited latest main
@yangzhuxinyzx
yangzhuxinyzx changed the base branch from agent/v100-dsv4-pp2tp4-mxfp4-direct-order-20260826 to main August 27, 2026 19:21
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 28, 2026 03:25
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

审计结论:安全的 FP8 prescale 实现已由后续 PR 吸收;该分支已通过修复 PR #370 同步到最新 main,最新完整 pre-commit 通过,且锁定提交的源码树与 main 完全一致。按零差异历史归档合并,不改变主线源码。

@yangzhuxinyzx
yangzhuxinyzx merged commit 68368d9 into main Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant