[Kernel][SM70] Prescale DeepSeek V4 FP8 decode - #309
Conversation
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
|
Audited against latest main. The original stacked branch was not replayed wholesale because it was opt-in, reused an older prefill-only operator as its capability signal, and prescaled FP16 values without an actual finite/reversibility gate. The performance work was repaired in #316: generic SM70 M=1 tensor/runtime admission with no model or checkpoint identity, default-on behavior, explicit rollback, safe fallback for older extensions and unsupported scale ranges, a distinct M=1 operator contract, targeted tests, and full local/remote pre-commit. The recorded 320/320 changing-input comparisons remain bitwise equal and the fused projection remains 3.078x faster. #316 is merged into main at d9d0da2. Closing #309 only because its audited and repaired functionality is now present on main. |
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
|
审计结论:安全的 FP8 prescale 实现已由后续 PR 吸收;该分支已通过修复 PR #370 同步到最新 main,最新完整 pre-commit 通过,且锁定提交的源码树与 main 完全一致。按零差异历史归档合并,不改变主线源码。 |
Purpose
Remove the redundant E4M3 exponent-bias FP16 multiply from the exact SM70 DeepSeek V4 M=1 fused-WQA/WKV path by absorbing its power-of-two factor into UE8M0 scales at load time. The route is opt-in and restricted to the measured PP2 x TP4 no-spec single-request contract.
Test Plan
Test Result
pytest -q tests/quantization/test_sm70_fp8_prefill_exact_dense.py -k prescaled: 3 passed.git diff --check: passed.