Skip to content

[Kernel][SM70] Add opt-in DeepSeek V4 FP13 GEMV - #322

Merged
yangzhuxinyzx merged 10 commits into
mainfrom
agent/v100-dsv4-pp2tp4-fp13-gemv-20260826
Aug 28, 2026
Merged

yangzhuxinyzx merged 10 commits into
mainfrom
agent/v100-dsv4-pp2tp4-fp13-gemv-20260826

Conversation

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Purpose

Add an opt-in packed-FP13 single-token GEMV for the fixed DeepSeek V4 SM70 router/compressor/indexer projections. Packing happens once after loading and retains the original FP16 GEMV fallback.

Test Plan

  • CPU bit-level pack/unpack audit
  • SM70 offline Triton compile and SASS resource audit
  • Exclusive-V100 CUDA Graph correctness and four-stream overlap screen
  • Matched PP2 x TP4 endpoint only if the overlap screen wins
  • Pinned aggregate quality gates before acceptance

Test Result

  • ruff check: passed
  • py_compile: passed
  • targeted CPU pytest: 3 passed
  • offline SM70 kernel: 29 registers, 227 static instructions, 17 global loads, 32 FFMA, 8 shuffles, 2 barriers
  • real-weight CPU audit: 99.87%--99.98% exact FP16 values; relative weight L2 1.18e-8--3.46e-7
  • GPU overlap/endpoint gates: pending shared-GPU availability

Draft: do not merge until the queued exclusive-GPU and quality gates complete.

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
yangzhuxinyzx added a commit that referenced this pull request Aug 26, 2026
Repair #322 on current main, retaining only the measured FP13 kernel and adding tensor-value, CUDA-device, shape, dtype, layout, memory, quality, and rollback safeguards. The production path defaults on without model/checkpoint identity admission.

Original-commit: 165ec09
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Superseded by merged PR #325 (8700861e). The replacement replayed only this PR’s final FP13 kernel onto latest main, preserved the GPU-screened kernel byte-for-byte, added same-CUDA-device and tensor-value safety gates, audited all 188 eligible checkpoint tensors, quantified PP2 memory overhead, defaulted the measured path with explicit =0 rollback, and passed local plus remote static gates. The eight stale stacked commits were intentionally not replayed.

@1CatAI 1CatAI locked and limited conversation to collaborators Aug 26, 2026
@yangzhuxinyzx
yangzhuxinyzx deleted the agent/v100-dsv4-pp2tp4-fp13-gemv-20260826 branch August 26, 2026 07:55
@yangzhuxinyzx
yangzhuxinyzx restored the agent/v100-dsv4-pp2tp4-fp13-gemv-20260826 branch August 27, 2026 14:40
@1CatAI 1CatAI unlocked this conversation Aug 27, 2026
@yangzhuxinyzx yangzhuxinyzx reopened this Aug 27, 2026
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

公开开发已恢复:本 PR 已取消归档、解锁并重新打开。为避免把旧基线上的问题直接带入最新 main,现统一作为 Draft 审计项;恢复可见性不代表已经批准合并。 该组涉及 SM70/V100 性能或数值路径,需先与当前主线及 #344#351 去重,再用同条件的聚焦质量和性能证据决定修复后合并。

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
#325 已按最新主线重建并默认启用经过张量值门控的 FP13 实现。

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…-20260828-0310

[CI][SM70] Sync #322 to audited latest main
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 28, 2026 03:26
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

审计结论:默认开启且带保护的 FP13 路径已由后续安全实现吸收;该分支已通过修复 PR #372 同步到最新 main,最新完整 pre-commit 通过,且源码树与 main 完全一致。按零差异历史归档合并。

@yangzhuxinyzx
yangzhuxinyzx merged commit 050e2ba into main Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant