Skip to content

[Core][Kernel][SM70] Default validated PP2 TP4 paths - #283

Merged
yangzhuxinyzx merged 13 commits into
mainfrom
agent/v100-dsv4-pp2tp4-nospec-100tps-20260824-165007
Aug 25, 2026
Merged

yangzhuxinyzx merged 13 commits into
mainfrom
agent/v100-dsv4-pp2tp4-nospec-100tps-20260824-165007

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Purpose

Promote the two measured inference-engine wins from the original campaign while removing superseded or unproven experiments.

  • Default-on SM70 block-FP8 QPN8 for exact serialized PP2 x TP4, B1, no-spec operator/tensor contracts. Admission uses device, topology, concurrency, operator role, tensor shape/dtype/layout, and workspace availability only—never model name, checkpoint, model_type, or architecture identity. Missing operators/OOM fall back to TurboMind; VLLM_SM70_FP8_QPN8=0 or VLLM_SM70_FP8_QPN8_PP2_TP4=0 rolls back.
  • Default-on metadata-free PP transfer for the exact replicated FP16 [1,4,4096] B1 schema. The engine validates the model-provided intermediate tensor schema on both stages and rejects DBO, ubatching, speculation, sequence parallelism, non-SM70, and non-CUDA-Graph configurations. VLLM_SM70_PP_STATIC_HIDDEN_TRANSFER=0 rolls back.
  • Strict paired quality artifacts: dataset hashes and evaluation contracts must match, artifacts must be self-consistent, and aggregate HumanEval/LongBench/GSM8K/needle quality must not regress. Individual flips are reported but greedy/token identity is not a gate.

The generic PP profiling repair and compact MXFP4 work were already merged through #295. The short-indexer experiment and unrelated large patch surface are not retained.

Performance and numerical result

  • QPN8: 59.248 -> 64.359 token/s (+8.63%), mean TPOT 16.878 -> 15.538 ms.
  • Static PP on top: 64.359 -> 64.502 token/s (+0.22%), about 0.034 ms/token.
  • Paired GSM8K-64: 63/64 -> 63/64, zero invalid answers; one regression is balanced by one improvement. No greedy identity requirement is imposed.
  • Operator screen: finite outputs, relative L2 <= 6.05e-4, cosine >= 0.9999997, max absolute difference <= 0.00390625. All 365 audited block scales are finite positive powers of two and transform exactly.

Test Plan

  • Exercise default/rollback and exact QPN8 shape/topology/concurrency admission.
  • Exercise default weight preparation and grouped dispatch.
  • Exercise static metadata-free send/receive, worker admission/schema, and self-copy avoidance.
  • Exercise aggregate quality, contract/hash mismatch, malformed artifacts, and directional flips.
  • Run changed-file pre-commit and focused CPU suites; do not repeat broad model E2E.

Test Result

  • Focused combined suite: 47 passed.
  • Changed-file pre-commit: passed, including Ruff, formatting, typos, Markdown lint, mypy, forbidden imports, SPDX, and repository policy hooks.
  • git diff --check: passed.
  • Added-line identity/private-path audit: no findings.
  • Current GPU rerun was not started because GPUs 0-3 host an unrelated live service and GPUs 4-7 are reserved by unrelated processes. No process was interrupted. The existing PR evidence contains the V100 source-integration, CUDA Graph, three-run endpoint, and paired quality results above.

yangzhuxinyzx and others added 3 commits August 25, 2026 00:51
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

Audit requirement from the current review: DeepSeek V4 may name the matched performance workload, but must not become a runtime activation identity. Any implementation should admit optimized routes only through inference-engine contracts such as SM70 capability, PP/TP topology, operator and collective support, tensor shape/dtype/layout, cache state, and CUDA Graph/concurrency conditions—never model name, checkpoint path, model_type, or architecture identity. Keep this PR Draft until source changes plus focused route, correctness/quality, cleanup, and matched pure-decode evidence are present. The current documentation-only scaffold is not merge-ready.

yangzhuxinyzx and others added 6 commits August 25, 2026 02:04
Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

本 PR 已完成拆分审计。可证明安全且与模型身份无关的引擎核心子集,已由 #295 在最新 main 上重新实现、验证并合并:PP profiling 按 semantic hidden_states 解包,以及两个精确张量形状的默认关闭 SM70 MXFP4 compact 路径。

本 PR 剩余实验不合并:QPN8 候选已有成对基线正确/候选错误的质量回归证据;静态 PP 传输仅约 0.034 ms/token 且现有记录已否定该方向;其余大范围实验缺少足够的通用安全收益闭环。保留本分支和证据供后续独立研究,避免把互不相关且存在风险的实验一次性进入 main。

Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx yangzhuxinyzx changed the title [SM70] Optimize DeepSeek V4 PP2 TP4 no-spec decode [Core][Kernel][SM70] Default validated PP2 TP4 paths Aug 25, 2026
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 25, 2026 18:58
@yangzhuxinyzx
yangzhuxinyzx merged commit 4b3d1e0 into main Aug 25, 2026
1 of 2 checks passed
@yangzhuxinyzx
yangzhuxinyzx deleted the agent/v100-dsv4-pp2tp4-nospec-100tps-20260824-165007 branch August 26, 2026 07:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant