[Core][Kernel][SM70] Default validated PP2 TP4 paths - #283
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
|
Audit requirement from the current review: DeepSeek V4 may name the matched performance workload, but must not become a runtime activation identity. Any implementation should admit optimized routes only through inference-engine contracts such as SM70 capability, PP/TP topology, operator and collective support, tensor shape/dtype/layout, cache state, and CUDA Graph/concurrency conditions—never model name, checkpoint path, |
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
|
本 PR 已完成拆分审计。可证明安全且与模型身份无关的引擎核心子集,已由 #295 在最新 main 上重新实现、验证并合并:PP profiling 按 semantic hidden_states 解包,以及两个精确张量形状的默认关闭 SM70 MXFP4 compact 路径。 本 PR 剩余实验不合并:QPN8 候选已有成对基线正确/候选错误的质量回归证据;静态 PP 传输仅约 0.034 ms/token 且现有记录已否定该方向;其余大范围实验缺少足够的通用安全收益闭环。保留本分支和证据供后续独立研究,避免把互不相关且存在风险的实验一次性进入 main。 |
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Purpose
Promote the two measured inference-engine wins from the original campaign while removing superseded or unproven experiments.
model_type, or architecture identity. Missing operators/OOM fall back to TurboMind;VLLM_SM70_FP8_QPN8=0orVLLM_SM70_FP8_QPN8_PP2_TP4=0rolls back.[1,4,4096]B1 schema. The engine validates the model-provided intermediate tensor schema on both stages and rejects DBO, ubatching, speculation, sequence parallelism, non-SM70, and non-CUDA-Graph configurations.VLLM_SM70_PP_STATIC_HIDDEN_TRANSFER=0rolls back.The generic PP profiling repair and compact MXFP4 work were already merged through #295. The short-indexer experiment and unrelated large patch surface are not retained.
Performance and numerical result
+8.63%), mean TPOT 16.878 -> 15.538 ms.+0.22%), about 0.034 ms/token.Test Plan
Test Result
git diff --check: passed.