[CI] Diagnose SFA PCP accuracy on unchanged upstream main baseline - #16175
zhao-stack wants to merge 1 commit into
Conversation
|
Note Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported. |
|
/e2e tests/e2e/pull_request/four_card/context_parallel/test_accuracy_v2.py::test_dsv3_2_sfa_pcp_model_runner_v2_graph_accuracy |
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
|
/e2e tests/e2e/pull_request/four_card/context_parallel/test_accuracy_v2.py::test_dsv3_2_sfa_pcp_model_runner_v2_graph_accuracy |
|
/e2e tests/e2e/pull_request/four_card/context_parallel/test_accuracy_v2.py::test_dsv3_2_sfa_pcp_model_runner_v2_graph_accuracy |
f5f6914 to
4eebd0b
Compare
|
/e2e tests/e2e/pull_request/four_card/context_parallel/test_deepseek_v4.py::test_deepseek_v4_dsa_pcp_dspark |
1 participant
What this PR does / why we need it?
Diagnostic baseline-only PR to determine whether SFA PCP accuracy fails independently of #16009. Do not merge.
033198cd656eaf5b309a3cf19814c3f3f2310a77, verified through the remote Git ref and GitHub commit API on 2026-09-09 at 20:33 +08:00.f5f69149bb4bfbb37779d1369ec3afe31f997aa9; one signed-off empty commit whose parent is the baseline.37257897c9b8d4ca4d94677b0aa899534df18cf2; the file diff is empty.b2f685834a6456197e7033966fdef52a23f1abcd, releasev0.28.0.Rund compassesinstead ofRund959arkiin two release executions. [Feature][MRV2] sfa support dcp +pcp #15809 restored previously skipped tests; this alone does not establish the numerical root cause.Does this PR introduce any user-facing change?
No. Empty file diff. No production code, golden values, assertions, skips, or version pointers are changed.
How was this patch tested?
诊断 PR16175：三轮均为 main 精确目标通过、release 精确目标断言失败。已独立复现精度失败；本次首先观测的偏差与 PR16009 已报告的第三条 prompt 偏差不同，不能证明相同根因。
基线远端核验时间：2026-09-09 20:33（Asia/Hong_Kong）；结果收集日期：2026-09-09。
033198cd656eaf5b309a3cf19814c3f3f2310a77。f5f69149bb4bfbb37779d1369ec3afe31f997aa9，一个 sign-off 空提交，parent 为上述基线。两者 tree 均为37257897c9b8d4ca4d94677b0aa899534df18cf2。codex/sfa-pcp-baseline-repro-20260909，fork 为 zhao-stack/vllm-ascend；独立 worktree 位于C:/Users/20202/Documents/main2main复盘/worktrees/sfa-pcp-baseline-repro-20260909。三轮双版本证据
精确节点：
tests/e2e/pull_request/four_card/context_parallel/test_accuracy_v2.py::test_dsv3_2_sfa_pcp_model_runner_v2_graph_accuracy。六次均 collected 1 item，以目标 pytest 日志为判据；没有把缓存、coverage 的 skipped 或整个 job 颜色当作目标结论。三轮 main 实际 checkout：
b2f685834a6456197e7033966fdef52a23f1abcd；三轮 release 实际 checkout：2cf0a6915ce544dc493a0990f2ea38d81601128a（v0.28.0）。这里的 main 不是 PR16009 的 a97dacb。评论工作流从 PR head 的两个版本指针读取 main_commit/release_tag，并作为 matrix 的两个版本传入 Selected Tests。三轮调度日志中的 PR_SHA 与 PR_BASE_SHA 均与上述 head/基线一致，六个 job 的 checkout 和 rebase 日志再次确认，rebase 均输出 HEAD is up to date。
具体偏差和结论边界
三次 release 都在第二条 prompt
Hello, my name is Tom, I am首先失败：因此，PR16009 补丁不是“该目标出现精度失败”的必要条件。但本次没有确认用户已报告的第三条 prompt
Rund compasses对Rund959arki偏差：断言在第二条 prompt 中止，日志未展示第三条完整实际输出，既不能称为同一偏差，也不能排除第三条另有偏差。main 3/3 通过，release 0/3 通过；没有观察到 release 概率性通过。三次样本不能证明失败必然发生，也不估计真实失败概率。main 与 release 的 golden 集合本身不同：main 允许第二条
ERIChiretailhallenging变体；main 通过不等于两版本生成值完全一致，也不能仅凭双版本通过/失败差异定位数值根因。#15809 恢复测试只解释为何现在能够观察到失败；此实验没有单独回退/隔离 #15809，不能据此把它认定为数值根因。该基线来自较新的 upstream main，并非 PR16009 的同时点配对对照；没有隔离上游代码、依赖、模型/驱动环境的单一变量，因此不能把偏差归因于某一补丁。
环境对照
六个 job 相同的可见配置：runner label
linux-aarch64-a3-800i-4，NPU 显示 Ascend910，npu-smi/Version25.5.1，镜像 tag9.1.0-a3-ubuntu22.04-py3.12，最终安装日志为 torch2.10.0+cpu、torch-npu2.10.0.post4。镜像仓库为swr.cn-southwest-2.myhuaweicloud.com/base_image/ascend-ci/cann。六次 csrc restore 均命中同一 key：
vllm-ascend-build-v2-ARM64-9.1.0-a3-ubuntu22.04-py3.12-41573a956e821e1609256907d647fc82a7ec576b6aee3a85ccd36f97387001c6。首轮先构建缓存，后两轮缓存协调器无需重复构建；六次目标 job 均使用缓存。linux-aarch64-a3-800i-4-cn12-001-szs24-runner-p8lqj033198cd656eaf5b309a3cf19814c3f3f2310a77linux-aarch64-a3-800i-4-cn12-001-szs24-runner-kr4t5033198cd656eaf5b309a3cf19814c3f3f2310a77linux-aarch64-a3-800i-4-cn12-001-szs24-runner-qxzqs9e1cab90c0a8ec49369eb9aa1e8f387dbe5e4f1dlinux-aarch64-a3-800i-4-cn12-001-szs24-runner-9gpxg9e1cab90c0a8ec49369eb9aa1e8f387dbe5e4f1dlinux-aarch64-a3-800i-4-cn12-001-szs24-runner-ptqmv9e1cab90c0a8ec49369eb9aa1e8f387dbe5e4f1dlinux-aarch64-a3-800i-4-cn12-001-szs24-runner-p9bv79e1cab90c0a8ec49369eb9aa1e8f387dbe5e4f1d第 2/3 轮工作流 SHA 相较第 1 轮变化，但对两 SHA 的
.github完整 diff 为空，实际测试 head 和 CI rebase 基线未变化。runner 实例不同；镜像 digest、模型权重哈希、完整 pip freeze、独立驱动/固件明细未被日志完整记录，不能确认严格环境一致，更不能当作严格概率实验。npu-smi Version仅按日志字段记录，不将其冒充完整固件指纹。