Skip to content

[Kernel][SM70] Repair #378 on latest main - #382

Merged
yangzhuxinyzx merged 36 commits into
codex/v100-qwen38-qsa-paged-prefill-20260828-041420from
agent/v100-repair-pr378-qsa-page4-latest-main-20260828-1308
Aug 28, 2026
Merged

yangzhuxinyzx merged 36 commits into
codex/v100-qwen38-qsa-paged-prefill-20260828-041420from
agent/v100-repair-pr378-qsa-page4-latest-main-20260828-1308

Conversation

@yangzhuxinyzx

@yangzhuxinyzx yangzhuxinyzx commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Purpose

Repair #378 on the latest public main while preserving its default-on SM70 QSA page4 XQA acceleration and contributor ancestry.

Changes

  • Preserve original [Kernel][SM70] Accelerate Qwen3.8 QSA paged prefill #378 head ba759b85836fbca326567af8e06f0d434476fba7 as an ancestor.
  • Convert selected four-token QSA blocks into locality-sorted virtual Flash-V100 pages for the exact FP16 Hq6/Hkv1/D256 prefill shape at 4096 or more rows.
  • Keep the accelerated route default-on, with VLLM_SM70_QSA_XQA_PAGE4=0 as an operational escape hatch.
  • Clamp padded/stale positions to the live request length and admit a partial tail only when the expanded QSA indices contain that exact token.
  • Add route-threshold, metadata-forwarding, invalid-request, stale-tail, and physical-tail mapping tests.
  • Record the locked V100 quality/performance, fallback, CUDA Graph, and existing-XQA compatibility evidence.

Test Result

  • Focused CPU contracts before final main sync: 9 passed.
  • Combined QSA route, metadata, and [Bugfix][QSA] Keep compressed block tables in physical-page units #381 physical-page contracts on V100:
    15 passed.
  • Full SM70 extension build from the locked original C++ source: passed with CUDA 12.0 / Torch cu128 for sm_70.
  • Changed-file pre-commit: passed, including Ruff, clang-format, mypy, Markdown, and the accelerator API gate.
  • Python compile: passed.
  • 4095-row boundary: exact Triton fallback, identical with or without forwarded metadata.
  • 4096-row interleaved-KV A/B: 27.8303 -> 8.84736 ms including table/sort (3.1456x); max abs 6.104e-5, relative L2 2.845e-4, cosine 0.99999994, all finite.
  • Nonmonotonic contiguous-page A/B: max abs 3.815e-6, relative L2 3.645e-4, cosine 1.0.
  • Tail counts 1/2/3: relative L2 at most 3.64e-4, cosine at least 0.99999988.
  • Hybrid 784/16 geometry with a nonmonotonic 128-entry page-16 table: max abs
    3.815e-6, relative L2 3.631e-4, cosine 0.99999994; Graph replay is
    bitwise equal to eager output.
  • Page-16 context sweep at 4096 rows remains positive from short to saturated
    selections: 64 tokens 29.225 -> 1.010 ms (28.95x), 512 tokens
    27.063 -> 3.204 ms (8.45x), and 2048 tokens 27.835 -> 11.203 ms
    (2.48x); every arm passes relative L2 below 3.84e-4 and cosine at least
    0.99999994.
  • CUDA Graph: capture passed; two replays are bitwise equal to eager output, hash 9b4c76f8420d6e349dc7d552c72d6f0a861332e7e8e8f62459a1c48f0faf278f.
  • Existing FP16 page-16/page-784 XQA smokes: relative L2 3.03e-5 / 4.10e-5, cosine approximately 1.0.
  • Shared-memory audit: the two-block page4 and pipeline variants remain below V100's 96 KiB limit, so the enlarged page-ID table does not reduce declared occupancy.

Locked candidate

Draft repair for #378. Merge only after GitHub reports this exact head and the final full pre-commit succeeds.

yangzhuxinyzx and others added 30 commits August 28, 2026 11:54
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…n-latest-main-20260828-1205

[Bench][SM70] Repair #375 without disabling QPN8
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…n-latest-main-v2-20260828-1228

[Core][SM70] Keep Qwen3.8 QPN8 default-on after #375 A/B
…ality-audit-20260828-025709

[Bench][SM70] Record Qwen3.8 QPN8 quality/performance A/B
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…-dynamic-latest-main-20260828-1242

[Bench][SM70] Repair #377 without slowing GLM NVFP4
Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
…f-20260828-040813

[Bench][SM70] Add GLM NVFP4 quality and performance screens
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…ctness

[Bugfix][QSA] Keep compressed block tables in physical-page units
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx marked this pull request as ready for review August 28, 2026 05:27
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor Author

审计完成并锁定 8cd30655d13d4d91d83b329924850180a0b72461(tree a6d9af1056de0db77294f2209f86c39e24eb5def)。

  • 已包含最新 main[Bugfix][QSA] Keep compressed block tables in physical-page units #381 的 QSA 物理页修复。
  • 远端完整 pre-commit 通过;组合聚焦测试 15/15 通过。
  • V100 page-16/page-784、连续/交错 KV、尾部 0/1/2/3、4095 回退和 CUDA Graph 捕获/重放均通过。
  • 4096 行 page-16 上,64/512/2048 token 选择分别约加速 28.95×/8.45×/2.48×;所有质量臂 relative L2 小于 3.84e-4、cosine 至少 0.99999994。
  • 路由保持默认开启,同时保留环境变量关闭开关;修复了过期位置、无效请求与虚假尾页边界。

准备按锁定提交合并回 #378

@yangzhuxinyzx
yangzhuxinyzx merged commit 7c4c795 into codex/v100-qwen38-qsa-paged-prefill-20260828-041420 Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants