[Test][SM70] Validate QSA allocation repair on the NVFP4 decode lane

[yangzhuxinyzx]

Purpose and dependencies

NVFP4 quality integration and regression, now targeting main directly at the user's merge request. Originally stacked on #507; its tested #506/#507 commits remain ancestors and will enter main with this merge. Synchronized public main: 4366d9d5fe80eeaf79575b51ec36a6a032673df0 (merge2168a13bc9); original numerical baseline base remains755baae1d075ee04fa9096b23fc0225b23589a86.

This is not a competing fix for #494. It reuses Leonccaa's original commit 5fa8a605dab12cc9ee15459d9ac6b88d95c7be3a with original author and cherry-pick provenance preserved. #494 is now already merged into main; its planner repair is no longer a new implementation diff here. This integration supplies the tested NVFP4 decode optimizations from #506/#507, previously missing real NVFP4 quality evidence, actual400/784-token-page regressions, and causal replay tooling.

Root cause / test results

• The actual first QSA layer has identical hidden, Q/K/V, gates, positions and logical selections across requests, but physical KV allocation changes the old hash-emission order.
• Frozen extension replay exactly reproduces both saved actual outputs on every TP rank. 1023/1024 query groups change logical page order.
• Planner-only intervention, keeping frozen attention arithmetic, removes every allocation-dependent output bit. Rebuilt arithmetic with the old plan remains bitwise identical. Restoring the old plan restores its original output.
• One model instance, TP4 GPUs0–3, old / fixed / old / fixed: old natural-prefix EOS forks at zero-based8/28; all six repaired 513-token outputs are identical; reversal restores the fork. All repaired full logits/final hidden are bitwise equal at all12 observed positions. All132 observed control/fixed positions have finite logits, equal final hidden across ranks, and correct actual full-vocabulary argmax.
• Two official-sampling natural-EOS checks pass before/after; three observer-detached requests retain the repaired token hash.
• Focused GPU tests: 59 distinct cases passed (original53 in17.62s; six additional actual native-state784-page cases in3.13s); targeted pre-commit hooks all pass.
• Native FP32 SSM state, second bounded model load: all4ranks confirmfloat32;4deterministic513-token outputs identical and all48observed complete logits bitwise equal.11natural-EOSchecks across9distinct cases pass, including261632-token dispersed retrieval with needles at5/50/95percent. No observers remain during performance measurements; after the long request, three short speed requests still retain the native-state token hash.
• Native-state pure decode97.90513tokens/s,10.21397ms/token;8192-input prefill6970.98tokens/s. Three decode runs97.90434/97.90695/97.90411. Relative to the historicalFP16-state98.965175record this is-1.07%, changingstate+planner together, not isolated planner overhead.

.venv/bin/python -m pytest -q tests/kernels/test_sm70_qsa_page4_plan.py tests/models/qwen4_exp/test_qsa_ops.py
.venv/bin/python benchmarks/kernels/verify_sm70_qsa_nvfp4_relocation.py --capture-dir <actual-captures> --frozen-dso <frozen-extension> --out <result.json>
pre-commit run --files benchmarks/kernels/verify_sm70_qsa_nvfp4_relocation.py tests/kernels/test_sm70_qsa_page4_plan.py docs/design/sm70_qwen38_nvfp4_quality_repair.md

Contract and acceptance limits

Qwen3.8 native NVFP4 experts/TurboMind W4A16; TP4/PP1 V100-SXM2-32GB; FP16 activation/KV, no MTP or prefix cache; max262144, chunk8192, V2 dual FULL_AND_PIECEWISE, hybrid mmap/pinned-UVA PLE. Torch2.10.0+cu128, NVCC12.0.140, driver580.173.02. At the actual model-tested head, production source differed from the frozen speed lane only by reused #494. The subsequent main synchronization preserves tested QSA/HC/router and Flash-V100 computation, while retaining main's independent AWQ/PP/load-allocator changes. It is not a fresh endpoint measurement of updated main.

The causal control deliberately retains the old launcher's explicit FP16 SSM override, though the checkpoint specifies FP32. The second run restores auto→nativeFP32 state and passes the bounded checks above. Native-state attention pages are784tokens (400in the F16control), both covered by logical-plan regression. It is not safe to declare every numerical/kernel gate complete from allocation stability and these bounded task checks. Known GDN/W13 rounding-reference gaps remain separately tracked. No broad dataset quality certificate is claimed.

The observer-detached FP16 control lacked request statistics: its6.35s total request time includes prefill and must not be labeled pure decode or used to assert98.965tok/s is preserved. Native-state tests enable separated metrics and supply the97.90513result above. Both model loads and the follow-up tiny operator test exited normally; no resident GPU service or backgroundqueue remains from this task.

See docs/design/sm70_qwen38_nvfp4_quality_repair.md for source/binary hashes, measured controls and limitations. Large tensors, weights, local paths and build outputs are not committed. Local artifact families: operator/real_replay.json, operator/pytest{,_page784}.xml, model_f16/{result,analysis}.json, model_native_f32/{result,analysis}.json, state_contract_comparison.json, and exact retained quality-input manifests.

Merge-time validation

-27CPU HC/QSA/router dispatch tests pass on synchronized main; no GPU/model startup.
-The old CI failure was diagnosed from job101447098235: clang-format, Markdown spacing, a short commit-SHA typo false positive, and deprecated device/synchronization/cache APIs in three standalone verification scripts. These are fixed without changing production arithmetic or disabling checks.
-All targeted pre-commit hooks pass; wait for the updated GitHub check before merging. No admin/protection bypass or direct main push.

Review

AI assistance: OpenAI Codex. Original #494 author attribution is preserved; new commits are DCO signed. The user explicitly requested merge on2026-09-06 after the passing bounded quality results and remaining GDN/W13 limitations were reported. This records merge authorization, not a claim that the user independently reran the tests or that every numerical audit is complete. Do not promote the old speed record as a universal no-token-reversal certificate.

[yangzhuxinyzx]

#494 已合入 main，原作者修复和归属已保留。本 PR 继续 open：待 #506/#507 基线链处理后，保留本 PR 独有的回归/证据增量。已看到新 head 911b6ba，本轮未把它宣称为已完整复审或已接受纯 decode 性能；请继续区分 FP16 状态因果诊断与原生 FP32 状态验收。